Uber Drivers Forum banner
Status
Not open for further replies.

Make Your Own Fast Amazon Flex Block Grabber - Step by Step

Flex 
452K views 1K replies 155 participants last post by  JDWhit_ 
#1 · (Edited)
The following steps are for you to make your own free fast flex block grabber. The scripts are provided for free in this thread and run on a vps. The only cost involved should be for the vps that you decide to get.

Step 1 to Writing your own Server Based Flex Block Grabber:

Signup for a linux based vps with the following specifications:

- able to load a recent ubuntu or similar distribution - (instructions assume Ubuntu 16.04)

- these instructions are based mainly for an iPhone running on wifi; but also work with Android. In some parts of the instructions, follow either the iPhone or Android section. Some things may look or output differently on Android but it should all work the same. Just keep going through the process.

- no less than 1 gb ram; a little more is better to help with caching and memory to run multiple parallel instances of the grabber. You can run about 6 at the same time which can grab at about the combined rate of 10-20 attempts per second.

- ideally the ability to share 4 cpus. This script is CPU intensive more than anything else - do not skimp on the CPU power;

- disk space is not really that important but try to get ssd based so the access time is fast;

- shop for a vps that has low latency (ping times) to the amazon flex server that the app communicates with:

flex-capacity-na
amazon
com

Try for 10 ms or less. For example a good ping response:

icmp_seq=1 ttl=234 time=7.64 ms

Quick Links to Other Steps:

Skill Requirements

Step 2 - Install the Scripts and Proxy Server

Step 3 - Start the Proxy Server and Test It

Step 4 - Watch and Extract the Proxy Output


Step 5 - Select Your Warehouse

Step 6 - Start Grabbing

Step 7 - Notifications

Step 8 - Time Delay Filter Update

Step 9 - Additional Filters
(Grab only Specific Warehouse and Block Size Examples)


Step 10 - Modify Notifications to Show Date and Time of Block
 
See less See more
#2 ·
Tired of getting beat by the professional block grabber resellers?

Are your complaints to Amazon about it falling on deaf ears?

Periodically I will discuss, at no charge, each step for you to make your own.

Step 1 to Writing your own Server Based Flex Block Grabber:

Signup for a linux based vps with the following specifications:

- able to load a recent ubuntu or similar distribution;

- no less than 1 gb ram; a little more is better to help with caching and memory to run multiple parallel instances of the grabber. You can run about 6 at the same time which can grab at about the combined rate of 10-20 attempts per second.

- ideally the ability to share 4 cpus. This script is CPU intensive more than anything else - do not skimp on the CPU power;

- disk space is not really that important but try to get ssd based so the access time is fast;

- shop for a vps that has low latency (ping times) to the amazon flex server that the app communicates with:

flex-capacity-na
amazon
com

Try for 10 ms or less. For example a good ping response:

icmp_seq=1 ttl=234 time=7.64 ms
Isn't this kind of similar to hacking? Rather than a auto accept?
 
#10 · (Edited)
No. The next step involves setting up a proxy server on the vps. This is part of the accelerator between the flex app on your phone and amazon. It's purpose will be to periodically read your temporary access key that lasts about 30 minutes at a time. This allows the scripts to then accelerate your access to Amazon.

The software for that step, when we get to it, is called mitmproxy. We will begin that step by just starting your own proxy server that works similar to many corporations and remote vpn access providers.
 
#7 ·
Interesting post. But this will go beyond what most people will be able to do.
Basmati maybe....the rest of us.....eh.....I dunno???

Here's VPS plans from godaddy:
https://www.godaddy.com/hosting/vps-hosting

Probably the cheapest you'll be able to find.

I look forward to hearing more as I have some experience setting up hosting servers etc but this sounds a little beyond
most capabilities.
 
#24 ·
Interesting post. But this will go beyond what most people will be able to do.
Basmati maybe....the rest of us.....eh.....I dunno???

Here's VPS plans from godaddy:
https://www.godaddy.com/hosting/vps-hosting

Probably the cheapest you'll be able to find.

I look forward to hearing more as I have some experience setting up hosting servers etc but this sounds a little beyond
most capabilities.
Cheaper than godaddy...at least the first year...

https://www.1and1.com/vps-hosting
 
#8 ·
I do not understand the point of this.

This is a competitive job. Whatever advantage you have will work as long as only YOU have it. Posting a method publicly helps noone.

The exact same thing will happen as with frep. If you are indeed able to get blocks this way, your advantage will be gone tomorrow.
 
#9 · (Edited)
Basic Skill Requirements

The skills required to install this are not overly complex. Anyone who can setup existing touch bots already has the aptitude to learn a little about Linux. If you're still tapping all day, stop. Use that tapping time to learn how to do this instead. It's also a good skill set to have for computer jobs.

If all this is too much, find someone who knows enough about Linux and can help. You can also ask for bids on freelancing sites for help. They may even know how to do this already.

Here are some skill areas you will need:

1) Basic getting around the command line. Copy files. Change directories. This is not that hard. Like cp a b copies file a to file b. mv a b moves or renames. Look up YouTube and do some testing on your VPS for example:



2) Know how to login to the vps remotely with ssh.

You will also later use this to start, stop, and check on the status of your grabber script.

For now use an SSH client on the desktop or the virtual console provided by your VPS host.

At the end, my tutorial will eventually use the app called Termius on the play and Apple Store. This is only when going live. It will allow simple one button access to start, stop, and check your grabber. In the field (be very near the warehouse), you can do everything from one phone and a few clicks.

I also use the Flex app on an iPhone. It may work on android also, but I haven't tested it yet.

3) Learn a simple file editor in Linux. Check YouTube for some videos. You could transfer files back and forth and edit on a PC but it's better to learn a file editor.

4) Get a brief overview of how cron works. It's just a file that starts scheduled commands.



The better question is, if the people who also developed this on their own kept quiet and used it only for their own use, how has it become a business of selling to already struggling drivers for $100 a week? There's no reason I should even be hearing about this. And I've even seen forum posts here and elsewhere and private messages trying to sell the access to me and others as well. That's not keeping a competitive advantage quiet for one's own use by any means.

Grabbing blocks has become a waste of time for me because of Amazon lowering the total pay via the tip allocation charade and releasing messed up block times that prevent getting many stops.

This is a public service to help those who still may find some profit in it and are being taken advantage of. Ideally, even better, Amazon should put a stop to this. They obviously are great at logistics and there has to be a better way than this duke em out, brute force, rip em off, Wild West approach.
 
#14 · (Edited)
Android sort of, iPhone no - it doesn't say that once you install the certificate for your proxy server. Even in android it's a just a pinned notification. I'm just providing instructions for iphone anyway.

Screenshot of proxy related profiles installed on iphone allowing fishing on the vps proxy over both cellular and wifi:

Font Screenshot Multimedia Number Logo

In any case, it's the same issue with someone else doing it. At least this way, the network doing the monitoring is the vps under your control.

If you want a server to do it fast this is how it's done. Otherwise tap away or do screen macros. If that's working, no need for all this extra work anyway. Optionally, you can disable using the proxy setting when not fishing for blocks and/or just use a cheap dedicated phone only for that purpose.

Older model iPhones or the SE are about $50 now refurb or $100 new. Use on a cheap carrier or use via wifi to an existing hotspot phone or network. Or adapt the instructions to work for android.
 
#15 ·
If the drivers are stupid enough to give up $4 per block hour to buy the block, then you can't help it. If you're gonna do that, then go work at McDonald's because it pays more. I still don't believe it until I see it. If it's actually happening, I'd like to know what kind of drivers are actually paying?
 
#18 ·
And anyone who can follow this, should probably be able to find something that pays at least 2-3x more than this gig.

I agree. I might do that again one day. For now, I'm semi retired, don't need to work for the most part, have various medical problems, and receive large amounts of incentives and government subsides (about $40k per year's worth) which disappear if I make too much.

As it may have been said: I can't afford to work.
 
#19 ·
Well that explains it.

Maybe Amazon will pay you off to stop this. But then again, I'd bet that less than one % will actually go this far to get a block.
 
  • Like
Reactions: KekeLo
#20 · (Edited)
Step 2 - Install the Scripts and Proxy Server on the VPS

Now that you have the VPS, it's time to install the scripts and proxy server on it.

The proxy server functions to receive traffic from one source, and forward it to another. Companies usually do this for security purposes and to easily allow controlled access to the Internet. In our case, we will need the proxy server for one reason: to capture the temporary access token generated by the Amazon Flex App.

In a later step, we will have your phone point to this proxy server. Another script will then use this temporary access token to check specific web pages for what blocks are available and accept them.

Here are the details of this step:

a) Download the two files attached to this post: proxy.tgz.zip and scripts.tgz.zip and put them in your VPS /home directory (using sftp for example).

They aren't actually zip files but .tar.gz/.tgz format - they have been named .zip so they can be attached to this post. So don't try to use the zip command on them.

b) On your VPS command line, go to the /home directory where you put the two script files.
Rename to the correct extension and extract the zip files as follows:
Code:
/home# mv proxy.tgz.zip proxy.tgz
/home# mv scripts.tgz.zip scripts.tgz
/home# tar xzf proxy.tgz
/home# tar xzf scripts.tgz
This has now extracted all the files and directory structures needed to run the scripts.
(Permissions assume running as root on Ubuntu. If not, change ownership of files.)

c) On your VPS command line, install the proxy server (all instructions assume Ubuntu):
apt install mitmproxy

also make sure your time zone is set correctly:
dpkg-reconfigure tzdata

The above steps installed the scripts and created your proxy server.

You can start it and restart it by using the command:
/home/proxy/doit.bat

The proxy is listening on the assigned IP of the VPS on port 8080.
It will continue running even if you disconnect from the VPS.
It will not auto restart if the VPS restarts.

Note: You may want to change the default port from 8080 to something else to avoid people using your proxy server. If so change it by changing the -p xxxxx (whatever the new port is) to the mitm dump command in the doit.bat file. You will also need to change it elsewhere in these instructions instead of using 8080. Make sure any firewalls have this port open.

Stop it when you are not actively using it since other people could potentially connect to it as an open proxy:
/home/proxy/stopit.bat

That's it for now. Make sure the proxy server is stopped.

The next step will be to setup your smart phone to use the proxy server. This will involve starting the proxy, setting the proxy settings for a simple wifi network, and installing the certificate.
 

Attachments

#102 ·
The above steps installed the scripts and created your proxy server.

You can start it and restart it by using the command:
/home/proxy/doit.bat

The proxy is listening on the assigned IP of the VPS on port 8080.
It will continue running even if you disconnect from the VPS.
It will not auto restart if the VPS restarts.
I'm stuck at this step. I connected to the VPS server and ran the command ./doit.bat from the proxy directory and I keep getting "screen: no process found"

anybody having this issue ?

root@ubuntu:~/home/proxy# ls
amazon-token.txt flex-id.txt proxy.last session-token.txt
doit.bat getparms.bat proxy.out stopit.bat
root@ubuntu:~/home/proxy# ./doit.bat
screen: no process found
Purple Handwriting Violet Rectangle Font

root@ubuntu:~/home/proxy#
 
#22 ·
Tired of getting beat by or paying the professional block grabber resellers?

Are your complaints to Amazon about it falling on deaf ears?

Periodically I will discuss, at no charge, each step for you to make your own the same way they make theirs.

Step 1 to Writing your own Server Based Flex Block Grabber:

Signup for a linux based vps with the following specifications:

- able to load a recent ubuntu or similar distribution;

- no less than 1 gb ram; a little more is better to help with caching and memory to run multiple parallel instances of the grabber. You can run about 6 at the same time which can grab at about the combined rate of 10-20 attempts per second.

- ideally the ability to share 4 cpus. This script is CPU intensive more than anything else - do not skimp on the CPU power;

- disk space is not really that important but try to get ssd based so the access time is fast;

- shop for a vps that has low latency (ping times) to the amazon flex server that the app communicates with:

flex-capacity-na
amazon
com

Try for 10 ms or less. For example a good ping response:

icmp_seq=1 ttl=234 time=7.64 ms

Quick Links to Other Steps:

Skill Requirements

Step 2 - Install the Proxy Server
A couple of questions:

1. Is this method able to differentiate among offers from multiple warehouses and can it filter to accept blocks only from your desired warehouse? Let's say I only want to deliver prime now packages at my desired warehouse and deny all restaurant blocks and warehouse pick ups from different areas surrounding me. Is this possible?

2. Can it get around the 2-step verification code when logging into your account (where it asks you to submit the code sent to your email)?

3. How do you check the status of your calendar in real-time? Is it through the app as usual?

4. Does this method bypass the error notification received when Amazon detects multiple logins?

Thanks
 
#23 ·
As it is written, it just takes whatever is offered. It wouldn't take too much to modify it for only selected warehouses. But, I only plan to post what I have already done. It worked great. I didn't get banned or anything. It's just the block pay is now too low for it to be worth anything to me anymore. But I understand everyone is in a different situation. Others can take it from there and modify it if they want.

I don't have any experience with the 2 step verification. But, the script does not login. It uses a token from an existing login which is why the proxy server is needed. So it may still work.

Check the status the same as always. The app on your phone runs as normal. You'll just see blocks fill into it. The script can send email or text notifications when it detects a block available. It may not have been successful but it will notify anyway.

As in the other question, the script doesn't login. It uses an existing login session. I didn't receive any such errors when I used it.

Overall, the script is not a refined application for all types of filters and situations. It's just enough to get the job done. That was all I needed. Others are welcome to modify it.
 
#25 · (Edited)
Step 3 - Start the Proxy Server and Test It

Now, we will start the proxy server and get it to work on our phone.

a) Login in to the VPS.
Then do this on the VPS:

cd /home/proxy

Type this line (only this once before you start it for the first time - required for it to work right in the background):
LANG=en_GB.UTF-8;export LANG

Run this command:
./doit.bat

Now the proxy server is running on your VPS.

If it says screen: no process found, that's normal - just keep going. That just means there wasn't an existing proxy to stop.

When you want to stop using it, run ./stopit.bat

b) Now do this on your phone to set it to use the proxy server. You can set the proxy to off when you are done or if you have problems with the phone afterward. For now, just connect to a wifi network not cellular data. That's something different to discuss later.

--------------------
For iPhone:
Settings - Wifi - Select the (i) on the far right of the network you are connected to.
Go down to the bottom, where it says HTTP Proxy. Select Manual.
In the Server field, type the public IP that was assigned to your VPS.
In the Port field, type 8080. Then just click on the very top left of the screen where it says < Wifi
(When you need to turn this off, go back and select Off for HTTP Proxy).
--------------------

--------------------
For Android:
(Based on Android 6 - Marshmallow)
Go to your wifi network settings. Click advanced. Click manual. Set the proxy host name to the ip of your vps. Set the port to 8080 or what you set it at. Then connect.
--------------------

When you're not grabbing, make sure to turn off the proxy on the VPS and remove it from your wifi settings. Otherwise it can cause problems with other apps and sites.

c) On your phone you will now need to install the certificate that will allow the script on your proxy server to use secure web sites. This is a one time thing. To undo it, you can delete the certificate. You can keep the certificate installed even when the proxy is not running.

On mobile Safari on the iPhone or Chrome with Android, go to the web site: mitm.it

--------------------
For iPhone:
Select Apple, Allow, Install, Your Phone Passcode, Install, Install, and Done.
It will say profile installed.

You will need to now trust that certificate on your iPhone.
Go to Settings, General, and then About.
Select Certificate Trust Settings.
Toggle the mitmproxy to on.
--------------------

--------------------
For Android:
Select Android at the menu that shows at the web site. A pop up shows up asking to input a name for the certificate. Put in grabber or whatever you want. It's a reference name only. Select OK.
--------------------

That's it. You should now be able to browse the web using your browser on your phone using your VPS as a proxy server. Try some websites. You can also verify it by going a website that checks your ip. It should show the ip of your VPS not your wifi network.

Lastly, just as a test, you can start the flex app on your phone. This won't run the script or anything because it's not installed or running yet. It should just work as normal. Check to see if a block is available for example. It should be working normally. If there's any kind of problem, you will need to troubleshoot it. It could be some type of firewall issue, or ip address located in the wrong country, etc. It's beyond my scope to be able to help for these things.

When done, go back to the VPS and run ./stopit.bat to turn off the proxy server.

To really see if it's still listening or not, type this on the VPS:
netstat -lnp | grep ':8080'
If it returns a line that has 8080 in it, the proxy server is still running.

Also, on your phone, go to your wifi and set the proxy to off. You can leave the certificate part as it is.

When all this is turned on and running, your VPS has the ability to read your Flex temporary access token whenever you check a block on the flex app.

All that's left is to get the scripts running on the VPS to check and grab the blocks.

Cheaper than godaddy...at least the first year...

https://www.1and1.com/vps-hosting
Be careful, you will be competing against other server based bots.

Most important is it being close to the flex server, having enough cpu power and ram to run the scripts quickly. See my first post. This is not something for which you want to shop for the cheapest vps. It could be in the $20 per month range for access to 2-4 fast CPUs and 1-2 GB Ram, SSD, and on the East Coast near Virginia, New York, New Jersey. You also want to avoid getting kicked off your VPS for overusing it's resources.
 
#45 ·
Step 3 - Start the Proxy Server and Test It

Run this command:
./doit.bat
I did it but it shows;

Usage: killall [OPTION]... [--] NAME...
killall -l, --list
killall -V, --version
-e,--exact require exact match for very long names
-I,--ignore-case case insensitive process name match
-g,--process-group kill process group instead of process
-y,--younger-than kill processes younger than TIME
-o,--older-than kill processes older than TIME
-i,--interactive ask for confirmation before killing
-l,--list list all known signal names
-q,--quiet don't print complaints
-r,--regexp interpret NAME as an extended regular expression
-s,--signal SIGNAL send this signal instead of SIGTERM
-u,--user USER kill only process(es) running as USER
-v,--verbose report if the signal was successfully sent
-V,--version display version information
-w,--wait wait for processes to die
What's wrong? And can you advise how to set the mitm proxy on Android 7.0?
 
#27 ·
discourse_mitmproxy_org/t/clear-flow-list-via-script/304

(replace _ with . -- forum won't allow url addresses in my reply)

Can you explain what this guy was trying to do using mitmproxy to grab blocks? You can see his justloop.py python source code in the second image. Can you give a general explanation of how the app communicates with flex-capacity-na_amazon?

Does your method basically mimic the information sent from the app to flex-capacity-na_amazon server and respond in the same way as the app to flex-capacity-na, thereby eliminating the requirement for the app to run on the VPS? Is the flex-capacity-na server located in the east coast, and if so, do you recommend subscribing to a VPS located somewhere in Virginia or NY?
 
#28 ·
Yes. My method does the same thing as the app. The app itself doesn't run on the vps. The links listed in that second post are from his script repeatedly querying for blocks.

Yes also on the VPS near Virgina / New York. If you go too far you might be wasting your time. As competition increases you will need to be closer and check faster and more times than others. Hence you also want a decent amount of CPU power and RAM. See first post.
 
#29 · (Edited)
Step 4 - Watch and Extract the Proxy Output

This step installs a program to watch the output of the proxy server.

Whenever you check a block from the flex app on the phone while connected to the proxy server, it will detect it and extract the variables needed for the next time you start checking blocks. These variables expire about every 30 minutes and will need to be refreshed by checking for another block on the app.

This is all work to do on the VPS:

a) Type these commands to install, start, and enable incron - a program which watches for file changes and does something when it does.

apt install incron
systemctl enable incron.service
systemctl start incron.service

b) Tell the proxy file monitor to watch for the changes to run this script:

incrontab -e

If you get an error about user not allowed to use incron, edit the file /etc/incron.allow and put the username that will be running the script in there

Put this in the text editor that opens up:
Code:
/home/proxy/proxy.out IN_MODIFY /home/proxy/getparms.bat
Control X to exit and Select Yes to save when prompted.

This completed step now will take all the variables that are needed by the flex app and copy them to the script directory. From the /home/scripts/amazon/source directory is where you will check and grab blocks.

You can test to see if this step works. Connect to your proxy server from your phone by the previous steps. Check for a block. Now, look in the /home/scripts/amazon/source directory. There should be three files with information in them:

amazon-token.txt
flex-id.txt
session-token.txt
 
#33 · (Edited)
Bonus: Switch Anytime Between Prime Now and Logistics

I have also recently discovered that with this method I can go back and forth at will between Prime Now and Logistics. I am only signed up with Prime, but when I point the script to the logistics warehouse it shows offers and accepts them as well.

As far as I can tell, the app on the iphone does not openly show this option, but the flex server openly allows access to anything Flex related in my city. The block appears in the app as normal once it has been accepted.

Tired of Logistics and want to try the other ? Or the other way around? This may be one solution if support jerks you around when you ask.
 
#34 ·
Thank you so much for your tutorial! I have learned so much in such a short period of time, can't wait for your next step tutorial.

I supposed the VPS will accept whatever is available from the flex server, the problem is I only work for one location in Berkeley, but right now they are releasing blocks for 3 locations, is it hard to alter the script to filter out the other locations?

Currently I am using bot on my phone, and I can forfeit any blocks( start within 45 minutes) as long as within 5 minutes time frame after the block is accepted, will this rule still apply if I use VPS to grab blocks?
 
#35 · (Edited)
I have the setup of the proxy right and its pulling my info when i look for blocks but it looks like cron isn't making amazon-token.txt
flex-id.txt
session-token.txt files ? any idea why ?


Update I now got flex-id.txt and session-token.txt but not amazon token is that my login token ?

Lol Update 2 I got it my typos
 
#37 · (Edited)
One more question in my output.txt im getting
-e
08/04/2017_03:55:56:318
{"Message":"before - TokenException validating token with Aztec; x-amz-access-token: ......
Is this correct ?
That means your token expired or perhaps something else wrong with it. You need to check for an offer again on your phone that uses the proxy server. This will update your 3 variables on the vps. Now do the checks again for offers on the vps and you should get offer responses (normally it just says offer list then blank until an offer comes through). If you see token exception, it usually means the variables are expired. They last around 15-45 minutes. The script sample I gave stops when it sees the token exception output.

Thank you so much for your tutorial! I have learned so much in such a short period of time, can't wait for your next step tutorial.

I supposed the VPS will accept whatever is available from the flex server, the problem is I only work for one location in Berkeley, but right now they are releasing blocks for 3 locations, is it hard to alter the script to filter out the other locations?

Currently I am using bot on my phone, and I can forfeit any blocks( start within 45 minutes) as long as within 5 minutes time frame after the block is accepted, will this rule still apply if I use VPS to grab blocks?
I think you can target the specific warehouse you want. Part of the offer check specifies the warehouse. I haven't tested it much except recently I went back and forth between Prime and Logistics. If the locations have different id's it should work. Or just add a line to the script to make sure the offer is only in the warehouse you want before trying for it.

I found my warehouse list with the ids in the proxy log file as a response to something that the app did. You'll need to extract the warehouse id from that when you query the flex server along with your 3 variables. I actually just had my warehouse hardcoded in my scripts.

As I mentioned, I only used it for a simple setup I had with Prime out of a single location. Others may need to refine it to work with their various setups.
 
#39 ·
in the
'Cookie: session-token="***SESSION-TOKEN***"'
-H 'x-amz-access-token: ***AMAZON-TOKEN***'
Am I calling it the right way
Ex:
'Cookie: session-token=",19:timestamp"'
-H 'x-amz-access-token: Atna|Eo464Mc'
if not could you give an Ex of how I should be calling these VARs
 
#40 · (Edited)
in the
'Cookie: session-token="***SESSION-TOKEN***"'
-H 'x-amz-access-token: ***AMAZON-TOKEN***'
Am I calling it the right way
Ex:
'Cookie: session-token=",19:timestamp"'
-H 'x-amz-access-token: Atna|Eo464Mc'
if not could you give an Ex of how I should be calling these VARs
The cookie session token is not right. Thats the beginning of a raw dump line not the extracted variable.

The access token looks right, but is way too short.

For actual examples for your blocks, just do a check for offers on your app.

Then go on the vps in the /home/proxy directory.

Look near the bottom of the file proxy.out for a request that includes:
GetOffersForProvider

That whole line will be a dump of the actual request for an offer. It will contain the variables and the url being checked. For example, it says:
session-token=" and then shows the session-token the actual app used.
GetOffersForProvider?serviceAreaIds= and then shows your warehouse id.

All you do is form those into a curl request with the same variables. I gave the exact format in my post above with the script code in it. Three are extracted into the three files. The warehouse id I actually just hard coded. Those four variables have to be put in to the script I posted instead of the *** marked areas.

All this process does is use the information that is output in the proxy.out file and automates it.
 
Status
Not open for further replies.
You have insufficient privileges to reply here.
Top