My Bandwidth is Being Throttled (And I’m Going to Do Something About It)

ajit-pai-banner

I live in someone else’s house. No, not my parents, someone else. Don’t ask. Suffice to say, I’m extremely poor and can’t afford my own apartment, so a group living situation is my only option at this point. Anyway, this house has WiFi, which is cool. For a while we didn’t have WiFi and I had to walk two miles to the library to use the Internet.

And I don’t know what it is, but sometimes I swear, something on the network is throttling my bandwidth. I don’t know if it’s the ISP or the local gateway, but whatever it is, it’s slowing down older connections in favor of newer connections. I know this because every time I do a long download, it starts off fast and then slows to a crawl after the first five seconds or so. I won’t take this abuse. I’m determined to outsmart this throttling device – whatever it is – so that it will let me download large files again.

I thought maybe if I had the connection time out after detecting a certain amount of throttling and then start again from where the download left off, the throttling device would interpret it as a new connection, and I wouldn’t have to sit there hitting Ctrl+C and repeating the command over and over. But that’s not enough, because the fact that the next download attempt comes immediately after the previous one is a sure indicator that it’s the same connection. Essentially I need to fool the throttling bot into thinking I’m manually sitting there and attempting download after download with space between downloads. It doesn’t want me to automate the process and go do something else. It wants to waste my time. It doesn’t want me to be smart and find creative solutions to the problems it’s causing me. It wants me to suffer.

So I got to work writing a shell script that attempts a download, times out after five seconds of low bandwidth (meaning < 1 byte per second), then waits for a random number of seconds and tries again. It takes forever, but at least I don't have to sit there constantly retrying like the throttling bot wants me to do, and to be frank, it doesn't take nearly as long as it did when I wasn't using the script.


 1 #!/usr/bin/env bash
 2 # Anti-Throttling GET
 3 
 4 file="$1"
 5 shift
 6 declare -i timeout=5
 7 declare -i randbase=500
 8 
 9 while getopts ":T:R:" opt
10 do
11         case $opt in
12           T) declare -i timeout=$OPTARG;;
13           R) declare -i randbase=$OPTARG;;
14           \?) echo "Invalid option: -$OPTARG" >&2;;
15         esac
16 done
17 
18 while true
19 do
20         wget -c -T $timeout "$file"
21         declare -i r=$RANDOM%$randbase+1
22         echo "Waiting $r seconds..."
23         sleep $r
24 done
25 
26 unset opt timeout randbase r

This is the first version of the shell script that I wrote. I made a modification to it which I will post later in this article, but I just want to focus a bit on the process I went through. This script is fairly succinct and easy to understand. It lets you set the timeout interval and the maximum time to wait. This maximum is used as the base for the modulus operator when it selects the random time interval. I’ve found that generally speaking the larger the file you’re downloading is, the longer you want to wait between download attempts.

I realized that waiting an integer number of seconds was still too obvious. What if the throttling bot is intelligent and can detect signatures of an automated process? I need to make the minimum increment small enough that the time it takes to process it will make the minimum waiting interval negligible. In other words, I want to make sure the time is not arbitrarily close to some interval, say 15.001 seconds, because an AI on the other end can easily see that that’s just an automated 15 second wait plus a small amount of time for processing. But if I make the intervals small enough, it won’t be able to separate the intervals from the processing time, which will make that aspect of automation undetectable. Here’s my final script:


 1 #!/usr/bin/env bash
 2 # Anti-Throttling GET
 3 
 4 file="$1"
 5 shift
 6 declare -i timeout=5
 7 declare -i randbase=500
 8 
 9 while getopts ":T:R:" opt
10 do
11         case $opt in
12           T) let timeout=$OPTARG;;
13           R) let randbase=$OPTARG;;
14           \?) echo "Invalid option: -$OPTARG" >&2;;
15         esac
16 done
17 
18 while true
19 do
20         wget -c -T $timeout "$file"
21         declare -i p=$randbase*100  # 1/100 seconds base
22         declare -i q=$RANDOM%$p+1   # 1/100 seconds delay
23         declare -i r=$q*10000       # microseconds delay
24         declare -i s=$q%100         # fractional part
25         declare -i t=$q/100         # integer part
26         echo "Waiting $t.$s seconds..."
27         usleep $r
28         unset p q r s t
29 done

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s