rigorousintuition.ca

Starting in the middle of December 2011, my servers got DDoSed by what I believe were Friends of Ryan Cleary (FoRC). Cleary himself is electronically tagged, under curfew at his mother's house (he must be accompanied by his mother if he leaves the house outside of curfew), and forbidden from using the Internet. Those are his bail conditions since last June. However, what if his girlfriend came over with a little laptop with a wireless Internet connection, stuffed in a backpack? What if Ryan picked up the telephone or wrote letters? I don't have much more than circumstantial evidence, but I believe it was FoRC that DDoSed my servers. Cleary or some friends of his may have hired someone to attack me.

The DDoS was heavy when it happened. All six servers were attacked at various times, and all were disconnected by my two different service providers when they were attacked. The mode of attack was a SYN flood. I caught thousands of SYN_RECV requests at various times in the TCP process table. Typically, the load would instantly jump from below 0.50 to about 90.0 and the server was out of action. Then after the DDoS subsided within an hour or so, the load would slowly recover and I could get into the box and see all the SYN_RECV I had caught via a cron job that runs once a minute. The IP addresses sending these were either from a botnet or they were spoofed. In other words, they were from all over the world, and there were hundreds of unique IP addresses. This was not one person with LOIC.

My two service providers disconnected my servers when they saw this. In December I was able to beg them to restore connectivity to me within a few days. There was one provider in Arizona with two dedicated servers that I leased, and another in Florida with four servers that I leased. I took wikipedia-watch.org down in December because I thought this was FoRC's target and they would leave me alone if it was gone.

Taking wikipedia-watch.org down did not stop the DDoS efforts. On January 17 the Florida provider disconnected all four servers, because the DDoS that just hit the four was so massive that it took down upstream switches. They had to consider their other customers on those switches who went down due to collateral damage. I tried begging, but they wouldn't reconnect my servers. They never want to hear from me again, and wouldn't refund $200 that was paid in advance on those servers. They were mad as hell.

It was clear by now that I was the target, and not just wikipedia-watch. The SYN_RECV that I captured in December showed that Scroogle IP addresses were targeted, and sometimes any other open port.

For seven years, I have been running Scroogle on those six servers. Now Scroogle is crippled because I have to run it on the two remaining servers. It's probably just a matter of time before those two are taken away from me. The thing about those six servers is that three were blocked in 2007 by Google, and the other three blocked in 2008 by Google. Each time an engineer at Google got them unblocked for me because I was running a nonprofit service and didn't show any ads. The second time he did this, he mentioned that it was getting hard for him to pull this off. (Each of my six servers appears at Google under a single IP address for that server. That's the way it works with dedicated servers for outgoing traffic. For incoming traffic, you can point them to various IPs assigned to your server.)

From 2009 to March 2011 I had no problems with Google blocks. It helped that I used up to 9,000 dedicated Google IP addresses on a random basis, spanning as many as 80 Google data centers. During that period Google had no centralized detection and throttling system. In March 2011 this changed, and after a one-minute delay, Google could detect and throttle a single IP address that had been fetching search results, even though I was randomly spreading out the requests from each server using 9,000 Google IP addresses. That throttling was not too severe — it lasted about two minutes. When it happened, I did an instant failover to a different server for those throttled requests.

At the end of last December, I leased a seventh server because I knew Scroogle was in trouble. It turned out that this new IP address was throttled much more severely than the six IP addresses from my six legacy Scroogle servers. Adding more servers is not a solution. While it takes about fifteen minutes to trip, this second form of throttling lasts for 90 minutes after the requests stop instead of just two minutes. The four servers in Florida had "legacy" IP addresses attached to them, and now they're gone forever. The load shifts to the two remaining Arizona servers with legacy IP addresses, which cannot handle the load during daytime hours.

Scroogle has gone from 350,000 searches per day to about 200,000 per day. I blame Friends of Ryan Cleary. For the attempted searches that don't go through, I show a screen blaming Google. After all, if Google hadn't started this "mild" form of throttling in March 2011, I could handle the load on two servers instead of six.

The entire Internet infrastructure is in trouble because the design did not anticipate DDoS attacks. You cannot do anything about a SYN flood attack that is halfway sophisticated, and uses many unique IPs, either from a botnet or spoofed. You need a hardware firewall in front of your server that has a huge amount of bandwidth, just so it can filter out the attack based on some sort of analysis that differentiates the malicious packets. That sort of capability costs a lot more than leasing the server itself.

"Sorry.

This URL has been excluded from the Wayback Machine."

wordspeak2 wrote:Why on Earth would he do that?

jingofever wrote:Removing Documents From the Wayback Machine. He probably did it himself.

jingofever wrote:

jingofever wrote:Removing Documents From the Wayback Machine. He probably did it himself.

I assume he does the same thing with Google.

02.19.2012
(was 02.13.2011)
Have you ever thought about disappearing... wiping out your old identity and starting fresh, with a new name, a new life, a new self? In this hour we try to find out how to disappear completely. You too can vanish without a trace!

jingofever wrote:

jingofever wrote:Removing Documents From the Wayback Machine. He probably did it himself.

I assume he does the same thing with Google.

eyeno wrote:From what I understand you can still get to sites that have been dns de-listed if you know the exact ip numbers? How do you go about getting the exact ip numbers for a site?

eyeno wrote:For instance, I would not know how to determine RI ip number if it disappeared. How does all that work?