Question: What is the Slashdot Effect?
Twitter
Facebook
Delicious
Digg
Stumble
Reddit
A valuable reader asks in response to the previous post:
What about the fark effect or slashdot effect ? Are all of these same or
different .. And how system admins handle these kinds of effects ?
Well, I’ll try to add my two cents here. Everyone is welcome to add their own insights to this (esp from Sys Admin side). Trekker: How would you suggest handling any traffic Netscape creates for sites?
The Fark effect or Slashdot effect I think is similar in nature to being highlighted by any major newspaper or journalist — if your product gets on the New York Times or CNN or something, you are bound to get a spike in traffic.
Maybe the iRecord people can share some actual stats from this since they were recently highlighted.
In a very interesting talk by the YouTube founder Jawed Karim given at his alma mater (you should be able to find that in YouTube) Jawed mentions that there wasn’t enough momentum on YouTube until the website was highlighted on Slashdot as “the flickr for videos”, and that “the traffic never really went down since”.
What is special indeed about Slashdot is the cult following it has among technical people and gadget fans.
I personally think it provides this satisfaction and insurance and trust to its reader by directly addressing their thirst for all things cool, without the added noise of whether or not it makes business sense.
I could be wrong on that though (not a big Slashdot person) so if you’re in the know with Slashdot please correct me.
How do System Admins handle these effects?
Again, our larger readers and audience are the best people to answer this. I am really no expert in system management, but here’s my take.
I think the servers that go down depends on what your hosting arrangement is. If you are a large site you will likely have your own mini data-center with a load balancer anyway. If you are a smaller site, you will have a hosting contract for either shared, virtual or dedicated hosting.
The problem naturally is that the incoming traffic crosses the peak of what your allocated web server(s) can handle.
If you are hosting from a contracter, you might be able to immediately call them and negotiate a deal where they can provision an additional server and assign a load balancer to you for the 3-4 days that one of these effects lasts.
Alternatively, you could set up mirrors of the website and set up a proxy that can automatically redirect to the mirror site if your main site goes down.
Most feasibly, however, you can try to look for Grid or Clustered hosting solutions. This would be particularly true if your website is part of a federation or collection of complementary sites (such as Green & White is part of the larger WordPress umbrella).
WordPress can provide the large uptime because it will cluster all the blogs across all the servers in the data center. As soon as one or two spike, they can transparently provision additional computer resources, and then unprovision them when the spike fades away — all of this without the blog ever finding out.
If you want to do that yourself, take a look at Linux Virtual Server (LVS). Otherwise, you can look for cutting-edge hosting providers that provide similar technology.
Gee, I didn’t know how much of a geek I can be.
4:36 pm
a very useful posting by Jeff Atwood for reducing load on you web servers
http://www.codinghorror.com/blog/archives/000807.html