So like everyone else I have been reading the back and forth between Alex from Twitter and DHH about scaling Rails up to the level of traffic Twitter is seeing at the moment.
However, I was thinking about this tonight and the numbers just don’t sit right with me:
At that time they were fielding spikes of up to 11,000 requests per second across some 16 cores with very little caching thrown into the mix to mitigate. No wonder their site had been feeling slow.
11,000 requests per second!! That seemed very high so I did a little digging for stats from some other sites:
| MySpace | 15,000req/sec (40billion/month – source) |
| 11,000req/sec | |
| YouTube | 1,500req/sec (4 billion/month – source) |
Even if you take the Twitter number as a peak against the others which are averaged out across a month, it just seems way too high, YouTube could spike to 7x its average traffic before it came close to the twitter peak. I know it has become very popular (and I am a fan, in case you think I am just biased against the service) but I find it hard to believe these numbers.
If these numbers are true, then it is great news for Rails on MySQL. If Twitter can just about scale to this level of traffic across only 16 cores with one back end MySQL server and “very little caching” then that pretty much blows any performance complaints about Rails out of the water.
However, even if those numbers are out by an order of magnitude or even two, I still think that this is good news for Rails scaling because it shows that it follows exactly the same scaling pattern as other web frameworks and you will eventually become database bound and that is the hard problem to solve. Even better I think it shows the power of Ruby/Rails that as a response to this argument Nic Williams has released a gem which implements multiple database server pooling in ~75 lines of Ruby and about a day or so of work. Using a master write and multiple read database cluster can scale a massive amount more than a single DB or an Active/Passive set-up.
Note: The above quote is taken from DHH’s article, I am sure that the original interview article mentioned 11,000req/sec and you can see plenty of references to 11,000req/sec in the comments it seems to have been edited now, maybe I am right to doubt the numbers.
