« When scale out goes wrong... | Main | Some APIs we have. Some we don't (yet). »
August 15, 2008
Gnip handles the entire Twitter feed and more
posted by ari
Using Terracotta on Amazon EC2, Gnip is handling the fire hose coming from many popular social networking sites. Gnip offers a single consolidated and filtered view of all those streams of data.
Gnip uses no DB. It uses Terracotta and backs that up to S3. The most interesting part to me is the throughput. They do more than 50K operations per second to their TC server. Each operation contains 1000 updates in it. This means Gnip is handling 50,000,000 (50 million) updates per second in EC2 on Terracotta.
We have a few engineers writing down what they did so be patient and you will be able to read more detail.
Amazing stuff!
--Ari
Trackback Pings
TrackBack URL for this entry:
http://blog.terracottatech.com/cgi-bin/mt/mt-tb.cgi/67
Comments
cool!!
Posted by: anjan bacchu at August 16, 2008 1:47 AM
Marvellous! Terracotta is as ground breaking as Amazon EC2 :)
Posted by: Vinay at August 29, 2008 6:31 AM
I'd love to see how they did the backing to S3 from Terracotta.
Posted by: Carl Byström at September 5, 2008 4:58 PM