« Avoiding the Palm / Treo Mistake | Main | Gnip handles the entire Twitter feed and more »
August 10, 2008
When scale out goes wrong...
posted by ari
I met with someone last week who has a massively scaled out system. The application contains user / document information. The system is designed for linear scalability today, and this is achieved mostly by hand.
With a series of load balancers and a good partitioning scheme, the architecture delivers about 500 partitions each with fewer than 10,000 users stored inside. This system scales linearly, all on top of a series of relational database instances. The nasty truth is that scale out has become too expensive.
The goal: get more out of each partition. How? By offloading the DB altogether. Detaching from it and keeping the transient data transient--in memory where it belongs.
Have you hit the scalability wall even though your scalability is linear? Tell me more.
--Ari
Trackback Pings
TrackBack URL for this entry:
http://blog.terracottatech.com/cgi-bin/mt/mt-tb.cgi/66