POJO Mojo

Spent the last month of 2005 traveling. I was in London to speak with Rod Johnson--learned a lot. I was in Miami for the Spring Experience. I was in Belgium for Javapolis. I felt it a great honor to hang out with Jonas Bonér and help him get Terracotta's message out there. In Miami we demonstrated our new product, Terracotta for Spring. At Javapolis, we had two 15 minute sessions where we did tech-heavy demonstrations as well.

Most solutions that claim to enable scale out for Java applications are really nothing more than in-memory databases - forcing the OO developer to think in relational terms. What is needed is a pure OO model to enable scale-out.

Seems to me that relational theory is of tremendous value (reasonably established by the multi-billion-dollar-per-year industry based on it). OO-theory is of tremendous value as well. They are somewhat orthogonal and needn't be compared. So why then do Java (an OO environment) clustering vendors ask me to use relational theory when working in an OO environment? Because it is easier for me or is it actually just easier for them?

First I should justify my assertion that clustered hashmaps are in-memory databases. If a product follows these few tenets, it should be considered a database:

1. SELECT / get() as a mechanism for looking up data
2. UPDATE / put() as a mechanism for making changes to that data
3. primary key construct for accessing and cross-referencing unique instances of data.
4. a query language to select and/or update groups of or unique data safely.

Is such a product solving problems for Java developers in the most natural, expedient way? And is such a product destined to become a [in-memory] database? If it walks like a duck and quacks like a duck...then its a duck, right?

I believe the Java development community needs to stand together and assert ourselves as wanting to maintain the object oriented / POJO programming paradigm as much as possible. Bob's recent post on Galactic forces explains our belief in more detail, but suffice it to say Spring, Hibernate (and EJB3), are on to something--a return to the simpler ways of single-JVM Java.
Read on if you are interested in my assessment on the performance [dis]advantages of clustered hashmaps / in-memory databases...

We were sponsors at The ServerSide Java Symposium in Las Vegas a few days ago. There was a very fun panel discussion. The panel included

Bruce Tate
Bruce Snyder
Cameron Purdy
Rod Johnson
and me

Lots of folks were asking questions that I felt reflected the general sentiment. One that stood out for me was, "What are vendors going to do to make my application simpler?" The audience member went on to ask about turning Java into a business-automation language (akin to Collaxa as a BPEL engine) as opposed to the systems-level language it is today.

I loved the question though. I think that we are on to something at Terracotta. I think that people are ready to take some Enterprise-level services from the API and JDK layer (i.e. J2EE) and move them into the JRE. After all, isn't the difference between an enterprise Java app and a standard Java app in the service levels (HA, and scalability) it needs to provide as opposed to the API's it should be using?

I think applications will get simpler when we stop providing "enterprise-level APIs" for "enterprise-level" design challenges. Then the anemic domain model that Rod Johnson refers to will start to be addressed.

I have to agree with the general sentiment out there that JavaOne this year was nothing short of amazing. The entire Terracotta team who helped with the show were inspiring to watch in action. We:
1. Spoke with over 3000 attendees about our technology
2. Spoke at sessions to approximately 800 attenedees
3. Had more than 10 articles written on our JVM clustering technology
4. And shipped over 1000 downloads to various folks (most of whom were not in attendance at JavaOne but were hearing the amazing coverage about the conference from folks like ServerSide, javaLobby, Artima, etc.)

I wouldn't want to try to pick one thing as "the most important" thing I saw happening at our booth. But, I have to say that perhaps the coolest was getting to spend time with Geert Bevin, of Rife fame. We poured over his source code level implementation of Continuations and discovered together that it is inherently clusterable--he won't have to change anything for our JVM clustering approach to get underneath. With Terracotta working alongside Rife, folks will get fault tolerant continuation failover and significant scalability.

I am convinced after this conference that Terracotta should work diligently to help cluster as many Open Source projects as we can as quickly as we can. Folks seem to need the functionality. Maybe we will even do it for free, like sessions .

Now I just need to find time to catch up on all of it. The most daunting task will be getting up to speed on all the new content being added non-stop to infoq .

It's [un]official. Both Rife and Terracotta agree that this is a good thing for users of continuations. And, while we are not explicitly supporting Rife with commercial products, I am hoping to get Jonas and Geert to post some samples up here for folks to quickly ramp up on.

I was recently in a meeting with a bunch of cool startup guys. I was explaining our DSO technology to some open source developers who had built a clustering solution by hand using JGroups. They were in the process of migrating from an object-based clustering model to a relational one with Hibernate as the back-end. They expressed significant frustration.

They asserted that:
1. JGroups doesn't scale for their needs
2. JPA is a big mess and doesn't really fix what was broken in EJB2

Interestingly enough, as soon as they learned of DSO's existence, they decided to pause the Hibernate work and take a look. If they can stay object oriented, they would rather do so.

But I digress. I was quite honestly shocked. Not having written an application dependent on JPA, I can't defend it, but on paper it seems to have cleaned up some of the boilerplate that EJB2 demanded of the developer. Seems to me you can get the value without the pain.

Is that not the case? I wonder. I will have to look into this by writing my own application. Stay tuned.

It is truly amazing what Open Source and the Java community in general have done for accelerating the rate of innovation in the enterprise software space. Taylor blogs about a really kewl application he wrote in a few minutes that implements something like Apple Stickies but on the Web.

A scant 5 years ago, I would have tried to build this using Java Applets in the browser for the GUI. It would have then used proprietary TPC/IP-based protocol (or HTTP) to send stickies back and forth to the central server. And, most importantly, it would have made those stickies durable / reliable via straight JDBC to a relational database. I have written such apps in the past 5 years. This prototype, with the exact same features and functions Taylor's app has, would have taken me 2 weeks to stabilize. And it would have been an applet which would not have worked well on all platforms.

Taylor built a flash app w/ OpenLaszlo.
He used SOAP for his client / server protocol via XFire.
And he made a clustered and restartable server by using our own DSO product.

Everything was POJO, soup-to-nuts. The state of the developer world is definitely improving.

The benchmark results showed that XXX enables linear scaling of parallel processing across the grid as the grid (and the data set) increases in size and is limited only by aggregate CPU cycles across the grid. In the test, XXX was able to linearly scale from 2 million aggregations with two servers to more than 60 million aggregations across the 96 servers. This 30 times increase in processing throughput was achieved with only one tenth of a second increase in processing time, or 1.2 seconds compared to 1.1 seconds. Additionally, the tests demonstrated that the data grid storage capacity increases linearly as additional resources are added to the grid and is limited only by the amount of RAM available to the data grid.

Impressive numbers indeed. This vendor did a great job building scalable software. Lately, however, I have grown tired of claims of "infinite linear scale" and even "linear scale." Look at this paper. It calls 30 times (2 vs. 60MM aggregations) throughput from 48 times (2 vs. 96 servers) the servers "linear scale." It is in fact a 38% degradation in performance as the application scales.

Over the next few months, Open Terracotta will have significant performance and availability improvements added to it. In all the tests our customers and prospects have run, we have run faster than they expected (usually 10X faster than a serialization-based clustering solution), but it is very much use case-specific.

How do we as a community define a performance benchmark for clustering? Read on for my thoughts...