POJO Mojo

Chris Richardson's POJO's in action is definitely worth a read. What I am particularly enamoured with, however, is not the good use cases nor is it the solid, logical step-by-step walk thru of the key design challenges in enterprise programming and how best to plug in Spring or Hibernate. For once, I am rivetted by the beginning of a book!

The first few chapters seek to prove some early assertions around the failure of EJB and J2EE. Specifically, Chris asserts that an API is [paraphrasing here] a way for a designer to help a devloper solve a particular problem. He posits that, when an API is extended outside its natural scope, that API can do more harm than good. Something to keep in mind, certainly.

I get asked over and over again, "what is the right API for this task? Can I use Spring? Can I use Hibernate? Should I use XXX?" It all seems backwards. Spring, for example, is so broad in scope that to ask "does Spring help solve this?" is futile...the answer is always yes and no. Yes it almost always can. And, no it cannot, if you use it incorrectly. Just validates my fundamental belief that in managing a career in software development, when we face the decision wether or not to seek broad knowledge or deep, the answer is BOTH. No point in being a jack of all trades, certainly, but there is sooo much "hammer / nail" going on out there in Java land, we need to start broadening our toolkit (why I love OSS...think about how many developers would not be able to ramp up on Lucene, Rife, Spring, or even Terracotta if it all cost $1MM for a license before you could roll up your sleeves and start learning.)

I guess this all sounds a little preachy. I definitely do not mean to preach. Just seems worth our collective time to avoid the J2EE pratfall. I feel passionate that the notion of ENTERPRISE PROGRAMMING is broken. JEE programmers don't get all the benefits of the language until the runtime starts abstracting things like clustering, reliability, and manageability. Annotations, Aspects, Dependency Injection, and the rest are taking us down a really good path. Of this, I am certain.

Terracotta just announced the world's best, most scalable, easiest to use grid on the market. Everything you need, trust me, it does it. We got:
1. Event grid
2. Data grid
3. compute grid
4. Hyper grid
5. über grid
6. Even, grid grid

We even got grid virtualization with virtualized grid scaling and our stuff can be used with commerce grid, concurrent grid, J2EE grid w/ Spring support, messaging grid, and all the usual use case shtuff.

In our grid you can store and work with POJO, PONO, POCO, JOPO, NOPO, COPO, any plain old object. We call it POxO (pronounced: poh - shows)! That's right! We are so down with grid we can handle ANY object and handle it at hyper linear scale.

What's hyper linear scale, you may ask? Simple...every machine you add makes all the other machines in the grid faster. Yes, we outrun the speed of light. We just roll like that. How can that be? We move the compute to the data. The partition to the network. We move the bits to the bytes and the gnats to the knights.

If you think you got grid, you don't. You think your grid scales but it won't. The virtual grid's whats what so give it a shot.

So much grid you gotta see it today. From Miami to the Bay, we got grid _your_ way!

--------------

When you think about it, vendors have to be joking with all this grid discussion they throw out there. Java in the enterprise is either about JSE / JEE (Spring, Hibernate or JPA, Rife and Wicket, Tomcat or Jetty or JBoss), or it is about workload distribution w/o a database (master / worker or divide & conquer or grid, like Google). I sometimes simplify things down to "scaled-out apps" meaning several application servers running copies of an application (EAR, WAR, classes, what have you) underneath a load balancer all trying to appear as one large scaled out server. The other option is "divide and conquer" where a business problem is spread across machines such that machines do not need to share data to complete a task or a portion of a task nor do they need access to each other's data for availability. So if we have 2 types of architecture:

1. Scaled-out apps
2. Divide and Conquer

Where can grids help? The answer is "Divide and Conquer." And there, the real challenge is not in moving bits and bytes on the network, but in defining the unit of work so that bits don't have to move across processing contexts. I have seen too many use cases on Wall Street and in the gaming & betting universes to try to sell you on the notion that there is a 1-size-fits-all grid API for dividing workload. Anyone who sells grid fundamentally gets the same questions from their customers:

1. How well does it scale?
2. Why can't I just use messaging for this?

The short answer is these grid-things scale only if you can make sure no nodes block on data transformations / workload from other nodes meaning workload must be embarassingly parallel (like searching / querying as opposed to order processing). And, yes, messaging will work. If you partition the problem into the grid, then you can pass workflow control around the grid using messaging...that is, as long as your messaging is orthogonal to your data (i.e., only control flow messages and not data flow). None of this is usually easy. Why? Because most business problems are not data analysis / read-only problems but are, in fact, serialized or workflow-like in nature which means one thing...

Stateful work passes around the grid alongside control flow data when you are trying to process a serialized workflow in a divide & conquer model. Example: a business function needs to transform input from a DB of type "A" into output of type "C" by going from A -> B -> C. So you create workers in your grid. Some take db records and turn those into "A". Some convert "A" to "B" and put "B"'s back in the grid. Other's take "B" and convert to "C". Grid gives you a very clean API for working with the db, and for abstracting the message-passing between A->B transformers and B->C transformers. And by having designed your system such that each transformation is done autonomously and unware of any other steps in the workflow, you can now scale different pieces of workflow separately. Say A->B transformations takes twice as long as B->C. You can now deploy twice as many servers / processes doing A->B as doing B->C. But grid is not going to scale any better than a fast non-persistent message queue. Its a fact that the 2 solutions are being used in the same way and, thus, ignoring implementation details, will scale in the same way.

So, if grid is not as much for scaling as it is for ease of programming, is there an easier way to program? Yes there is. POJO / SEDA / CommonJ / Master-Worker. I and others at Terracotta keep talking about it, but the benefits are several:

1. performance: since objects are not serialized and since objects seemlessly fault in and out of a processing context, the app will benefit in ways a developer may not have contemplated. I saw a use case last week where the start-up time of a grid worker was greatly improved simply by the fact that it did not need to rehydrate its state on restart and could lazily fault in the objects it needed on demand. The notion of network attached memory that is implicit inside the Terracotta model proved powerful enough to speed up the app without a rewrite in this case.

2. simplicity: it is all POJO w/o serialization, w/o put-backs, and if you use Master / Worker, you can even abstract synchronization altogether.

3. flexibility: I have already seen users of our master / worker framework who require intelligent routing that (like a layer 7 http load balanacer) needs to inspect the payload to decide where to route it. Since our master / worker engine is OSS, you can change it to suit your needs. Proprietary / one-off implementations can't do this. You find shortcomings in the interface, and you have to file change orders / feature requests and wait.

Which do you prefer? Open or closed? Flexible or proprietary? Reality and source code or mystery and hype?

So,

A HUGE customer asked today, "what is Terracotta's latency for a simple update at a fine-grained level." Of course, I wish he had asked the question in such simple terms but this is how the question boiled down, at least. This is a very interesting question. I have told folks for a while now that latency with Terracotta is quite different than latency with clustering tools because Terraocotta is not clustering, but is in fact HA / scalability infrastructure for Java apps. But let's avoid marketing hype. Here's the basic logic behind my assertion (the results of the actual test follow directly after I explain the concepts).

Terracotta is first and foremost, TCP-based Network Attached Memory (NAM) which means there is no "n-1 ACK" problem where a cluster of servers must ACK the transaction. The Terracotta server is the only node that needs to know about the change. Secondly, Terracotta works like network attached memory which does what it implies in that it works with Java heap at the byte level. So it is only moving the data that changes from the application's JVM to and from Terracotta. Latency, therefore, on a given network technology should be lower for Terracotta than any other HA / scalability solution, as long as Terracotta's internal overhead is kept in check. This is not to say Terracotta must be faster than anything else; more that it might be faster. So let's measure it now.

I started to make a very fine-grained change in a very tight loop. Right away, this begets a question: given the ethernet windowing size of 1460 bytes, would the smallest memory delta at 1-4 bytes, done many times quickly better test latency than more changes with a less tight loop? Put another way, since each change should logically end up going in its own TCP packet, does it really matter if I change a Boolean, an int, or a 20 character string? And, should I change that field with some rest / recovery time between updates or should I have a compact loop? After all, if I go as fast as Java can, am I testing the network, the CPU, or Terracotta? This is important in understanding Terracotta's performance. I decided to keep it simple for now and go with a double, a tight loop, and Terracotta running on the same server as my application. We will just have to run the other options later to better isolate our goal--computing Terracotta's latency.

Let's look at the code I wrote (it is from the inventory demo available in the standard Terracotta kit):

   private void testLatencyWithPrice() {
      Product p = null;
       {
         String s = "1GFR";
         p = (Product) store.inventory.get(s);
         if (p == null) {
            out.print("[ERR] No such product with SKU '" + s + "'\n");
            return;
         }
      }
      double d = 0.99;
      long latency;
      for( int i = 0; i < 1000000; i++ ) {
        latency = System.currentTimeMillis();
        synchronized (p) {
           p.setPrice(d);
        }
        latency = System.currentTimeMillis() - latency;
        out.print( "Latency for price change: " + latency + "\n" );
      }
   }

         switch (input.charAt(0)) {
          case 'I':
             printInventory();
             continue;
          case 'Q':
             return;
          case 'D':
             printDepartments();
             continue;
          case 'U':
             updatePrice();
             continue;
          case 'H':
             printHelp();
             continue;
          case 'L':
             testLatencyWithPrice();
             continue;
         }

Now, we are pushing the price update about 3000 times per second through to Terracotta. And I confirmed that Locality of reference (I will explain this in another blog entry later just in case people are unsure what it is) is honored by running a 2nd and a 3rd inventory client and observed that the update throughput was unaffected. Anyways, back to the observation on latency.

I put a call to System.currentTimeMillis() in there which somewhat obviously proved too coarse grained to measure the latency of sending a single double to the Terracotta Server. The latency was always measured as zero with both the inventory app and the Terracotta Server running on my laptop (mackbook pro core duo @ 2GHz). I was CPU-bound, BTW (maxed out both cores) so I am not sure this is an accurate test. Nonetheless, I did 3000 updates per second, which means in any one millisecond, I did 3 Terracotta transactions or a latency of 333 microseconds. How did this super low latency occur? The answer is batching and windowing. Rest assured Terracotta did not send 3000 TCP packets as I naively assumed (would have amounted to 34Mbits per second). Terracotta's implementation batches up many changes in a single network call to the Terracotta server.

Well, the next step will be to take it to 2 computers and see if I get more or less throughput. If I get less throughput when Terracotta and my app are each running on different machines (on gigabit ethernet), then the loopback interface is the answer to the low latency.

If I get more throughput with Terracotta on its own linux server / commodity server, then the fact that I was CPU-bound suggests I under-estimated the Terracotta Server's latency and its latency is in fact lower. I would then need to go back to 1 machine but with a test that does not peg CPU prematurely. Granted I am backing in to latency via throughput testing, but it illustrates how tricky performance testing is and it also test latency under load which is far more important than one-off latency.

Stay tuned.