POJO Mojo

When running in a single Java VM, I use the java.util.concurrent.CyclicBarrier. A CyclicBarrier takes a thread count N. Every thread that calls await() is blocked until await() has been called N times.

Shouldn't it be that easy when I startup multiple VM's? As a DSO developer I certainly think it should be, so I gave it a try.

Problem

I need to start up N load generating nodes for a performance test. I want the load generation to start at the same time for all the load generating nodes on all the machines.

Solution

I simply marked the enterBarrier and exitBarrier variables in my application as DSO roots and included the CyclicBarrier for instrumentation in the tc-config.xml file (see below).

The Code

Rather than include my whole Performance Application, I stripped out the particular Design Pattern and created the following sample:

package demo.barrier;

import java.util.concurrent.BrokenBarrierException;
import java.util.concurrent.CyclicBarrier;

/**
 * Sample to demonstrate how to instrument things like CyclicBarrier
 * and use them as a distributed mechanism.
 */
public class BarrierSample {
   private int           expectedParticipantCount;
   private CyclicBarrier enterBarrier;
   private CyclicBarrier exitBarrier;

   public BarrierSample(int count) {
      this.expectedParticipantCount = count;
      this.enterBarrier = new CyclicBarrier(expectedParticipantCount);
      this.exitBarrier =  new CyclicBarrier(expectedParticipantCount);
    }

   /**
    * Start up multiple threads and wait.  Once all the theads have
    * started, execute some code.  When all threads have finished
    * executing the code, coordinate the shutdown of the participants.
    */
   public void run() {
      try {
         System.out.println("Starting: " +  expectedParticipantCount +
            " nodes");
         System.out.println("Waiting for all nodes to join...");
	 enterBarrier.await();

         //Do what ever it is you do...
         System.out.println("I'm node: " + this +
            expectedParticipantCount);

         System.out.println("Waiting for all nodes to finish...");
         exitBarrier.await();
         System.out.println("Done");
      } catch(InterruptedException ie) {
         ie.printStackTrace();
      } catch(BrokenBarrierException be) {
         be.printStackTrace();
      }
   }

   public final static void main(String[] args) throws Exception {
      int participantCount = -1;
      try {
         participantCount = Integer.parseInt(args[0]);
      } catch (Exception e) {
         //do nothing
      }
      if (participantCount < 1) {
         System.out.println("Usage: dso-java -cp classes " +
	    "-Dtc.config=tc-config.xml " +
            "demo.barrier.BarrierSample ");
         System.exit(1);
      }
      BarrierSample sample = new BarrierSample(participantCount);
      sample.run();
   }

}

In order to run this sample using DSO I started the Terracotta server using the bin/start-tc-server command, created an tc-config.xml in the directory I am going to run the application from (see below) and started my BarrierSample N times using bin/dso-java. You can also get setup quickly by going into the samples directory and making a copy the jtable directory and editing the files as appropriate for your application. The samples are already configured to work with Ant and have scripts to launch with DSO.

Here is the tc-config.xml that went with it:

<?xml version="1.0"?>
<tc:tc-config xmlns:tc="http://www.terracottatech.com/config-v1">

   <servers>
      <server name="localhost" />
   </servers>
    
   <clients>
      <logs>%d/client-logs-%h</logs>
   </clients>

   <application>
      <dso>

         <!-- mark objects as shared roots -->
         <roots>
            <root>
               <field-name>demo.barrier.BarrierSample.enterBarrier</field-name>
            </root>
            <root>
               <field-name>demo.barrier.BarrierSample.exitBarrier</field-name>
            </root>
         </roots>

         <!-- instrument classes in application -->
         <instrumented-classes>
            <include>
               <class-expression>demo.barrier..*</class-expression>
            </include>
         </instrumented-classes>

      </dso>
   </application>

</tc:tc-config>

Terracotta makes CyclicBarrier a distributed construct instead of a single VM construct. We declare enterBarrier and exitBarrier as roots of shared objects, and the synchronization on shared objects becomes distributed synchronization on shared objects. Finally we include everything under the demo.barrier package and the CyclicBarrier class for instrumentation.

By using the above pattern I was able to coordinate threads between processes in the same manner in which I would coordinate them in a single Java VM. Using dso-java, a script that sets things up for you and then calls your regular "java" command, and an tc-config.xml file I was able to take a multi-threaded application construct and turn it into a multi-threaded multi-virtual machine construct.