Transentia » Groovy

Another GroovyMag ‘Masterpiece’: Gambling on Griffon And Grails Going Gangbusters

bob — Mon, 10 May 2010 00:03:35 +0000

Another GroovyMag article.

Should stop the other ones from feeling lonely!

Somehow, the zip bundled with the article didn’t have the full sources…don’t know what happened. My apologies to all who tried to find them.

Never mind, here they are.

Maybe its a case of “Better Late Than Never”? Or “All Good Things Come To Those Who Wait”?

Enjoy!

Gr3

Gambling on Griffon and Grails Going Gangbusters

Bob Brown

bob@transentia.com.au

http://www.transentia.com.au

The Grails and Griffon frameworks both strive to the same end: to bring the goodness of Convention over Configuration to your life. Individually, they are excellent technologies but when working together using the REST architectural pattern they can enable you to achieve great things.

A Brief Overview

They go by many names: ‘Pokies’, “one-armed bandits”, “slot machines”, “fruit machines” and even “video gaming terminals.” Some call them good clean fun and harmless entertainment; others denounce them as scourges on society, preying on those members least able to afford feeding them. Governments are addicted to the tax they generate, and big business gets bigger profits from them each year. Socially-minded campaigners regularly call for them to be banned and in an attempt to minimize their negative impact on his clientele, Russell “The (Social) Gladiator” Crowe took the unusual step of removing them from the clubhouse when he invested in his favorite Australian football team—the Sydney Rabbitohs—in 2007.

Given all this controversy, and also given my desire to explore the wonderful REST-ful world of Griffon and Grails, what else could I do but risk the ire of Russell and the many Rabitohs supporters and build a pokie (albeit very simple and cashless) of my very own!

Figure 1 shows Gr3, the “Griffon, Grails and Groovy Pokie Machine.”

Figure 1: The Gr3 Client Application

As the name suggests, this is a Griffon desktop application that talks to a Grails backend application. Griffon provides a REST plugin and Grails has native support for REST, so developing REST-style interactions is especially easy.

The Gr3 application is simple to use: select an amount to play with and press the ‘Play’ button. If lady luck deigns to smile on you, you will win. More likely, the heartless random number generator underlying things will more than happily conspire to take your ‘money’ from you. The application displays a number of regularly-updating interesting statistics, most notably the overall totalizer value which reflects the total amount of money in the server’s ‘pot’ at any instant [1]. Each instance of Gr3 will add to and take money from this single pot as gameplay continues. There is also a simple preferences configuration GUI.

This application uses the “hot off the press” Grails 1.2.0-RC-1 and Griffon 0.3-SNAPSHOT versions.

REST-Style Interactions

REST defines a set of architectural principles for designing simple HTTP-oriented Web services. These principles are:

Make proper use of the various defined HTTP methods
Expose all services as stateless services; assume that the client best knows how to manage its own state
Expose directory structure-like URIs that uniquely identify interesting or useful resources
Transfer data in structured form amenable to further processing using technologies such as XML, JSON (JavaScript Object Notation) or YAML (YAML Ain’t Markup Language)

REST is increasingly popular and is on its way to becoming a dominant technology due to its fundamental simplicity (especially when compared to the alternative [but more comprehensive] SOAP/XML-based ecosystem).

Figure 2 shows the REST-style interactions that take place between the Gr3Client and Gr3Server. Although the actual sequence is quite simple, there are some guiding rules that I have adopted:

The register and unRegister operations are treated akin to Create and Delete and (as prescribed by REST) use the HTTP POST and DELETE methods. They return a plain text status string.
The play and win operations are treated as Update and so use the HTTP PUT method. They return a JSON object containing the most-up-to-date totalizer value.
The totalizer operation is purely idempotent [2] and so uses HTTP GET.

While these principles may not be in keeping with a purist approach to REST, they are still quite appropriate and allow me to demonstrate various aspects of REST in Griffon and Grails. In any case, the pure REST approach is more focussed on CRUD [3] update of network accessible resources than dealing with the more process-oriented service such as we have with Gr3Server.

This may sound rather complex but you will soon see that the support built into Grails and Griffon makes it all as easy as losing your shirt in Las Vegas. Figure 2 shows that the interaction between the Gr3Client and Gr3Server is really quite simple.

Figure 2: Sequence Diagram Showing the Simple Interactions between Gr3Client and Gr3Server

The ‘Gr3Server’ Grails WebApp

This is the simpler of the two applications. It is made up of a single TotalizerController class, with a little bit of supporting code thrown in.

Let’s get started.

Listing 1 shows the TotalizerController class in its entirety [4].

package gr3

class TotalizerController {
  private static final JSON_CT = "application/json"
  private static final TEXT_CT = "text/plain"
  def totalizerService
  def clientRegistryService
  static allowedMethods =
    [startAction: 'POST', stopAction: 'DELETE',
     playAction: 'PUT', winAction: 'PUT',
     totalizerAction: 'GET']
  def beforeInterceptor = {
    def remIp = request.remoteAddr
    def registered = clientRegistryService.registered(remIp)
    if (registered && (actionName == 'registerAction')) {
      log.warn "Incoming register request from registered client: $remIp"
      return false
    }
    else if ((! registered) && (actionName == 'unRegisterAction')) {
      log.warn "Incoming unRegister request from unregistered client: $remIp"
      return false
    }
  }
  def registerAction = {
    def remIp = request.remoteAddr
    log.debug "Register: $remIp"
    clientRegistryService.register(remIp)
    render(text: "OK", contentType: TEXT_CT)
  }
  def unRegisterAction = {
    def remIp = request.remoteAddr
    log.debug "UnRegister: $remIp"
    clientRegistryService.unRegister(remIp)
    render(text: "OK", contentType: TEXT_CT)
  }
  def totalizerAction = {
    log.debug "totalizerAction request from ${request.remoteAddr}"
    render(contentType: JSON_CT) {
      [tote: totalizerService.getCurrentValue()]
    }
  }
  def playAction = {
    def amt = params.int('amount')
    def tote = totalizerService.accumulate(amt)
    log.debug "playAction request from ${request.remoteAddr}: ${amt}; tote now: ${tote}"
    render(contentType: JSON_CT) {
      [tote: tote]
    }
  }
  def winAction = {
    def playValue = params.int('playValue')
    def winnings = params.int('winnings')
    def tote = totalizerService.accumulate(-winnings)
    log.debug
      "winAction request from ${request.remoteAddr}: *** WIN *** played: ${playValue}, won: ${winnings}; tote now: ${tote}"
    render(contentType: JSON_CT) {
      [tote: tote]
    }
  }
}

Listing 1: the Main TotalizerController class

This code is quite unremarkable. There are only a few points of interest, the first of these being the allowedMethods map. This is a standard facility in Grails that makes it possible to restrict how a given action closure may be invoked. I have shown it here because it also helps make the correspondence between the code and my guiding rules for the REST interactions in this application clear.

A second point of interest concerns TotalizerController’s reliance on two injected services. The TotalizerService (shown in Listing 2) is basically an extremely simple wrapper for an AtomicInteger, which represents the instantaneous accumulated value of the global totalizer. The use of AtomicInteger, which is inherently thread safe, keeps the coding for the shared singleton service clean and clear and free from nasty synchronized blocks.

[…elided…]

class TotalizerService {
  boolean transactional = false
  private final tote = new AtomicInteger()
  def getCurrentValue = { tote.get() }
  def accumulate = {amt -> tote.addAndGet(amt) }
}

Listing 2: the TotalizerService class

The third major point of interest in TotalizerController is the injected ClientRegistryService (shown in Listing 3).

[…elided…]

class ClientRegistryService implements InitializingBean {
  def authorisedIPs
  def clientRegistry
  boolean transactional = false
  def register = {ip -> clientRegistry[ip] = true }
  def registered = {ip -> clientRegistry[ip] == true }
  def unRegister = {ip -> clientRegistry[ip] = false }
  void afterPropertiesSet() {
    def authorisedMap = [:]
    for (ip in authorisedIPs)
      authorisedMap[ip] = false
    clientRegistry = authorisedMap.asSynchronized()
  }
}

Listing 3: the ClientRegistryService class

ClientRegistryService is concerned with providing a simple veneer of security; it maintains a list of IP addresses and enforces the rule that only one client may be present at any given IP address. As shown in Figure 2, a client registers itself at startup and unregisters at shutdown.

ClientRegistryService relies on having a configured list of IP address injected into it via the authorisedIPs local variable. It will restrict access to those clients only and will also maintain the connected status of these clients. To achieve this operational rule, the service is defined to be an InitializingBean. This means that the underlying Spring Framework infrastructure will call into the defined afterPropertiesSet method after it has created the service instance and subsequently injected any needed resources into that instance. The afterPropertiesSet method examines the injected authorisedIPs list and constructs a client status map according to the list’s contents.

The authorisedIPs list is configured externally to the service via Grails’ conf/spring/resources.groovy file, as shown in Listing 4.

beans = {
  xmlns util: "http://www.springframework.org/schema/util"

  util.list(id: 'authorisedIPs') {
    value '127.0.0.1'
  }
}

Listing 4: resources.groovy, showing the Configuration of the authorisedIPs List

Note that this is a slightly ‘artificial’ example: for ‘real’ production-quality code I could (and probably should) have easily defined and initialized the actual requisite clientRegistry map instance via resources.groovy; this path gave me a chance to introduce the useful InitializingBean facility though, so please forgive my slightly roundabout coding.

It is important that access to the clientRegistry be synchronized to guard against wayward concurrent update. The authorisedMap.asSynchronized() expression achieves this.

TotalizerController generally invokes the clientRegistryService from within its beforeInterceptor closure. This gives a single point of control for the access checks and helps keep the controller code DRY [5].

REST-Style URL Mappings

It is slightly “bad form” to directly expose the names of the underlying controller actions to the client application; doing so results in fragile code that potentially cannot be changed without affecting the client. Since “fearless change” is one of the basic tenets of Agile development, I have established a series of indirect mappings in conf/URLMappings.groovy. This is shown in Listing 5.

class UrlMappings {
  static mappings = {
    "/totalizer/play"(controller:"totalizer",
      action: 'playAction', parseRequest:true)
    "/totalizer/win"(controller:"totalizer",
      action: 'winAction', parseRequest:true)
    "/totalizer/register"(controller:"totalizer",
      action: 'registerAction')
    "/totalizer/unRegister"(controller:"totalizer",
      action: 'unRegisterAction')
    "/totalizer/totalizer"(controller:"totalizer",
      action: 'totalizerAction')
  […elided…]
  }
}

Listing 5: REST URL Mappings

Of interest here is the way that the mappings for play and win also enable Grails’ automatic JSON-parsing behavior.

The ‘Gr3Client’ Griffon Desktop Application

The client side portion of the pokie application is quite a lot bigger than the server-side portion that we have just examined. This is to be expected: GUI handling code is often quite involved and the client contains a fair bit more functionality.

The Main Application MVGGroup

Every Griffon application consists of one or more groups of Model, View and Controller (called, naturally enough, an MVCGroup). In this section we’ll take a look at the main MVG group; this is what produces the portion of the GUI shown in Figure 1.

The underlying model is quite straightforward, as Listing 6 (excerpted) shows.

import groovy.beans.Bindable
class Gr3ClientModel {

  […elided…]

  @Bindable String playedValue = '$0'
  @Bindable Integer plays = 0
  @Bindable Boolean playButtonEnabled = true
}

Listing 6: the Main MVCGroup’s Model Class

There is an @Bindable property for each value displayed in the view, as well as a property defining the state of the play button. The @Bindable annotation invokes an AST transformation [6] that wraps up the PropertyChangeSupport facility that all Swing developers have learned to know and love.

The application’s main view (shown in part in Listing 7) makes good use of Griffon’s ‘UberBuilder’ to allow a view to be created using features from several underlying libraries (in this case, Swing and SwingX [7])

[…elided…]

def leftReelIcons = [imageIcon('/reel/card_figures_-_club_01.png'),
        imageIcon('/reel/card_figures_-_diamond_01.png'),
        imageIcon('/reel/attenzione_rotondo_archi_01.png'),
        imageIcon('/reel/card_figures_-_heart_01.png'),
        imageIcon('/reel/card_figures_-_spade_01.png')]
def middleReelIcons = (leftReelIcons[2..-1] + leftReelIcons[0..1])
def rightReelIcons = (middleReelIcons[2..-1] + middleReelIcons[0..1])
def reelIcons = [left: leftReelIcons,
                 middle: middleReelIcons, right: rightReelIcons]
application(title: 'Gr3Client',
        size: [320, 740],
        resizable: false,
        defaultCloseOperation: WindowConstants.DO_NOTHING_ON_CLOSE,
        windowClosing: controller.exitAction,
        locationByPlatform: true,
        […elided…]) {
  menuBar(mainMenuBar)
  vbox(constraints: NORTH) {
    jxheader(title: "Gr3",
            description:
              'Pokie Machine using Griffon, Grails and Groovy.',
            icon: imageIcon('/griffon-icon-48x48.png'),
            border: emptyBorder(4),
            titleForeground: Color.RED,
            backgroundPainter: new PinstripePainter(Color.LIGHT_GRAY)
    panel() {
      tableLayout(cellpadding: 4) {
        tr {
          td(align: 'left') {
            label('Plays:')
          }
          td(align: 'right') {
            label(id: 'plays', text: bind { model.plays})
          }
          td(align: 'left') {
            label('Played:')
          }
          td(align: 'right') {
            label(id: 'played', text: bind { model.playedValue})
          }
        }
        tr {
          td(align: 'left') {
            label('Earnings:')
          }
          td(align: 'right') {
            label(id: 'earnings', text: bind { model.earnings})
          }
          td(align: 'left') {
            label('Totalizer:')
          }
          td(align: 'right') {
            label(id: 'totalizer',
                  text: bind { model.totalizerValue})
          }
        }
      }
    }
  }
  panel(new Spindle(), id: 'spindle',
        border: loweredBevelBorder(), constraints: CENTER) {
    borderLayout()
    tableLayout(cellpadding: 4) {
      tr {
        for (reel in ['left', 'middle', 'right']) {
          td(align: 'center') {
            widget(id: "${reel}Reel".toString(),
                   new Reel(icons: reelIcons[reel]))
          }
        }
      }
    }
  }
  panel(constraints: SOUTH) {
    tableLayout(cellpadding: 4) {
      tr {
        buttonGroup(id: 'playValue').with {group ->
          for (v in [1, 2, 5, 10, 20, 50])
            td {
              radioButton(text: "\$$v".toString(),
                          buttonGroup: group, selected: v == 20,
                          actionCommand: "$v".toString())
            }
        }
      }
      tr {
        td(colspan: 6, align: 'center') {
          label(id: 'message', text: bind { model.message })
        }
      }
      tr {
        td(colspan: 6, align: 'center') {
          button(playAction, id: 'playButton')
        }
      }
    }
  }
}

Listing 7: the Main MVCGroup’s View

The clean, declarative nature of Listing 7 is a thing of beauty to my eyes! For those of us who have dreams formatted into discrete pages marked up by HTML, Groovy’s tableLayout layout manager is A Good Thing indeed.

Of interest is the WindowClosing event handling. To allow the client to communicate with its server at shutdown time, custom exit-handling code is established in the application stanza. To allow the custom processing to work, it is important to ensure that conf/Application.groovy is edited with:

autoShutdown = false

Note the way the ‘playValue’ Button group is defined. This adopts Josh Reed’s tip for easing the creation of groups of mutually-exclusive radiobuttons. It’s worth taking a look at Josh’s site. See “Learn More” for the URL.

The Panel/Spindle teamup is also worth taking a look at. The documentation for the panel widget says: “A javax.swing.JPanel is returned, unless the user passes in a subclass of JPanel as the value argument, in which case the value argument is returned.” The Spindle class used here uses this oft-overlooked ability and is a ‘bog-standard’ class that extends JPanel so that the red ‘win’ line is painted across the panel and on top of the ‘reels.’

The use of SwingX painters is discussed in “A Little Gloss: SwingX Painters”, below.

The main MVCGroup’s controller contains very little ‘trickery’ and all the REST-related work is offloaded to an injected ServerService class, so (in the interest of saving space) it won’t be shown here. Various sections of the controller will be examined as this article continues.

Additional Conventional Handling

The Gr3Client application is not only comprised of models, view and controllers; there are actions, menus and services as well. Gr3Client takes advantage of Griffon’s conventional handling for these aretefacts in addition to the standard MVC conventional handling.

Griffon processes actions and menus if they are contained in eponymously-named directories under griffon-app. Figure 3 shows the Gr3Client’s resulting directory tree structure.

Figure 3: Additional Conventional Directories

It is also necessary to modify the entry for the MVCGroup in Application.groovy to be modified appropriately. For example:

'Gr3Client' {
  model = 'Gr3ClientModel'
  controller = 'Gr3ClientController'
  actions = 'Gr3ClientActions'
  menus = 'Gr3ClientMenus'
  view = 'Gr3ClientView'
}

Ordering is important in this definition: actions and menus are required to be processed first, since they are dependencies of the view.

In addition to making the code clearer, this feature also makes it slightly shorter. Consider this excerpt from actions/Gr3ClientActions.groovy:

action(id: 'playAction',
       name: 'Play',
       closure: controller.playAction,
       shortDescription: 'Play',
       enabled: bind { model.playButtonEnabled }
)

Normally, the list of actions would be required to be wrapped in an actions {} stanza. By using this facility, the actions wrapper is assumed. The same is true of any menus defined by an application.

Services can be placed in a services directory and are recognized and injected where needed ‘automagically.’

Scheduling Periodic Updates

The main view displays a global totalizer value that is continually updated. To achieve this, a ‘background’ instance of javax.swing.Timer is created during MVCGroup creation to periodically poll the Gr3Server and update the associated model value. As would be expected, the code is very simple and is excerpted in Listing 8.

import java.awt.event.ActionListener
import java.text.NumberFormat

class Gr3ClientController {

  […elided…]

  def serverService

  private static final nf = NumberFormat.getCurrencyInstance()
  def totalizerAction = {evt = null ->
    doOutside {
      model.totalizerValue = nf.format(serverService.totalizerAction())
    }
  }
  void mvcGroupInit(Map args) {
    def timer = new javax.swing.Timer(1500, totalizerAction as ActionListener);
    timer.setInitialDelay(500);
    timer.start();

[…elided…]

Listing 8: Background Timer Handling

The totalizerAction closure is periodically executed by the timer in the context of Swing’s Event Dispatching Thread (EDT). To ensure that the GUI remains responsive, the closure performs its time-consuming communications, formatting and model update operations outside of the EDT. In a traditional Swing application, threading is quite awkward but the Groovy/Griffon SwingBuilder DSL makes life much easier.

Further points of interest in Listing 8 include the use of the standard Java java.text.NumberFormat facility to enable currency formatting, and Groovy’s powerful capability that enables the totalizerAction closure to be coerced to an instance of java.awt.event.ActionListener, which is what the Timer instance requires.

The ‘PrefsPanel’ MVCGroup

In addition to the main application MVCGroup, Gr3Client has an additional PrefsPanel MVCGroup accessible via the File->Preferences menu. This simple dialog is shown in Figure 4.

Figure 4: the Preferences MVCGroup Dialog

As Listing 9 shows, the model portion of the dialog’s MVC triumvirate is straightforward; the model is simply a value and associated validity flag for each textfield.

import groovy.beans.Bindable

class PrefsPanelModel {
  @Bindable String server = PrefsUtils.getServer()
  @Bindable String port = PrefsUtils.getPort()
  @Bindable Boolean serverTfValid = true
  @Bindable Boolean portTfValid = true
}

Listing 9: the PrefsPanel Model class

The PrefsPanel’s view (Listing 10) is similarly simple.

panel(border: loweredBevelBorder(), constraints: CENTER) {
  tableLayout(cellpadding: 2) {
    tr {
      td { label("Server:") }
      td {
        textField(id: 'server', columns: 24,
          text: bind(source: model, 'server', mutual: true,
            validator: { controller.isNonBlank(server) }))
      }
    }
    tr {
      td { label("Port:") }
      td {
        textField(id: 'port', columns: 5,
          text: bind(source: model, 'port', mutual: true,
            validator: { controller.isInteger(port) }))
      }
    }
  }
}

Listing 10: an Excerpt from PrefsPanelView.groovy

Of interest here is the use of mutual binding. Mutual binding allows for view interactions to update the model (as is normal in a GUI application) and also allows the model to update the view component…a less frequently undertaken activity. Given that validation is also required, almost the complete model/view binding syntax is on display here.

The PrefsPanelController is mostly concerned with setting up and performing validation of the textfields. What is not related to validation is concerned with persisting the various preferences values and controlling the view’s destruction.

[…elided…]

class PrefsPanelController {
  // these will be injected by Griffon
  def model
  def view

  final gb = BorderFactory.createLineBorder(Color.GRAY)
  final rb = BorderFactory.createLineBorder(Color.RED)

  def isNonBlank = {tf ->
    model.serverTfValid = (tf.text ==~ /\S+/)
  }
  def isInteger = {tf ->
    try {
      def n = Integer.parseInt(tf.text)
      if ( ! (0..Short.MAX_VALUE - 1).contains(n))
        throw new IllegalArgumentException("Port out of range")
      model.portTfValid = true
      return n
    } catch (x) { /* NumberFormatExn, IllegalArgumentExn*/
      model.portTfValid = false
    }
  }
  private handleSaveButton = {->
    view.saveButton.enabled =
      model.portTfValid && model.serverTfValid
  }
  void mvcGroupInit(Map args) {
    // this method is called after model and view are injected
    model.addPropertyChangeListener({evt ->
      handleSaveButton()
    } as PropertyChangeListener)
    model.addPropertyChangeListener("portTfValid", {evt ->
      view.port.border = model.portTfValid ? gb : rb
    } as PropertyChangeListener)
    model.addPropertyChangeListener("serverTfValid", {evt ->
      view.server.border = model.serverTfValid ? gb : rb
    } as PropertyChangeListener)
  }
  def postInit() {
    view.server.text = model.server = PrefsUtils.getServer()
    view.port.text = model.port = PrefsUtils.getPort()
    model.serverTfValid =  isNonBlank(view.server)
    model.portTfValid = isInteger(view.port)
  }
  def closeWithoutSavingAction = {evt = null ->
    view.preferencesFrame.visible = false;
    destroyMVCGroup('PrefsPanel')
  }
  def savePreferencesAction = {evt = null ->
    PrefsUtils.setServer(model.server)
    PrefsUtils.setPort(model.port)
    closeWithoutSavingAction evt
  }
}

Listing 11: the PrefsPanelController class

Validation

Validation is one area where Griffon is still a rookie player. The basic framework is in place, but there are still a few ‘wrinkles’ that have yet to be worked out. The mvcGroupInit closure in Listing 11 shows how to react to changes in the model properties and how to update the view appropriately. Figure 5 shows the result of this.

Figure 5: Indicating an Error

The postInit closure defined in PrefsPanelController is required to work around a small problem where the initial values for a model are not validated before being bound to the view component(s). One has to ensure that the model always sees one change after binding so that validation can proceed [8].

This still provides a useful learning opportunity. The Gr3ClientController class contains this code:

def preferencesAction = {evt = null ->
  def (m, v, c) = createMVCGroup('PrefsPanel')

  c.postInit()
}

This excerpt shows the use of Groovy’s multiple assignment facility to ease the processing of the list instance returned from the createMVCGroup method.

Griffon’s powerhouse lead developer Andres Almiray has discussed an alternative–and probably more general—approach to this issue on the griffon-user mail list:Binding and validating seems at odds….

Persisting Preferences Data

Since Griffon is capable of generating a standalone application, an applet and a Java WebStart application from the same codebase, preferences handling is not as straightforward as one might think initially: where to store a configuration file in the face of different operating system filesystem conventions and security schemes? Is the JNDI available? Is the Windows Registry an option? What API is most convenient? Something ostensibly so simple is actually quite a problem for portable code [9].

The java.util.prefs package provides a way for applications to store and retrieve user and system preference and configuration data. The data is stored persistently in an implementation-dependent backing store but is accessed in a consistent fashion regardless of type of code or underlying system.

The Gr3Client application wraps the Preferences API into a simple PreferenceUtils class, as shown in Listing 12.

import java.util.prefs.Preferences

class PrefsUtils {
  private static final SERVER_KEY = 'server'
  private static final DEFAULT_SERVER = 'localhost'
  private static final PORT_KEY = 'port'
  private static final DEFAULT_PORT = '8080'
  private static final prefs = Preferences.userRoot().node('/Gr3')
  static setServer = {s ->
    prefs.put(SERVER_KEY, s)
    prefs.sync()
  }
  static getServer = {
    prefs.sync()
    prefs.get(SERVER_KEY, DEFAULT_SERVER)
  }
  static setPort = {p ->
    prefs.put(PORT_KEY, p)
    prefs.sync()
  }
  static getPort = {
    prefs.sync()
    prefs.get(PORT_KEY, DEFAULT_PORT)
  }
}

Listing 12: the PreferencesUtils class

At first sight, this code seems to contain lots of unnecessary syncs (which flush changes and refreshes the local cached data from the underlying persistent store). Strictly speaking this is indeed so, but I found that the regime I have used here was very helpful when testing: I could manually ‘munge’ the preferences store [the windows registry in my case] and see those changes happen immediately. Frequent syncing actually represents quite a trivial overhead in the grand scheme of things, so I left them in.

Using the REST Plugin

REST support in Griffon is provided courtesy of a plugin written by Andres Almiray. Andres’ weblog (see “Learn More”) is a “must see” site for any Griffon developer. Andres has written an equivalent plugin for Grails which makes a visit to his weblog doubly rewarding.

Following normal Griffon practice, the REST plugin (version 0.2 at time of writing) is installed via:

griffon install-plugin rest

I have configured the REST plugin to only work its magic on services (“out of the box” it is configured to work on controllers). This line needs to be added/edited in conf/Application.groovy:

griffon.rest.injectInto = ["service"]

I have split the network-handling code into a separate service, created via the command line:

griffon create-service ServerService

ServerService is a very simple class. Listing 13 shows just one action closure from this class but the others are essentially identical.

def playAction = {value ->
  withRest(id: 'play', uri: NetworkUtils.URI) {
    def resp = put(path: 'play',
            body: [amount: value],
            requestContentType: JSON,
            responseContentType: JSON)
    assert resp.status == 200
    resp.data.tote
  }
}

Listing 13: Excerpted ServerService Action

The REST-related code is exceptionally clear! If you walk through the invocation sequence, you will be able to see easily how the incoming request and associated response is processed.

Note how automatic JSON response parsing makes for clean code: there is no need for either side to marshal/unmarshal the value for tote to/ from the textual on-the-wire representation, for example.

One slight subtlety is concerned with the id given to the withRest method. To quote the REST plugin’s documentation: “All dynamic methods will create a new http client when invoked unless you define an id: attribute. When this attribute is supplied the client will be stored as a property on the instance’s metaClass. You will be able to access it via regular property access or using the id: again.” This gives a nice ‘tuning’ ability in that it makes it possible to reduce the number of instances that are created and almost immediately destroyed. The ServerService makes use of this facility: playAction and winAction share an id (play), totalizerAction has an unique id (and thus an unique HTTPClient instance), while registerAction and unRegisterAction do not reuse connections (on the assumption that the time between register and unRegister operations could be very long and keeping an unused connection around would be counterproductive).

Configuring Logging Support

Even though the syntax enabled by the REST plugin is quite simple, as I developed the application I still needed to see the request-response interaction between the Gr3Client and Gr3Server. To do this, I turned to that tried and trusted standby: Log4J. Although Griffon uses Log4J internally, it does not actually make it available to an application at runtime.

Configuring run-time support for Log4J is not such a “no brainer” as it is for Grails, but is still a fairly simple 3-step process:

Step 1: Copy the log4j jar from the Griffon distribution’s lib directory into the application’s lib directory.
Step 2: Create conf/Events.groovy with the contents shown in Listing 14.

import org.apache.log4j.Logger

onNewInstance = {klass, type, instance ->
  instance.metaClass.logger =
    instance.metaClass.log =
      Logger.getLogger("gr3Client.${klass.name}")
}

Listing 14: Events.groovy Needed for Logging

This is quite a ‘shotgun’ approach: it injects a log property into everything. A more sophisticated variation of this code (“Left as an exercise for the reader”) would be better for ‘production’ use. For the current purposes, however, it is fine.

Step 3: Add the code shown in Listing 15 to lifecycle/Initialize.groovy. This is mostly an ‘inline’ Log4J configuration file.

import groovy.xml.DOMBuilder

def xml = """

     
          
          
              
          
     
     
          
     
     
     
          
     
     
          
     
     
          
     
     
          
     
     
          
          
     

"""

def reader = new StringReader(xml)
def doc = DOMBuilder.parse(reader)
new org.apache.log4j.xml.DOMConfigurator().
  configure(doc.documentElement)

Listing 15: Inline Log4J Configuration

The above three steps will turn on some quite extensive (and very useful!) debugging output from the REST plugin, as this excerpt (Listing 16, showing a single ‘play’ interaction) illustrates:

DEBUG RESTClient - PUT http://localhost:8080/Gr3Server/totalizer/play
DEBUG wire - >> "PUT /Gr3Server/totalizer/play HTTP/1.1[EOL]"
DEBUG wire - >> "Accept: */*[EOL]"
DEBUG wire - >> "Content-Length: 13[EOL]"
DEBUG wire - >> "Content-Type: application/json[EOL]"
DEBUG wire - >> "Host: localhost:8080[EOL]"
DEBUG wire - >> "Connection: Keep-Alive[EOL]"
DEBUG wire - >> "User-Agent: Apache-HttpClient/4.0 (java 1.5)[EOL]"
DEBUG wire - >> "Expect: 100-Continue[EOL]"
DEBUG wire - >> "Accept-Encoding: gzip,deflate[EOL]"
DEBUG wire - >> "[EOL]"
DEBUG wire - << "HTTP/1.1 100 Continue[EOL]"
DEBUG wire - << "[EOL]"
DEBUG wire - >> "{"amount":20}"
DEBUG wire - << "HTTP/1.1 200 OK[EOL]"
DEBUG wire - << "Server: Apache-Coyote/1.1[EOL]"
DEBUG wire - << "Content-Type: application/json;charset=UTF-8[EOL]"
DEBUG wire - << "Transfer-Encoding: chunked[EOL]"
DEBUG wire - << "Date: Sat, 12 Dec 2009 11:34:05 GMT[EOL]"
DEBUG wire - << "[EOL]"
DEBUG RESTClient - Response code: 200; found handler:
  org.codehaus.groovy.runtime.MethodClosure@a4911d
DEBUG RESTClient - Parsing response as: application/json
DEBUG wire - << "b[EOL]"
DEBUG wire - << "{"tote":40}"
DEBUG wire - << "[\r]"
DEBUG wire - << "[\n]"
DEBUG wire - << "0[EOL]"
DEBUG wire - << "[EOL]"
DEBUG RESTClient-Parsed data to instance of: class
  net.sf.json.JSONObject

Listing 16: Debugging a REST Interaction

This configuration can be easily ‘tweaked’ for any purpose, of course.

Don’t Panic! Recall that it is still quite “early days” for Griffon, and I believe that there has been some discussion aimed at eventually developing a logging plugin or addon. Forthcoming Spring framework support and the new ArtefactManager will also make the sort of configuration currently being performed in onNewInstance easier and more targeted in the future.

A Little Gloss: SwingX Painters

For those not already familiar with the SwingX project, here is the overview blurb from the SwingX home page:

“Contains extensions to the Swing GUI toolkit, including new and enhanced components that provide functionality commonly required by rich client applications. Highlights include:

Sorting, filtering, highlighting for tables, trees, and lists
Find/search
Auto-completion
Login/authentication framework
TreeTable component
Collapsible panel component
Date picker component
Tip-of-the-Day component

Many of these features will eventually be incorporated into the Swing toolkit.”

Griffon’s SwingX plugin packages all this goodness up and makes it easy to use alongside the normal Swing components. Don’t head down to the gaming tables without these components safely packed into your kitbag.

The SwingX JXHeader component supports a relatively new facility called painters. Listing 7 shows how a simple PinStripePainter can be defined and passed to the jxHeader stanza in the view builder and Figure 1 shows the results of this. Not too shabby, but not particularly breathtaking, either.

Listing 17 shows a more ‘beautiful’ alternative using a CompoundPainter.

def gloss = new GlossPainter(
        paint: new Color(1.0f, 1.0f, 1.0f, 0.2f),
        position: GlossPainter.GlossPosition.TOP)
def stripes = new PinstripePainter(
        paint: new Color(1.0f, 1.0f, 1.0f, 0.17f),
        spacing: 5.0)
def matte = new MattePainter(fillPaint: new Color(51, 51, 51))
def cp = new CompoundPainter(matte, stripes, gloss)
[…elided…]
jxheader(title: "Gr3",
      […elided…]
      backgroundPainter: cp)

Listing 17: a Compound Painter

As Figure 6 shows, this is much nicer and has a distinctly more ‘modern’ feel to it.

Figure 6: the Compound Painter “In Action”

Win Evaluation

To perform win evaluation, the Gr3ClientController’s playAction closure calls the injected serverService. This is shown in Listing 18.

def playAction = {evt = null ->
  doOutside {
    def lr = view.leftReel
    def mr = view.middleReel
    def rr = view.rightReel
    lr.activate()
    mr.activate()
    rr.activate()
    edt {
      view.spindle.repaint()
    }
    model.plays++
    def playedAmount =
      Integer.valueOf(view.playValue.selection.actionCommand)
    def pTote = serverService.playAction(playedAmount)
    model.totalizerValue = nf.format(pTote)
    model.playedValue =
      nf.format(playedAmount + nf.parse(model.playedValue))
    def res = evaluator.evaluate(lr, mr, rr)
    switch (res) {
      case EvaluatorResult.SMALLWIN:
      case EvaluatorResult.BIGWIN:
        model.playButtonEnabled = false
        def winnings =
          winningsService.evaluate(res.multiplier, playedAmount)
        model.earnings =
          nf.format(winnings + nf.parse(model.earnings))
        def wTote = serverService.winAction(playedAmount, winnings)
        model.totalizerValue = nf.format(wTote)
        switch (res) {
          case EvaluatorResult.SMALLWIN:
            model.message = "Congratulations! You just won: ${nf.format(winnings)}."
            SoundUtils.smallWin()
            break
          case EvaluatorResult.BIGWIN:
            model.message = "Hey BIG WINNER! You just won: ${nf.format(winnings)}."
            SoundUtils.bigWin()
            break
        }
        model.playButtonEnabled = true
        break
      default:
        model.message = CondolencesUtils.condolences()
        break
    }
  }
}

Listing 18: Win Evaluation

Within playAction threading issues are noteworthy, in particular the use of doOutside and edt to ensure correct and responsive view behavior.

Separating the evaluation into a separate service is beneficial and gives flexibility. It allows for the definition of a very simple ruleset like “A win occurs if each icon under the line is the same” or for more sophisticated rules taking into account such things as diagonal or alternate arrangements and the actual icons involved.

Listing 19 shows one possible (very simple) evaluator.

// simple function: a win is when all icons in the
//   middle position are the same
def evaluate = {lr, mr, rr ->
  // assume that all reels have the same number of icons
  def mrm = mr.model
  int mid = mrm.size() / 2
  def le = lr.model.getElementAt(mid)
  def me = mrm.getElementAt(mid)
  def re = rr.model.getElementAt(mid)
  (le == me) && (me == re) ?
    EvaluatorResult.SMALLWIN : EvaluatorResult.LOSE
}

Listing 19: A Very Simple Win Evaluator

It is possible to easily replace this simple code with a more sophisticated evaluator with very little knock-on effect [10].

A Little Whimsy: JFugue

No pokie can possibly be considered playable if it doesn’t have the capability to produce at least a few ear-splitting bells and whistles! In this case, incipient deafness comes courtesy of the excellent JFugue. In its own words: “JFugue is an open-source Java API for programming music without the complexities of MIDI.”

Listing 20 shows how simple adding sound to your application can be.

import org.jfugue.Pattern
import org.jfugue.Player
class SoundUtils {
  private static final player = new Player()
  private static final smallPattern =
    new Pattern("T[Presto] I[Rock_Organ] Db4minH C5majH C5majW")
  private static final bigPattern =
    new Pattern(
      "T[Presto] I[Rock_Organ] Db4minH C5majH C5majW C5majH Db4minH")
  static smallWin = {-> player.play(smallPattern) }
  static bigWin = {-> player.play(bigPattern) }
}

Listing 20: the SoundUtils class

If you look back to Listing 18, you will see how this code is used within the playAction closure.

Wrapping Up

I hope that I have been able to show you that it’s not a gamble to use these technologies together. Both are powerful frameworks fully capable of permitting sophisticated development. The REST support in both makes writing distributed applications simple while the ability to incorporate existing code such as Log4J and JFugue simply cannot be beat.

I leave you with this quote from one of the ‘greats’ of Science Fiction, writer and social commentator Robert Heinlein: “There is no such thing as ‘social gambling.’ Either you are there to cut the other bloke’s heart out and eat it—or you’re a sucker. If you don’t like this choice—don’t gamble.”

Learn More

Grails home page, http://groovy.codehaus.org
Griffon home page, http://griffon.codehaus.org
Andres Almiray’s Weblog, http://www.jroller.com/aalmiray/
Griffon REST Plugin page, http://griffon.codehaus.org/Rest+Plugin
Painter Tutorial, http://swinglabs.org/docs/frameworks/painters/index.jsp
JFugue, the Java API for Music Programming, http://jfugue.org/
Josh Reed’s ButtonGroup shortcut, http://josh-in-antarctica.blogspot.com/2009/10/griffon-tip-silly-swingbuilder-tricks.html

Bob Brown is the director and owner of Transentia Pty. Ltd.. Based in beautiful Brisbane, Australia, Bob is a specialist in Enterprise Java and has found his niche identifying and applying leading-edge technologies and techniques to customers’ problems.

[1] As a youngling, I lived in Macau (the “Las Vegas of the East”) for a few years. I clearly remember the astonishment I felt when I saw the Casinos totalizer boards in action for the first time. The amount displayed updates so rapidly that the last 2-3 whole-dollar figures of the value are simply a continually-changing blur. The amount shown is strictly monotonically increasing, which added to the sense of awe I experienced.
[2] An idempotent operation is one that has no side-effects and so can be repeated without changing the state of the system in any way; it is a pure read operation.
[3] CRUD = Create, Retrieve, Update, Delete.
[4] I will not go into the ‘mechanics’ of how to drive either Grails or Griffon here. The website for each technology (see “Learn More” for the URLs) covers those aspects very nicely.
[5] Don’t Repeat Yourself; a key tenet of any developer worth their stake at the table.
[6] According to Groovy’s documentation: “AST Transformations provides Groovy with improved compile-time metaprogramming capabilities allowing powerful flexibility at the language level, without a runtime performance penalty.” They give us compile-time code generation for boilerplate patterns, in other words.
[7] SwingX support is provided by the swingx-builder plugin
[8] Groovy Jira GROOVY-3939 tracks this issue, for the curious.
[9] And all Java developers should aspire to write good portable code!
[10] Indeed, I have a version that uses JBoss Drools but that’s probably a story for another time…

Playing With GPars

bob — Thu, 04 Mar 2010 23:17:40 +0000

Way, way, waaaay back, I became something of a minor Occam ‘guru’ (I actually implemented a fair proportion of a compiler/runtime for Occam 1) and I have maintained an (academic and practical) interest in the field throughout my career (which is what I laughingly call what I do these days, see Will Code For Food for more ;-))

It’s therefore natural for me to take a look at GPars.

In its own words, GPars is:

an open-source concurrency library for Groovy that aims to give you multiple high-level abstractions for writing concurrent code in Groovy – map/reduce, fork/join, asynchronous closures, actors, agents, dataflow concurrency and other concepts, which aim to make your Groovy code concurrent with little effort.

Ever played the charmingly-named Chinese Whispers game at a party? That’s what I’m going to play with here.

It’s a very simple example of a pipelined processing task. This sort of thing is very common in signal processing, process control and image munging.

In the applications I build here, there will be a source thread generating messages which are each then sent through a chain of intermediate threads to a sink thread, which simply bounces the message back down the chain. On each ’sinkward’ step the intermediaries may or may not munge the message in some way as it whispers it to the next step, so that the version that eventually makes its way back to the source may be changed in some unpredicatable way.

Here’s the image to keep in your head:

GPars provides several alternative ways of tackling this problem.

Scala made the idea of Actors popular, so that’s a good starting point.

This is what the GPars doco says about Actors:

The actor support in gpars were inspired by the Actors library in Scala but have meanwhile gone beyond that.

Actors allow for a messaging-based concurrency model, built from independent active objects that exchange messages and have no mutable shared state.

If you are a webby-person, you can think of an Actor as a Servlet…you won’t go far wrong.

Coming up is a version of “Chinese Whispers” implemented using GPars’ Actors:

package chinesewhispers

@GrabResolver(name = 'jboss', root = 'http://repository.jboss.org/maven2/')
@Grab(group = 'org.codehaus.gpars', module = 'gpars', version = '0.9')

import groovyx.gpars.actor.*
import chinesewhispers.utils.Whisperer

Actors.defaultPooledActorGroup.resize 16

class Stop {}
class Go {}

final intermediaries = []

def src = Actors.actor {index ->
  println "Source; Starting..."
  react { Go go ->
    println "Source; time to 'Go'"
    def msg = 'The sky is green, the trees are blue...SNAFU!'
    println "Source; message is: $msg"
    for (i in 0..<8) {
      intermediaries[1].sendAndContinue msg, { ix, reply ->
        println "Source[$ix]; received: '$reply'"
      }.curry(i)
    }
    intermediaries[1].send(new Stop())
    stop()
    print "Source; ...Stopped."
  }
}

intermediaries << src

for (i in 1..8) {
  intermediaries << Actors.messageHandler({ index ->
    when {Stop msg -> intermediaries[index + 1].send msg; stop(); }
    when {msg ->
      def sender = msg.sender
      intermediaries[index + 1].sendAndContinue Whisperer.whisper("${msg}"), { reply ->
        sender.send reply
      }
    }
  }.curry(i))
}

intermediaries << Actors.messageHandler {
  when { Stop msg -> stop() }
  when { msg -> reply msg }
}

src < < new Go()

intermediaries*.join()

This simple script creates a list of Actors, with a source Actor first, followed by a number of essentially identical intermediary Actors and finally a sink Actor.

Here it all is, “in action” (pun intended):

GPars creates Actors in a runnable state so that it is easy to start things off by sending the Source a unique Go message.

Notice how each Actor sends a message and then allocates a closure to asynchrounously handle the appropriate return message. This ensures that intermediaries get new work as soon as possible, so the pipeline is always running at full capacity. This makes for a very performant solution.

It’s a truism that getting a parallel system off and running is fairly easy, but getting one to Stop properly can be surprisingly tricky. For this application, the Source Actor’s last activity before it finishes is to send a (guaranteed-to-be-unique, obviously) Stop message up the pipline. On receipt of a Stop, each intermediate Actor simply forwards the message and then terminates in a nice orderly fashion. the final Sink Actor receives the forwarded Stop and simply stops without trying to send the message back down the chain of intermediaries (since the chain is effectively no longer there).

The messageHandler closure’s distinct ‘when’ syntax makes the sort of message dependent processing shown here very easy.

The final line of the script simply waits for all Actors to finish shutting down before allowing the overall script to exit.

Simple, performant and easy to write and comprehend. What’s not to like!

GPars actors are not perfect (they lack some of the Scala library’s more subtle abilities such as pattern matching to guard message reception, and the ability to see the length of the incoming mesage queue), but they’re not too shabby, either.

For a second bite at the apple, here’s the DataFlow version.

According the the doco:

Dataflow concurrency offers an alternative concurrency model, which is inherently safe and robust. It puts emphasis on the data and their flow though your processes instead of the actual processes that manipulate the data. Dataflow algorithms relieve developers from dealing with live-locks, race-conditions and make dead-ocks[sic] deterministic and thus 100% reproducible. If you don’t get dead-locks in tests you won’t get them in production.

What does a Dataflow-based Chinese Whispers program look like? Viola:

package chinesewhispers

@GrabResolver(name = 'jboss', root = 'http://repository.jboss.org/maven2/')
@Grab(group = 'org.codehaus.gpars', module = 'gpars', version = '0.9')

import groovyx.gpars.dataflow.*
import chinesewhispers.utils.Whisperer

class Link {
  final up = new DataFlowVariable()
  final down = new DataFlowVariable()

  Link(index) {
    /*
      up >> {println "The up[$index] variable has just been bound to $it"}
      down >> {println "The down[$index] variable has just been bound to $it"}
    */
  }
}

final NINT = 8

for (iteration in 0..<8) {
  final intermediaries = []
  for (i in 0..
      intermediaries[index].up << Whisperer.whisper(intermediaries[index - 1].up.val)
    }.curry(i))
    DataFlow.task({ index ->
      intermediaries[index - 1].down << intermediaries[index].down.val
    }.curry(i))
  }

  DataFlow.task {
    intermediaries[NINT - 1].down << "Did you REALLY mean to say: '${intermediaries[NINT - 1].up.val}'?"
  }
}
"OK"

Neat, eh? Simple assignment to a variable, or reading a variable’s value performs inter-task communication. No messy send and receive mechanisms here!

Here’s what it looks like when running inside IntelliJ:

Of course, you lose a fair bit of flexibility, but for straightforward tasks the simplicity of the solution more than compensates…

Each bi-directional link between intermediary tasks is modelled as a Link with a separate ‘up’ and a ‘down’ variable. Although GPars offers alternatives, for this application I have chosen to use plain “Dataflow Variables”, which are write-once, read-many entities; each iteration through the main loop thus recreates the requisite Link instances (and hence the Dataflow Variables) from scratch.

Uncomment the body of the Link constructor and you’ll be able to see the inter-task communication in action. Simple but very effective debugging:

Just for completeness sake (it’s a minor supporting actor, not a star): here’s the Whisperer class that the examples use:

package chinesewhispers.utils

class Whisperer {
  final static r = new Random()

  final static map = [
          'sky': 'elephant',
          'trees': 'kettles',
          'green': 'effervescent',
          'blue': 'smelly'
  ]

  static whisper(msg) {

    if (r.nextBoolean())
      return msg

    def split = msg.split(/\W/).findAll { it }
    def l = split.size()
    def sel = r.nextInt(l)
    def word = split[sel]

    def w = map[word]
    w ? msg.replace(word, w) : msg
  }
}

That’s all folks; two versions of the same application is enough for any mortal!

I am eagerly looking forward to the time when the work to integrate JCSP into GPars bears fruit.

By the way, each of these examples should happily work with GPars 0.9 and Groovy 1.7.1. Just copy the listings into GroovyConsole and go (you might need to bring the Whispers class inline into each application, but that’s trivial to do).

The Old (Groovy) Switch-eroo Again…

bob — Tue, 23 Feb 2010 11:29:17 +0000

Recently, I had the little task of looking at a string and deciding what to do based on the various values of an embedded prefix field.

I had to ask around a bit (thanks Paul!) to get the solution to my liking, but here’s yet another example of my favourite Groovy construct, the switch statement:

def days = [
  "Monday:I could wait till Tuesday",
  "Tuesday:must make up my mind",
  "Wednesday:would be fine",
  "Thursday:she's on my mind",
  "Friday:give me time",
  "Saturday:I could wait",
  "Sunday:would be too late"
]

final MON = 'Monday:'
final TUE = 'Tuesday:'
final WED = 'Wednesday:'
final THU = 'Thursday:'
final FRI = 'Friday:'
final SAT = 'Saturday:'
final SUN = 'Sunday:'

days.each { value ->
  switch(value) {
    case ~/^${MON}.*$/:
      println "$MON: ${value - MON}"
      break
    case ~/^${TUE}.*$/:
      println "$TUE: ${value - TUE}"
      break
    case ~/^${WED}.*$/:
      println "$WED: ${value - WED}"
      break
    case ~/^${THU}.*$/:
      println "$THU: ${value - THU}"
      break
    case ~/^${FRI}.*$/:
      println "$FRI: ${value - FRI}"
      break
    case ~/^${SAT}.*$/:
      println "$SAT: ${value - SAT}"
      break
    case ~/^${SUN}.*$/:
      println "$SUN: ${value - SUN}"
      break
  }
}

I know it’s a very contrived example, but take a look at the case ‘labels’: regular expressions that reference properties found elsewhere in the code. Neat-o!

Apologies to Sting (did you get the song?)!

GroovyMag February 2010: Another Article By Yours Truly

bob — Tue, 09 Feb 2010 22:58:40 +0000

Another GroovyMag USD$4.99 blockbuster.

This time taking a look at the combination of Grails, Griffon and a REST-like interaction style. It’s all Good Stuff.

Worth every cent.

Copy Image From URL To File

bob — Mon, 08 Feb 2010 09:22:21 +0000

Just saving this away for a rainy day…something I concocted in response to a question on the Groovy mailing list:

new File("local.gif").withOutputStream { os ->
  new URL("remote.gif").withInputStream { is ->
    os << is
  }
}

The important thing to note here is the way the streams are carefully ’scoped’ and maintained so that no leaks can occur even if exceptions or other sticky situations arise.

The Groovy Internet Mood Meter in SVG

bob — Thu, 04 Feb 2010 10:24:36 +0000

This is yet another GroovyMag article.

This one was published in September 2009 and is still running on Stax (just follow that last link to learn more).

Since the time of writing, Microsoft has announced that it’s joining the SVG working group (about bloody time!).

The Groovy Internet Mood Meter in SVG
Embedding and updating scalable vector graphics in XHTML

XML is a very effective tool promoting data interchange between disparate systems. It’s a useful weapon that is slowly helping to quell the numerous “format wars” that have been fought throughout the history of IT. One of the ongoing but lesser known skirmishes concerns graphics. The plethora of current formats is confusing, costly, and simply not appropriate for the needs of advanced web applications. With most current formats, intricate drawings are far too large, difficult to produce and hard to use. Anyone who has battled with HTML image maps knows that the traditional image formats provide almost no opportunity for sophisticated scripting and interactivity. Scalable Vector Graphics (SVG) is an application of XML that solves all of the previously mentioned problems and also provides an open, standards-based alternative to the common graphics formats in use today. GrIMMiS is a very small Grails application I have cooked up that lets me use AJAX to dynamically update portions of an SVG drawing–and bring a peaceful end to the format wars.

A brief overview

This little dish has a bunch of tasty ingredients! There’s Grails, JavaScript (plain and using Grails’ support for prototype/AJAX), SVG (with a small amount of declarative animation), and XHTML, all rolled into one neat little savory package.

The Groovy Internet Mood Meter in SVG (GrIMMiS) itself is intentionally pretty simple: when you click on the smiley that best reflects your mood you influence the meter value, which is updated appropriately. The meter also periodically updates to reflect the ongoing cumulative mood of “the Internet.” With each update, GrIMMiS also retrieves a pithy saying or joke that varies depending on the overall mood.

Figure 1 shows GrIMMiS in all its glory running in Opera 10 beta 2, which currently (to me) seems to offer the best SVG support.

[images/image_1.png]

Figure 1: The GrIMMiS application

GrIMMiS mirrors the basics of a real SCADA HMI (Supervisory Control and Data Acquisition Human-Machine Interface) application that I developed for a client a long time ago–even before Jesse James Garrett’s coined the term ‘AJAX’ in the shower[1]. My original application was written in Java with plain JavaScript and had a fair amount of messy XML processing. In writing GrIMMiS, I wanted to see how Groovy and Grails would improve things. Of course, I found that the two technologies made things a lot simpler.

The SVG image

In preparation for the upcoming HTML5 standard, most modern browsers are supporting SVG more and more faithfully with each release. SVG is now available as a standard feature in Opera, Firefox, Safari and Chrome. It is found everywhere from JEE applications to mobile phone displays and frequently crops up in unexpected places. For example, much of the KDE4 Desktop’s user interface has been styled with SVG, and SVG can also be used to render UML diagrams in Oracle’s JDeveloper IDE.

Notably absent is Microsoft’s Internet Explorer [2]. In My (not-so) Humble Opinion, Redmond should be thoroughly ashamed of itself; SVG support is by far the most requested new feature for IE yet the community’s pleas have continually fallen on deaf ears. Adobe also should be ashamed of their “bait and switch” behavior in supplying the ‘anointed’ SVG plugin for IE and then unceremoniously pulling the plug, leaving the community high and dry with no alternative but to move to proprietary systems like flash/flex, and indirectly opening the door for Silverlight. Even worse, Adobe still supports SVG in applications like Adobe Reader and many other products but will no longer provide that support for IE. All is not lost however: there is evidence that Google is tiring of waiting for Microsoft to support SVG on IE and has launched its own SVG project called “SVG Web” [3]. Examotion is also developing the RENESIS plugin, which is intended to be a re-engineering of the Adobe plugin [4].

Enough background chit-chat! Listing 1 shows the source for the meter portion of the image that is shown in Figure 1.

Listing 1: The SVG Meter Image source code

There are a number of points of interest in this code (it sounds a little weird talking about the source-code for an image, doesn’t it, but that’s what we have here).

First, SVG is an application of XML and therein lies one of its great strengths: there are many XML tools and techniques available for processing XML, and used with SVG as well, making it easy to create and process images programmatically.

Secondly, the ability to use Cascading Style Sheets (CSS) is another great strength. Given that SVG is a first-class member of the large set of web technologies promulgated by the World Wide Web Consortium (W3C), one would expect nothing less than that CSS and (as we will see) JavaScript work without hassle.

Various elements of the drawing are self explanatory: circle, line, text . Note that several elements have been given unique values for their id attribute: this will simplify later processing.

The g element introduces a group. The important point of interest here is that the g element allows the specification of a transform attribute. Transforms can be used to rotate, translate, scale and skew any portion of an image. In this image, the line element within the ‘pointerGroup’ g element is initially defined as if it were at position 0,0 and pointing straight to the right, and then translated and rotated into its proper place and orientation. Although this may seem strange at first, this facility allows complex images to be built up from independent sub-parts that are ‘munged’ into place as needed. The power of all this becomes apparent when one realizes that the various sub-parts may have been produced in isolation, or perhaps generated as the result of a data query or other program action, without reference to any particular subsequent use. In this way, SVG is designed to make it easy to put together reusable ‘clip-art’-style libraries.

The three path elements draw the three wedges that make the dial of the meter. The key point of interest here is the mini “turtle graphics-style” language embedded into the d (path data) attribute. For example, the first line element decodes as:

M   absolute moveto         x, y

L   absolute lineto             x, y

A   absolute arc               rx ry x-axis-rotation short-or-long-way-flag clock-anticlock-flag x, y

The first circle makes use of the declarative animation provided by SVG. This is actually another W3C-originated technology called Synchronized Multimedia Integration Language (SMIL; pronounced “smile”). The animation rule declared here periodically manipulates the circle’s CSS fill property to provide a ‘busy’ indicator: every 5 seconds the circle will flash blue for 1.2 seconds. Not every browser currently handles SVG declarative animation; for instance, most fairly recent versions of Opera do but Firefox 3.5.1 does not.

The final point to note is the rule that defines z-order layering of the image elements: textually later elements appear on top of earlier ones.

If all this XML still leaves you unconvinced, consider this: the SVG document is less than 1.5Kb in size, while the equivalent PNG image is about 16Kb; more complex diagrams save even more. When you consider that Yahoo [5] estimated that roughly 80% of its web traffic involves image downloads, savings of one or two orders of magnitude start to look really attractive.

The XHTML ‘Container’ GSP page

In GrIMMiS, the SVG image source shown in Listing 1 is itself embedded into an XHTML document. This is the first true ingredient of the GrIMMiS Grails application because the XHTML is produced via a Groovy Server Pages (GSP) page, shown as Listing 2.

< %@ page contentType="application/xhtml+xml" %>




  GrIMMiS

  

  

  


GrIMMiS—the Groovy Internet Mood Meter in SVG


  
    How is the Internet feeling today?
  
  
    
      
    
  
  
    How are you feeling today?
  
  
    
      
    
    
    
      
    
    
    
      
    
    
  

transentia pty. ltd.; 13/July/2009

Listing 2: The Container XHTML GSP page

Listing 2 has a number of ingredients worth noting.

Since this is a GSP, I am able to use Grails taglibs like , and to reference resources (such as grimmis.css) in a nice neat way, and to cleanly use AJAX to talk back to the GrIMMiS controller’s various actions. This allows me to manipulate the remote meter values or to update the local SVG drawing.

Note that the page retrieves the actual SVG image via the GrIMMiS controller’s svg closure and includes it inline (by using ), rather than load it via a separate HTTP connection. This makes slightly more efficient use of the network, and having the image included inline in this fashion gives a unified Document Object Model (DOM). This greatly simplifies DOM processing and helps with cross-browser compatibility.

I have also made use of SVG’s ability to have nested SVG elements. This lets me build the actual image–without worrying how it might eventually be used in the page–while applying SVG-specific styling, and a unique ID to the outer SVG element.

I am using Grails’ prototype library for AJAX support and since I need the meter to regularly update to show the prevailing “mood out there,” I am hand-crafting a small bit of JavaScript/prototype code: using Ajax.PeriodicalExecutor in conjunction with a Ajax.Request for regularly (every POLL_TIME seconds) polling back to the GrIMMiS controller’s poll closure.

Polling is initiated by the client’s browser as it executes the XHTML page’s onload handler.

The JavaScript callback function ‘onSuccessFunction’ is the meat of this little recipe. It is called after every successful AJAX operation to process the retrieved data. As expected, The GrIMMiS controller sends a small XML document to the client in response to each AJAX request. onSuccessFunction takes this incoming document, extracts the relevant fragments from it (those that represent new values for the elements of the drawing that are to be updated), locates the now-superseded original fragments in the SVG document and replaces them. The meter is automatically redrawn after being updated.

Note how onSuccessFunction uses the suspendRedraw/unSuspendRedraw functions to turn document redraw on and off during updates; this can greatly improve browser performance.

onSuccessFunction is slightly trickier than I would really like: it is able to use prototype’s $(‘element’) syntax for addressing elements in the larger HTML document (and thus, the embedded SVG as well), but must use plain JavaScript to work with the elements of the XML fragment received from Grails. This is slightly painful, but bearable, and the various workarounds that exist (such as adding the received document’s elements into a hidden div and then shuffling them into place) may be worse than the cure in terms of readability.

The final point of interest on this page concerns the use of Grails’ tag to handle AJAX-style interaction with the controller when activated by the smiley-face image buttons that increase/decrease/‘neutralize’ the meter’s value.

The Grails controller

Listing 3 shows the single, very simple controller which is responsible for serving the SVG image and handling any interactions with the client browsers.

class GrIMMiSController {

  def drawingService

  def index = { redirect(action: svg) }

  // get full document
  def svg = {
    render(text: drawingService.svg(), contentType: 'image/svg+xml', encoding: 'UTF-8')
  }

  // get changes for document
  def poll = {model = null ->
    render(text: drawingService.poll(), contentType: 'text/xml', encoding: 'UTF-8')
  }

  def happy = {
    drawingService.happier()
  }

  def sad = {
    drawingService.sadder()
  }

  def neutral = {
    drawingService.neutraller()
  }

  // this interceptor means that happy, sad and neutral don't have to explicitly call poll()...DRY goodness
  //
  // must be elaborated after def poll = { } ... otherwise poll will be null
  def afterInterceptor = [action: poll, only: ['happy', 'sad', 'neutral']]
}

Listing 3: The Grails controller

This controller is a thin adapter that mostly delegates to the injected drawingService instance.

The major point of interest here is the use of an afterInterceptor. Without this interceptor, each happy, sad and neutral smiley-face image buttons would need to look something like:

def action = {

// body

poll()

}

With this interceptor in place, these actions are much simpler and more DRY.

Note that the poll action can be invoked either as a result of an incoming HTTP request as an action in its own right or as the callback closure associated with the defined afterInterceptor. In the former case, no parameter is passed into the closure, while in the latter case a parameter is passed in by the actual interceptor code (although it isn’t used in this example). The use of a default parameter value helps avoid redundancy.

The single tricky point here is that afterInterceptor must be declared after poll has been fully declared, otherwise poll will be null at the point of declaration, and Grails will complain when one of the designated actions is invoked.

The Grails service

Most of the meat of the Grails application lives in the service class and a couple of small subsidiary classes.

Listing 4 shows the DrawingService.groovy file.

import groovy.util.XmlNodePrinter

class DrawingService {

  boolean transactional = false

  static scope = "singleton"

  def jokeService

  final MIN = 0
  final MAX = 180
  final QMAX = MAX / 4
  final HMAX = MAX / 2
  final BOTTOM_QUAD = MIN..
    def start = System.currentTimeMillis()
    log.debug "Initialise START: $file"
    log.debug "Parsing..."
    this.document = new XmlParser().parseText(new File(file).text)
    log.debug "Isolating important elements for updating..."
    this.pointerRotateTransform = document.g.g.g.find { it.'@id' == 'pointerRotateTransform' }
    log.debug "Found pointerRotateTransform: $pointerRotateTransform"
    this.joke = document.g.text.find { it.'@id' == 'joke' }
    log.debug "Found joke: $joke"
    this.reading = document.g.text.find { it.'@id' == 'reading' }
    log.debug "Found reading: $reading"
    log.debug "Initialise END; (${System.currentTimeMillis() - start}ms)"
  }

  // get full document
  def svg = {->
    synchronized (this) {
      toSVG document
    }
  }

  def poll = {->
    synchronized (this) {
      joke.value = message()
      reading.value = fint(mood.value.current)
      pointerRotateTransform.'@transform' = "rotate(-${fint(MAX - mood.value.current)})"
      toSVGFragment pointerRotateTransform, joke, reading
    }
  }

  def happier = {->
    synchronized (this) {
      mood.happier()
    }
  }

  def sadder = {->
    synchronized (this) {
      mood.sadder()
    }
  }

  def neutraller = {
    synchronized (this) {
      mood.neutraller()
    }
  }

  private fint = { v ->
    String.format("%d", (int)v)
  }

  private message = {->
    switch (mood.value.current) {
      case TOP_QUAD: return "Wow...what's in the collective waters today :-)"
      case HIGH_QUAD: return "The world seems to be going swimmingly...so far, anyway."
      case HMAX: return "Keep on in that Groove, guys."
      case LOW_QUAD: return "You need a joke: ${jokeService.joke()}"
      case BOTTOM_QUAD: return "Oh dear! We all need a group session with a psycho-therapist, it seems."
      default: return "???"
    }
  }

  private toSVG = {n ->
    def writer = new StringWriter()
    def xnp = new XmlNodePrinter(new PrintWriter(writer))
    xnp.setNamespaceAware true
    xnp.setPreserveWhitespace false
    xnp.print(n)
    writer.getBuffer().toString().trim()
  }

  private toSVGFragment = {Object ... elems ->
    def s = new StringBuilder("")
    elems.each { s << toSVG(it) }
    s << ""
    s
  }
}

class Mood {
  private final BIG_JUMP = 20
  private final SMALL_JUMP = 10

  private final randFromOne = RandomUtils.&random.curry(1)
  private final bigRandom = randFromOne.curry(BIG_JUMP)
  private final smallRandom = randFromOne.curry(SMALL_JUMP)

  final value

  Mood(m) {
    value = new RangedValue(range: m.range)
  }

  def happier = {->
    value.current += bigRandom()
  }

  def sadder = {->
    value.current -= bigRandom()
  }

  def neutraller = {->
    // force value toward center
    switch (value.current) {
      case { it > value.middle }: value.current -= smallRandom(); break
      case { it < value.middle }: value.current += smallRandom(); break
      default: break /* do nothing */
    }
  }
}

class RangedValue {
  final range
  final middle
  def current

  RangedValue(m) {
    this.range = m.range
    this.middle = this.current = mid(m.range)
  }

  public void setCurrent(v) {
    switch (v) {
      case { it > range.to }: current = range.to; break
      case { it < range.from }: current = range.from; break;
      default: current = v; break
    }
  }

  private static mid = {r ->
    return (r.to - r.from) / 2
  }
}

Listing 4: The Grails service and associated classes

This service class has a number interesting points worth noting.

The initialise closure is defined to permit configuration of which SVG file to use via the Grails bootstrap, as shown in Listing 5.

import org.codehaus.groovy.grails.commons.ApplicationHolder as AH

class BootStrap {

  def drawingService

  def init = {servletContext ->
    def file = "${AH.application.parentContext.getResource('images/').file}${File.separatorChar}GrIMMiS.svg"

    drawingService.initialise file
  }
  def destroy = {
  }
}

Listing 5: The modified Grails Bootstrap.groovy class

Note the use of Grails’ ApplicationHolder class, which lets us discover the location of the images folder. This is much better than using some hard-coded path.

The initialise closure also pre-resolves the various dynamic elements of the drawing so that they can be easily manipulated by other code later. I’ll be the first to agree that this is a micro-level performance optimization, but one that I found became important as the size of an image (and also the number of dynamic elements) increased.

As the saying goes [6]: “Nothing makes you want Groovy more than XML”, and initialise lets you see Groovy’s superb XML-handling capabilities first-hand. For example, the following navigates through the SVG document to find the unique g element with id attribute value equal to ‘pointerRotateTransform’:

document.g.g.g.find { it.'@id' == 'pointerRotateTransform' }

The first of the two of the major methods in Listing 4 is svg. This method is responsible for writing the entire SVG document as a string (which is of course eventually returned to the client browser as part of the XHTML document produced by the GSP shown in Listing 2).

Note that svg always writes the most up-to-date state of the SVG document so that a new client retrieves an accurate representation of the meter’s value right from the start. The alternative approach would be to write the original (as-stored-on-disk) image document to the client and then wait for a refresh to occur. Although possibly simpler and less server-side resource intensive, I have found this alternative to be a generally unsatisfactory approach. It becomes even more so as the image increases in size and complexity (remember that GrIMMiS after all is only a trivial example application), or the drawing poll rate becomes longer.

The second major method in the service is poll. This method is responsible for updating the underlying SVG document to reflect the changing mood of “the Internet,” as well as for generating an XML document fragment containing only the updated elements of the image. An example of the document fragment produced is shown here:


  
    
  
  
    Oh dear! We all need a group session with a psycho-therapist, it seems.
  
  
    44

The method poll makes use of a simple helper method called toSVGFragment. If you look at toSVGFragment, you can see that I have made good use of Groovy’s simple varargs facility to define a method capable of handling an arbitrary number of SVG elements in one invocation. This again helps the DRY-ness of the code and aids development and maintainability: the need to render an additional dynamic element is easily met by simply adding that element to the parameters passed toSVGFragment. There is no need to take alternative, less direct, approaches such as passing parameters as lists or maps, or to define toSVGFragment to take some arbitrary number of (mostly default) parameters.

DrawingService makes effective use of Groovy’s ranges. Listing 6 (excerpted from Listing 4) shows how I split the range from 0 to 180 into four equal-sized, non-overlapping subranges that corresponding to the angle of the meter’s pointer. These subranges are then used in conjunction with Groovy’s excellent switch statement to determine an appropriate message to display.

final MAX = 180
final QMAX = MAX / 4
final DMAX = MAX / 2
final BOTTOM_QUAD = 0..
  switch (constrain(mood)) {
    case TOP_QUAD: return "Wow...what's in the collective waters today :-)"
    case HIGH_QUAD: return "The world seems to be going swimmingly...so far, anyway."
    case DMAX: return "Keep on in that Groove, guys."
    case LOW_QUAD: return "You need a joke: ${jokeService.joke()}"
    case BOTTOM_QUAD: return "Oh dear! We all need a group session with a psycho-therapist, it seems."
    default: return "???"
  }
}

This excerpt also shows how the JokeService is used. Listing 6 shows this trivial class.

class JokeService {

  boolean transactional = false

  private jokes = [
          /Q: What's yellow and dangerous? A: shark-infested custard!/,
          'The only culture in Australia is found in its youghurts.',
          'When someone emigrates from NZ to Oz, the IQ of both countries goes up.'
  ]

  def joke = {->
    return jokes[RandomUtils.random(0, jokes.size())]
  }
}

Listing 6: The JokeService class

Of course, a more sophisticated implementation would pull jokes from a database or perhaps an Internet WebService.

Fancy some curry? In keeping with the slightly gastronomic theme to this article, the Mood class makes use of a technique called currying.

Currying is a functional programming technique that lets you create a new closure based on an existing closure where one parameter is pre-loaded and fixed with a given value (there is a long and very boring formal definition of currying on Wikipedia [7], but this will do).

Rather than having calls to the general-purpose RandomUtils.random closure dotted around my code using (nearly) the same set of parameters at each call, you can see how I successively fix parameters in place to obtain a group of specialized closures perfectly adapted for Mood’s specific needs:

final randFromOne = RandomUtils.&random.curry(1)

final bigRandom = randFromOne.curry(BIG_JUMP)

final smallRandom = randFromOne.curry(SMALL_JUMP)

The specialized closures are used throughout the DrawingService class. This is a very powerful technique that can help simplify your code and improve its readability.

Time for a bit of pop philosophy! The Mood class embodies GrIMMiS’ business rules, such as they are. The philosophy embodied here is that although one is easily capable of saying “I am emotional,” it is extremely hard to definitely quantify emotion. Mood thus uses a random number for each operation. When a user hits the happy or sad smiley button, a value taken from a large random range is used (the thinking is that client X could be very happy, while client Y only mildly sad). A user that presses the neutral smiley face is presumed to be less subject to wild “mood swings” and so the value used is taken from a smaller random range.

One more point of interest in this code is the use of Groovy’s ‘&’ operator: this takes a reference to a method and treats it as a closure (so that it can be subsequently curried, for instance, as shown in the above excerpt).

For completeness, Listing 7 shows RandomUtils, a tiny helper class in src/groovy.

public class RandomUtils {
  private static final rg = new Random()

  public static int random(off, lim) {
    off + rg.nextInt(lim)
  }
}

Listing 7: The RandomUtils class

The following snippet taken from the RangedValue class shows how I have used an explicit setter for the derived property current (the Groovy automatically-supplied getter continues to be used as normal). The supplied setter method ensures that current is maintained within the bounds of the range defined for it at construction.

def current

public void setCurrent(v) {
  switch (v) {
    case { it > range.to }: current = range.to; break
    case { it < range.from }: current = range.from; break
    default: current = v; break
  }
}

Both JokeService and DrawingService are marked as non-transactional. This is clearly appropriate and may help to conserve server-side resources.

The final point to note from Listing 4 concerns the various synchronized blocks that you will see in the code. In contrast to a controller, which typically has many active instances (usually one instance per request), a service by default is a singleton: there is only one instance regardless of how many controllers are trying to use it. This is quite appropriate here: the drawingService should only maintain a single SVG drawing, regardless of however many clients are using GrIMMiS. The singleton nature of drawingService does mean that many operations (such as updating the current mood value, or modifying the server-side SVG diagram to reflect that mood) should be carried out in a serialized manner to avoid possible mis-correspondence of mood value and rotation transform value or message.

Wrapping Up

I hope you enjoyed taking a look at the recipe for this little entrée.

Recall that I started to make this dish because I wanted to see if Groovy and Grails would make it easier to build an application similar to one I built for a client several years ago using plain Java. It is clear that Groovy’s superb XML support and simpler file handling has improved things. It is also clear that features like varargs and currying have let me produce much more maintainable code. By providing a clean web-based environment, with interceptors and effective support for JavaScript libraries (such as prototype), Grails has made it possible for me to write much better code than before. In general, even when taking into account the significantly smaller size and reduced capabilities, this application has a cleaner, more comprehensible feel to it.

I wish I’d had these “Gr8 technologies” available to me many years ago!

Learn More

The origin of the term ‘AJAX’, http://en.wikipedia.org/wiki/Ajax_(programming)
SVG support matrix for the various browsers, http://www.codedread.com/svg-support.php
Google SVG Web, http://code.google.com/p/svgweb/
RENESIS SVG Plugin for IE, http://www.examotion.com/index.php?id=product_player
Yahoo User Interface Blog, http://yuiblog.com/blog/2006/11/28/performance-research-part-1/
Stuff I’ve learned recently, http://kousenit.wordpress.com/2008/03/12/nothing-makes-you-want-groovy-more-than-xml/
Wikipedia: Currying, http://en.wikipedia.org/wiki/Currying

Batch Processing with Spring Batch

bob — Wed, 03 Feb 2010 10:44:23 +0000

I have blathered on about Spring Batch a few times in the past.

The June 2009 edition of GroovyMag carried my article on how to use Spring Batch with Groovy; it’s republished (with permission) here. This was my second article for GroovyMag, I have also republished the first one, on Spring Integration with Groovy.

As I lurk on the Groovy and Grails mailing lists, I see a real need “out there” for this sort of infrastructure. Hopefully, this article will contribute to a small improvement in awareness. Writing code should, after all, be the last resort and not the first…

The source code is available, of course!

Batch Processing with Spring Batch
Dealing with Large Volumes of Data using Spring Batch and Groovy

Even though a major focus of modern ideas such as Service Oriented Architectures and Software As A Service is to facilitate and enhance live interactions between systems, batch processing remains as important and widely-used as ever: practically every significant project contains a batch processing component. Until the arrival of Spring Batch (version 1.0 was released in March, 2008), no widely available, open source, reusable architecture framework for batch processing had existed; batch processing had always been approached on an ad-hoc basis. This article will examine how Groovy joins with Spring Batch to ease the pain of dealing with large data sets.

A Brief Overview of Spring Batch

Spring Batch (SB) is a relatively new member of the Spring family of technologies. There is no better introduction than this excerpt from the documentation (see the “Learn More” section for the URL):

Spring Batch is a lightweight, comprehensive batch framework designed to enable the development of robust batch applications . . . Spring Batch builds upon the productivity, POJO-based development approach, and general ease of use capabilities people have come to know from the Spring Framework, while making it easy for developers to access and leverage more advance [sic] enterprise services when necessary . . . Spring Batch provides reusable functions that are essential in processing large volumes of records, including logging/tracing, transaction management, job processing statistics, job restart, skip, and resource management. It also provides more advance technical services and features that will enable extremely high-volume and high performance batch jobs though optimization and partitioning techniques. Simple as well as complex, high-volume batch jobs can leverage the framework in a highly scalable manner to process significant volumes of information.

SB provides a great deal of out-of-the-box functionality: very flexible adapters for reading from flat files, facilities for dealing with JMS queues, JDBC-based database adapters (of course), along with a simple workflow ability allowing conditional, repetitive and parallel processing. Sophisticated error handling/recovery capabilities and simple job control round out the package. Following the standard Spring Framework convention of “don’t reinvent the wheel,” SB does not include a complete set of scheduling/job control tools, but works instead in conjunction with existing Spring-friendly schedulers such as Quartz and Tivoli. Over time, SB will make more use of other members of the Spring family and in particular Spring Integration, and this pairing in particular should make for a formidable partnership.

SB was seeded and driven by technology and developers from Accenture–as well as SpringSource and the general Open Source community–and so claims to represent the distillation of a fair bit of experience with “real world” needs and situations.

A Small Example Application

Because SB is rooted in the standard Spring Framework, it is quite compatible with Groovy (and is also simple to integrate into Grails). As a demonstration, I’ll implement a simple batch processing job, as follows: read and parse an input data file (where each record is formatted according to a custom multiline format); validate each record (rejecting invalid records and writing the relevant record to a dump file); apply a specific transformation to each remaining (valid) record; and finally, write the valid, transformed data into an XML file.

Figure 1 provides a simplified, high-level view of the application’s overall workflow.

Figure 1: Simplified, high-level view of the workflow in the example application

Figure 1 shows how an application initiates a SB job. The job is composed of a number of steps, which are themselves composed of a number of substeps or tasklets. Tasklets are the building blocks of the application and may read/write/validate/transform or otherwise ‘munge’ the data.

Even though this is a very simple batch-processing task, there are a number of tricky areas. Consider the input data file format (an example is shown in Listing 1): this format contains data split into logical sections (customer ID, contact information, credit card information, customer balance), with one section to a line. Each line is a CSV-formatted record but some fields may themselves contain fixed-width records. The whole record is “book-ended” by BEGIN/END markers, each of which must have the same record number. The first two lines of the file are free-form commentary.

; this is a nasty file format
; it's going to be a challenge to process!
BEGIN0000000001
CUST,9876543210
CONT,0416123456,0712340987,fred@nowhere.com
CARD,visa:1234 1234 4321 4321:000
BAL,1000.00
END0000000001

Listing 1: Example input data record

Processing this data file is going to be quite a challenge and it is probably worth taking some time to consider how you would tackle this task in plain Java or Groovy.

It is safe to assume that the input data will contain numerous errors that makes validation and error handling a priority consideration for this application. Validation is a common necessary chore that is not difficult but is tedious and error-prone; by relying on standard Spring technologies, SB helps simplify this task.

Another minor challenge is concerned with producing the output XML document. Listing 2 shows how the record given in should be written.



  
    9876543210
    0416123456
    0712340987
    fred@nowhere.com
    visa
    1234 1234 4321 4321
    000
    1010.0

Listing 2: The resultant XML-formatted data record

As is always the case with infrastructures and frameworks, one often gets the feeling of overkill when working with a simple example such as the one in this article. Keep in mind however that as a problem gets bigger, a framework becomes more and more necessary. Remember also that SB was written with these large problem spaces in mind and so you may have difficulty seeing SB’s full potential using this one simple example application.

The Driver Application

The example application uses a small command-line application to kick off the real batch job. SB actually provides a simple command-line launcher to do this, but it is instructive to see how to deal with a SB job by hand. As Listing 3 shows, the launcher is very simple and is a typically simple Spring Framework-aware application.

package au.com.transentia.sb

import o.s.batch.core.JobParametersBuilder
import o.s.context.support.ClassPathXmlApplicationContext

public class SpringBatch {

  public static void main(String[] args) {
    def context =
      new ClassPathXmlApplicationContext(['applicationContext.xml', 'job.xml'] as String[], true)

    def jobLauncher = context.getBean('jobLauncher')

    def job = context.getBean('job')

    def adjustmentPercent = 1.01D;

    def jobExecution =
      jobLauncher.run(job,
        new JobParametersBuilder().
          addDouble("adjustment.percent", adjustmentPercent).toJobParameters())
    jobExecution.with {
      println """
Job: $jobId
StartTime: $startTime; EndTime: $endTime
Duration: ${endTime.time - startTime.time} ms
Status=$exitStatus
"""
      stepExecutions.each { println "STEP: $it" }
    }
  }
}

Listing 3: The Groovy driver application

Note: throughout this article, the package name prefix ‘org.springframework’ is abbreviated to ‘o.s’ to reduce line length and aid formatting and readability.

Of interest here is the creation of the Spring application context. The application is looking for two files on its classpath: applicationContext.xml defines the boilerplate SB infrastructure and job.xml defines the SB job itself. This is a standard Spring development technique. I’ll look at these files in more detail later.

The job instance is created by the Spring application context and is looked up by the application itself; this is the real meat of the batch definition, as we shall see.

The jobLauncher instance obtained from the application context is part of SB. As the name suggests, it is concerned with mediating access to a SB job instance. In this case, it will control execution of the job instance defined in job.xml and retrieved by the application. The joblauncher.run() method returns a SB jobExecution instance that allows the application to determine the state of the associated job and its composite steps.

JobParametersBuilder provides a way of defining a map of parameters that can be passed into a job and subsequently accessed by the various steps and tasklets.

Listing 4 shows (slightly edited) the application in action.

Job: 0
StartTime: Fri May 01 16:25:50 EST 2009; EndTime: Fri May 01 16:25:54 EST 2009
Duration: 3117 ms
Status=exitCode=COMPLETED;exitDescription=
STEP: StepExecution: id=0, name=startupStep, status=COMPLETED,. . .
STEP: StepExecution: id=1, name=processStep, status=COMPLETED,. . .
STEP: StepExecution: id=2, name=endingStep, status=COMPLETED, exitStatus=COMP. . .

Listing 4: Output from executing the application

Boilerplate Configuration

SB requires a certain amount of standard configuration to be put in place in preparation for job execution.

Listing 5 excerpts the relevant portion of the applicationContext.xml file that contains this configuration.

Listing 5: Minimal-functionality SB configuration

This is the least sophisticated SB configuration possible. This configuration establishes a no-op transaction manager and a pure in-memory jobRepository. The latter configuration option means that no distribution, persistence or job restart capabilities are available. For the purposes of this application, this configuration is sufficient.

In this example application, all processing is sequential and synchronous with the application; however, it is possible to configure the jobLauncher instance to execute a job asynchronously to the application. An asynchronous configuration would be appropriate if using SB in conjunction with a Grails/AJAX application, which could initiate a job and then poll for status and update a visual progress indicator until the job completes.

The Job Definition

The keystone of this application is actually the job.xml application context file. Because this is quite long, I will go through it in sections.

Note: The listings shown here have been edited and excerpted to save space (while hopefully remaining clear).

The full source code for this example is supplied with this edition of GroovyMag, of course.

Listing 6 shows the place where it all begins: the job definition.


  
    
  
  
    
      
        
          
        
        
          o.s.batch.item.file.FlatFileParseException
          o.s.batch.item.validator.ValidationException

Listing 6: The job definition

This definition constructs a three-stage processing pipeline.While the first and last steps merely print a status message, the middle step (with id=processStep) is the most important and what I focus on here. The processStep step identifies the various input and output processors and also defines the intermediate transformations/processes that will be executed on each record.

An important SB concept introduced here is that of a chunk. A chunk defines the processing that is to be done within a transaction boundary: a batch of records that may be written/rolled-back as a whole, for instance. For this application, each record is treated as constituting a separate chunk and so error handling, etc., is done on a per-record basis.

The batch:streams, batch:listeners and batch:skippable-exception-classes configuration elements are all related to the way that erroneous input records are handled. This will be looked at later.

Processing Step One: Input

Listing 7 defines the itemReader bean (and some of the necessary associated configuration), which deals with reading and parsing the multiline record from the input data file.

Listing 7: Input record handling

Take your time to read through this code; the itemReader is quite a sophisticated piece of infrastructure.
As you piece things together, you will see how almost everything is delegated to standard SB classes: the actual reading of the file and skipping the comment lines is deferred to a FlatFileItemReader, and the recognition and handling of the various logical sections in a record are handled by the PatternMatchingCompositeLineTokenizer class (which itself defers to a number of line tokenizers). In fact, the only custom activity here is the mapping of the parsed data to a simple application-specific class via the au.com.transentia.sb.MultilineRecordReader class. A tremendous amount of processing is being performed here with very little development effort.

Note how the use of abstract parent definitions (e.g., fixedLineTokenizer), makes it possible to write clearer configurations for several elements (e.g., beginLineTokenizer/endLineTokenizer). This is a standard Spring technique that helps to keep the configuration file DRY (i.e., without unnecessary repetition).

Again, consider how much effort would be involved if you had to do all this by hand.

Listing 8 and Listing 9 shows the MultilineRecord and MultilineRecordReader classes.

package au.com.transentia.sb

public class MultilineRecord {
  String sequence
  String endSequence

  Long id

  String mobile
  String landline
  String email

  String provider
  String number
  String security

  BigDecimal balance

  @Override public String toString() { …elided… }
}

Listing 8: The MultilineRecord class

package au.com.transentia.sb

import …elided…

public class MultilineRecordReader implements
  ItemReader, ItemStream {
  private FlatFileItemReader flatFileItemReaderDelegate

  public MultilineRecord read() throws Exception
  {
    MultilineRecord mlr = null

    // flags to indicate the presence of component lines
    def customerSeen = false;
    def contactsSeen = false
    def ccSeen = false
    def balSeen = false

    def line
    while (line = this.flatFileItemReaderDelegate.read())
    {
      String prefix = line.readString(0);
      switch (prefix)
      {
        case 'BEGIN':
          mlr = new MultilineRecord(sequence: line.readString(1))
          break

        default:
          Assert.notNull(mlr, "MultilineRecord not yet intialised")
          switch (prefix)
          {
            case 'CUST':
              mlr.id = line.readLong(1)
              customerSeen = true
              break

            case 'CONT':
              mlr.with {
                mobile = line.readString(1)
                landline = line.readString(2)
                email = line.readString(3)
              }
              contactsSeen = true
              break

            case 'CARD':
              mlr.with {
                provider = line.readString(1)
                number = line.readString(2)
                security = line.readString(3)
              }
              ccSeen = true
              break

            case 'BAL':
              mlr.balance = line.readBigDecimal(1)
              balSeen = true
              break

            case 'END':
              // check all record fields seen
              Assert.isTrue(mlr && customerSeen &&
                            contactsSeen && ccSeen && balSeen,
                  "Incomplete Record Found")
              mlr.endSequence = line.readString(1)
              return mlr
              break
          }
      }
    }
    null
  }

  … elided …
}

Listing 9: The MultilineRecordReader class

The MultilineRecordReader class is responsible for allocating the various fields of each tokenized line (offered as SB FieldSets) to a single new instance of a MultilineRecord. It also performs some minor validation to ensure that there are no missing logical sections. Note how Groovy’s versatile switch statement and the use of Groovy’s enhanced Object with method makes the processing much clearer than it would be in plain Java.

Processing Step Two: Output

As every first-year CompSci student learns, after input comes processing, followed by output. Naturally, I am not going to follow this sequence! In an attempt to achieve a less convoluted narrative, I’ll now look at output processing (the third stage of the processStep step). Listing 10 shows the requisite configuration for the itemWriter.




  
    
      
        java.lang.String

Listing 10: Output XML processing

The itemWriter configuration is quite straightforward. Output handling is actually delegated to a marshaller instance. In this case, the Spring OXM project is brought to bear to simplify XML generation. The XStreamMarshaller is only very minimally configurable (but quite performant…the age-old tradeoff): it is possible to render the sequence field as an XML attribute and not to render the endSequence field at all, but that is pretty much all the configuration possible.

Processing Step Three: Validation and Transformation

Now that we’ve seen the mechanics of writing a file, it is time to move on to (or more accurately: back to) the middle processing step.

Listing 11 shows the configuration of the processing tasks.


  
    
      
      
    
  



  



  
    
      
        
          
        
      
      
        
           0 :
  'id field must be a natural number' }
{ endSequence : ? is not blank :
  'end sequence number field missing' }
{ sequence : ? is not blank :
  'begin sequence number field missing' }
{ endSequence : endSequence == sequence :
  'mismatched begin/end sequence numbers' }
{ mobile : match('\\d{10}',?) == true :
  'mobile field must be 10 digits' }
{ landline : match('\\d{8,10}',?) == true :
  'landline field must be 8-10 digits' }
{ provider : ? in 'amex', 'visa', 'macd' :
  'card provider field should be one of "visa", "amex" or "macd"' }
{ number : match('\\d{4}[ ]\\d{4}[ ]\\d{4}[ ]\\d{4}',?) == true :
  'card number field must match the format "xxxx xxxx xxxx xxxx"' }
{ number : luhn(?) == true :
  'card number luhn check failed' }
{ security : match('\\d{3}',?) == true :
  'card number field must be 3 digits' }
{ email : ? is not blank :
  'email field missing' }
{ email : email(?) == true :
  'email field is not a valid email address' }
{ balance : ? is not blank :
  'balance field missing' }
]]>

Listing 11: Configuration of processing tasks

For this application, the middle processing part of the pipeline is itself a two-stage composite process: first comes validation (and possibly rejection of invalid data), followed by transformation. This is configured through the compositeItemProcessor bean (which is referenced by the processStep step in ).
SB allows the use of any of the available Spring-compatible validation systems. For this example I have chosen to use the Valang Spring module, because of its clear declarative nature.

The validator bean contains a plaintext valang property which defines a series of expressions that should be evaluated against the various properties of the bean to which it is applied. For example:

{ id : ? > 0 : 'id field must be a natural number' }

If the given expression evaluates to false, validation fails and the associated message is added to the list of errors being maintained by the validator bean.

While Valang provides a series of standard functions (such as email, which checks to ensure that a field contains a valid email address), it cannot account for all possible requirements. It can be augmented via the customFunctions property, however, and it is this ability that allows me to define an application-specific function. To illustrate this, I’ll introduce a check for the validity of the credit card number field (using the so-called Luhn function; see “Learn More” for a reference to how this works), as is shown in Listing 12.

package au.com.transentia.sb

import …elided…

public class LuhnFunction extends AbstractFunction {

  public LuhnFunction(Function[] arg0, int arg1, int arg2) {
    super(arg0, arg1, arg2);
    definedExactNumberOfArguments(1);
  }

  @Override
  protected Object doGetResult(Object target) throws Exception {
    def str = getArguments()[0].getResult(target).toString()
    isValid(str)
  }

  public static boolean isValid(String cardNumber) {
    def sum = 0
    def addend = 0
    def timesTwo = false

    cardNumber.replaceAll(' ', '').each {dc ->
      def digit = Integer.valueOf(dc)
      if (timesTwo)
      {
        addend = digit * 2
        if (addend > 9)
          addend -= 9;
      }
      else
        addend = digit
      sum += addend
      timesTwo = !timesTwo
    }

    (sum % 10) == 0
  }
}

Listing 12: Luhn function class

This allows the use of the luhn() function as if it were a built-in Valang function:

{ number : luhn(?) == true : 'card number luhn check failed' }

Valang is a powerful and effective validation framework. Its most important feature is probably that since it uses “near natural language” configuration, an application’s validation rules can be reviewed and changed by any appropriate product owner or business representative and this can make for a higher quality product. Valang is a standalone framework and well worth further study (see the “Learn More” section for the URL).

Refer back to Listing 6. You will see that in the event of a ValidationException, processing of a chunk is skipped. This application registers a listener for this situation that simply writes the offending record to a configured itemWriter. Listing 13 shows the appropriate class.

package au.com.transentia.sb

import org.springframework.batch.core.listener.SkipListenerSupport

public class SkipListener extends SkipListenerSupport {

  def writer

  @Override
  public void onSkipInProcess(MultilineRecord item, Throwable t) {
    writer.write ( [ item ] )
  }
}

Listing 13: Error handling skip listener class

The configured itemWriter has an associated line aggregator that performs preprocessing before the actual write takes place. In this case, the PassThroughLineAggregator class simply performs a toString operation on the presented item, as Listing 14 shows.

package au.com.transentia.sb

import org.springframework.batch.item.file.transform.LineAggregator

public class PassThroughLineAggregator implements LineAggregator {
  public String aggregate(MultilineRecord item) {
    item.toString()
  }
}

Listing 14: PassThroughLineAggregator class

The second part of the composite processing step shown in deals with transforming the now-valid data record. This particular transformation represents a business rule: as each record is transformed, its balance should be adjusted by a certain percentage. This requires a small piece of Groovy, the EmbiggenProcessor. Listing 15 shows this simple class in its entirety.

package au.com.transentia.sb

import org.springframework.batch.item.ItemProcessor

public class EmbiggenProcessor
  implements ItemProcessor {

  def percent

  public MultilineRecord process(MultilineRecord mlr) throws Exception {
    mlr.balance *= percent

    mlr
  }
}

Listing 15: The EmbiggenProcessor class

If you refer back to Listing 3 and Listing 11, you will see how the percent value is injected into the EmbiggenProcessor class from the application via the job.xml file.

Performance

To paraphrase Benjamin Disraeli, first Earl of Beaconsfield: “there are lies, damn lies and performance measurements.” Since many batch jobs deal with very large data sets, the performance of SB is bound to be of paramount interest to some. I am a coward! I am going to merely dip my toe into this potentially turbulent topic and just let you know that on my laptop, the complete processing of 42,500 records took 92,381 ms. That makes about 2.17 ms per record. Not too shabby in my opinion–but of course, your mileage may vary.

Wrapping Up

You’ve now walked through a complete SB application. A large proportion of the application is declarative configuration. I like this style of working: it reduces the amount of ‘real’ coding that is required and thus minimizes the opportunity for error. For those parts of the problem space that are not directly covered by the standard components, Groovy has proved to be a very effective tool; with minimal effort and coding, Groovy has allowed me to very effectively concentrate on creating a clear solution, which also minimizes the opportunity to introduce bugs.

I continue to find it impressive that Groovy—a dynamic language—can still effectively work with highly-typed interfaces such as those exposed by SB: checked exceptions, strongly typed parameter lists, and even generic classes can be handled with ease. At the same time, Groovy can let me work with frameworks such as Valang without forcing me to deal with ‘nasties’ such as adapter classes, proxies, out-of-process adapters, etc. This means that an inveterate Java programmer like myself can continue to apply his existing skillset—with its hard-won collection of lessons learned, tricks and tips picked up over time—while also taking advantage of the productivity and ease of use of Groovy’s modern dynamic language. In my not-so-humble opinion, this is important and will surely contribute to a rapid increase in Groovy’s popularity.

Learn More

Spring Batch: http://static.springsource.org/spring-batch/
Spring Valang module: https://springmodules.dev.java.net/
Spring OXM: https://springmodules.dev.java.net/
The Luhn Function: http://www.merriampark.com/anatomycc.htm

MarkupBuilder, How Do I love Thee…

bob — Wed, 03 Feb 2010 01:09:43 +0000

In oh, so many ways!

Groovy’s MarkupBuilder class can really clean up your code.

import groovy.xml.*

def out = new StringWriter()
def b = new MarkupBuilder(out)
b.table(id: 'test', border: 0) {
  tbody {
    tr {
      td "hello"
      td(id: 'hello_id', 'hello')
      td(id: 'test', /hello/)
      td(id: 'esc') { mkp.yield ' ' }
      td(id: 'unesc') { mkp.yieldUnescaped ' ' }
      td(id: 999) { mkp.yield 999 }
      td(id: 99) { mkp.yieldUnescaped 99 }
      td(id: 88) { mkp.yieldUnescaped 'hello' }
      td(id: 77) { mkp.yieldUnescaped "hello2" }
      td(id: 66) { mkp.yieldUnescaped """hello3""" }
      // gives compilation error "unexpected token: } at line: 19, column: 48":
      //   td(id: 55) { mkp.yieldUnescaped /hello4/ }
      td(id: 55) { mkp.yieldUnescaped "" + /hello4/ }
    }
  }
}

println out.toString()

This produces:


  
    
      hello
      hello
      hello
       
       
      999
      99
      hello
      hello2
      hello3
      hello4

This was just another of those “external brain dump” posts…I didn’t want to loose my memory and there are a few subtle points (or at least a few things that I found out by trial-and-error). There’s not too much other stuff around.

Solving the Enterprise Integration Puzzle with Spring Integration

bob — Mon, 25 Jan 2010 12:28:45 +0000

Back in May 2009, I mentioned that I had an article published in GroovyMag.

GroovyMag is very nice in that it allows the author to republish their work after a ‘decent’ interval has passed (either 60 or 90 days, can’t quite remember at the moment). That period is well and truly passed for this article so here it is in its full resplendency. Enjoy!

[edit]
Forgot to make the source available when I first republished this article. Silly me. All fixed now!

Solving the Enterprise Integration Puzzle
Getting Started with Spring Integration

Groovy is frequently promoted as “the scripting language for the JVM.” Grails is often described as being “great for smaller, quick-to-build web applications.” Neither of these pieces of perceived wisdom really gives the Groovy/Grails combination its due: a powerful tool in the Systems Integrator’s armory. This article will show how Groovy and Grails–in conjunction with SpringSource’s up-and-coming “Spring Integration” project–can make complex systems integration tasks as easy as building a standalone web application.

A Brief Overview of Spring Integration

Systems integration is frequently likened to attempting to piece together a badly made jigsaw puzzle. The end goal is often quite fuzzy; there may be several potential solutions to consider; there may be many conflicting approaches to evaluate; and when all is said and done, sometimes all that can be done is to say “To heck with the Grand Plan!” and ‘adapt’ the puzzle using a pair of nice sharp scissors.

Spring Integration (SI) is a recent project within the Spring stable that provides numerous tools to make the systems integration task less frustrating. SI builds on several existing Spring stablemates, but provides a unified way of configuring these components and linking them together. According to SI, all message endpoints (which act with and on behalf of application components) interact via messages sent over various flavors of messaging channels, perhaps undergoing various transformations along the way. In addition, external resources (files, JMS-based queueing systems, SOAP-based WebServices, etc.) interact with these channels via simple-to-configure adapters.

SI imposes very little onto the developer; almost everything in the system can be implemented using simple POG/JOs (Plain Old Groovy/Java Objects) and configuration is typically via XML or Annotations.

SI provides out-of-the-box support for many of the patterns defined in Hohpe and Woolfe’s Enterprise Integration Patterns book and associated website and–as this article will show–is quite “Groovy friendly.”

A Small Example Application

In this article, I am going to take a very simple systems integration task and build a small example application. The application is a small Grails application that performs a simple function: given a customer number, it calculates the shipping cost for a single widget.

To show how Spring Integration can be used, I am assuming that a customer will enter their customer ID into a Grails-originated web page, which is then routed to one instance of a set of JMS servers. The associated address data is then retrieved, the city field extracted and then passed on to a SOAP-based WebService that calculates the cost of shipping widgets from Brisbane to the customer’s home city. To add a smidgen of realism, I have built the various services to be completely unrelated; in particular, they do not reference any form of standardized messaging schema. The integration pipeline needs to handle various simple message types, including strings, CSV-formatted data records, XML-formatted strings and floats.

Figure 1 provides a high-level view of the application.

Figure 1: The processing flow and major components comprising the example application

The various steps highlighted in this flow are:

The customer ID (a simple string) is entered into the Grails web application and passed off to a generic SI gateway. The gateway places the customer ID on an internal channel that leads to a message router.
The router retrieves the message from its inbound channel and determines which available channel to use to ensure that the message is handled by the appropriate JMS server instance.
The message is then handled by a SI JMS “channel adapter,” which deals with the mechanics of working with the underlying external JMS server.
The nominated JMS server instance produces a record (a CSV-formatted string) that is placed onto a shared reply channel.
The CSV record is removed from the shared channel and transformed. The result of the transformation is an XML-formatted string which is again placed on a ‘downstream’ queue.
The WebService channel adaptor removes the string from its request channel and invokes the external SOAP WebService for further processing.
The WebService response (a single float) is then passed back to the originating messaging gateway, which returns it to the Grails controller.

This is a very simple system and it may seem that this is overkill. There is a payoff, however: SI makes it easy to plug in disparate (often legacy) systems and thus produces an expandable framework that is capable of being built out to cover future, more complex, situations.

The Pieces of the Jigsaw Puzzle

Before starting out on the integration exercise proper, it is worth taking a quick look at the various pieces of the jigsaw puzzle that are going to be joined together.

You will quickly see that all the pieces are exceptionally simplistic. I haven’t even tried to make them correspond to a multitude of real-life issues such as error handling, logging, configuration, etc., which have been almost completely ignored. This is by design: for the purposes of this article, I want to focus mostly on SI, not on the various services.

The Grails Web Application

Figure 2 shows the Grails web interface “in action.” It is very simple, with only a single form and a single results page.

Figure 2: The deployed Grails web application

The ‘meat’ of the application (as far as this article is concerned) lies in the single Grails controller class shown in Listing 1.

public class SpringIntegrationController {

  def siGateway

  def index = { }

  def submit = {
    long start = System.currentTimeMillis()
    def msg = params.custid
    siGateway.send(msg)
    def got = siGateway.receive()
    flash.message =
      "Query took: ${System.currentTimeMillis() - start} millis."
    [shippingCost: got]
    }
  }

Listing 1: The Grails controller class

There are two points of interest in this simple class.

The siGateway instance injected from the Spring application context constitutes the sole point of contact between the Grails web application and the SI framework. (In theory, SI allows for an even less intrusive interface than this, based around dynamically constructed proxies, but a bug surfaced as I started writing the application…no matter, the hard way is not too onerous to use and I am sure that the SI guys will soon squash the bug.)

The submit closure is responsible for handling the POSTed form data and for driving the injected siGateway instance through its paces. It is pretty straightforward: all the action is happening over in “SI-land.”

The JMS Application

For the example application, I have written a very simple JMS service (using Apache ActiveMQ as the messaging provider), as shown in Listing 2.

import javax.jms.Session
import org.apache.activemq.ActiveMQConnectionFactory

public class AMQService
{
  private static factory =
	  new ActiveMQConnectionFactory("tcp://localhost:61616")

  public static void main(String[] args) throws Exception
  {
    def cKey = args[0]
    def cFile = args[1]

    println 'Starting...'
    println "Config key: $cKey"
    println "Config file: $cFile"

    def config =
      new ConfigSlurper(cKey).parse(new File(cFile).toURL())

    def database = config.database
    println "Database: $database"

    def inQ = config.inQ
    println "Incoming Queue: $inQ"

    def replyTopic = config.replyTopic
    println "Reply Topic: $replyTopic"

    def qConn = factory.createQueueConnection()
    def qSession =
      qConn.createQueueSession(false, Session.AUTO_ACKNOWLEDGE)
    def consumer =
      qSession.createConsumer(qSession.createQueue(inQ))

    def tConn = factory.createTopicConnection()
    def tSession =
      qConn.createTopicSession(false, Session.AUTO_ACKNOWLEDGE)
    def producer =
      tSession.createProducer(tSession.createTopic(replyTopic))

    qConn.start()
    tConn.start()

    for ( ; ; )
      process(database,
              tSession.createTextMessage(),
              consumer,
              producer)
  }

  private static process(database, replyMessage, consumer, producer)
  {
    try
    {
      def textMessage = consumer.receive()  // indefinite blocking
      replyMessage.with {
        setText(getResponse(database, textMessage.getText()))
        setJMSCorrelationID(textMessage.getJMSCorrelationID())
      }
      producer.send(replyMessage)
    }
    catch (Throwable t)
    {
      t.printStackTrace()
    }
  }

  private static getResponse(database, inMsg)
  {
    def resp = database[inMsg]
    println "Request: '$inMsg'; Response: '$resp'"
    resp
  }
}

Listing 2: The JMS server

This is a standard JMS service; no need to elaborate.

One point to note (that is not directly SI-related, but of interest nonetheless) is that the service is written to allow multiple instances to run at the same time, dealing with a partitioned data space. For the purposes of the example application, I have configured two instances. One instance is prepared to deal with customer IDs whose first character is in the range a-l and the second will deal with the rest of the alphabet.

The service accepts input via a uniquely-designated input queue, but sends its response to a shared topic. This structure simplifies things by ensuring that any downstream component merely has to handle a single publish/subscribe channel (which can carry data from multiple sources), rather than trying to deal with a multitude of point-to-point links.

Each server instance is driven by a pair of application parameters, defining:

The location of the configuration file to use
The appropriate environment section in the designated configuration file

The configuration file supplies:

The name of the input queue to use
The name of the output topic to receive the responses
The actual static database to use (I did warn that these applications were extremely simple!)

The service makes use of ConfigSlurper’s ability to deal with different environment sections and so I can define a single configuration file (see Listing 3) to configure the different instances uniquely.

replyTopic="SI.REPLY.TOPIC"

replyTopic="SI.REPLY.TOPIC"

environments {
  al {
    inQ="SI.AL.REQUEST"
    database=['b9876':'123 Nowhere St,Brisbane',
              'f0234':'42 Imaginary Place,Darwin']
    }
  mz {
    inQ="SI.MZ.REQUEST"
    database=['x5555':'123 Fake St,Hobart',
              'w8888':'000 Talkfest Lane,Canberra']
    }
  }

Listing 3: JMS server configuration file

This service accepts a string, but returns a CSV-formatted record.

The WebService

The final part in the jigsaw puzzle is a SOAP WebService built on top of the facilities provided by GroovyWS.

The service’s raison-d’être is to retrieve the cost of shipping n widgets between major cities in Australia; see Listing 4 and Listing 5.

public class ShippingCostWebService {

  static costs = [
    Brisbane: [ Brisbane:0.0F,
                Darwin:44.44F,
                Canberra:22.22F,
                Hobart:88.88F ],
    Darwin: [ Brisbane:44.44F,
              Darwin:0.0F,
              Canberra: 33.33F,
              Hobart: 99.99F ],
    Canberra: [ Brisbane:22.22F,
                Darwin:33.33F,
                Canberra: 0.0F,
                Hobart: 77.77F ],
    Hobart: [ Brisbane:88.88F,
              Darwin:99.99F,
              Canberra: 77.77F,
              Hobart: 0.0F ],
  ]

  Float calculateShippingCost(String fromLoc,
                              String toLoc,
                              Integer nItem) {
    def cost = costs[fromLoc][toLoc] * nItem
    println "calculateShippingCost($fromLoc,$toLoc,$nItem) => $cost"
    cost
    }
}

Listing 4: The shipping cost WebService

println 'Starting...'

def ws = new groovyx.net.ws.WSServer()

ws.setNode("ShippingCostWebService",
           "http://localhost:6980/ShippingCostWebService")

ws.start()

println '...Started.'

Listing 5: The GroovyWS WebService server harness

I can’t lie! This is just an old WebService that I had lying around that I created for a conference presentation, and which I have adopted for this application. It doesn’t have a customized API and isn’t necessarily a perfect fit to the task. In the systems integration world, improvisation is often required to squeeze data out of the numerous ‘sub-optimal’ nooks and crannies that are found all over an enterprise.

Joining the Pieces Together

Now that you are familiar with the basic componentry, it is time to look at SI proper.

As with any Spring project, SI is configured separately to the actual code in an application. Since this application is based on Grails, I have chosen to use the nice resources.groovy Grails Spring Beans DSL configuration file, rather than the ‘traditional’ XML-based configuration.

Listing 6 shows the complete configuration.

beans = {
  xmlns si:"http://www.springframework.org/schema/integration"
  xmlns jms:"http://www.springframework.org/schema/integration/jms"
  xmlns stream:"http://www.springframework.org/schema/integration/stream"
  xmlns ws:"http://www.springframework.org/schema/integration/ws"

  // SI componentry and plumbing
  si {
    poller(default: true) {
      "interval-trigger"(interval: 1, "time-unit": "SECONDS")
    }
    router("input-channel": "routerChannel",
           ref: "highLowRouter",
           method: "route")
    transformer("input-channel": "InboundTopic",
                "output-channel": "TransformedInbound",
                ref: "csvStringTransformer",
                method: "transform")
    channel(id: "routerChannel")
    channel(id: "ALOutboundChannel")
    channel(id: "MZOutboundChannel")
    channel(id: "RoutingRejectChannel") {
      queue(capacity: "256")
    }
    channel(id: "TransformedInbound") {
      queue(capacity: "16")
    }
    channel(id: "InboundTopic") {
      queue(capacity: "16")
    }
    channel(id: "ShippingCostChannel",
            dataType: "java.lang.Float") {
      queue(capacity: "16")
    }
  }

  stream {
    "stderr-channel-adapter"(channel:"errorChannel",
                             "append-newline": true)
  }

  // WebService stuff
  ws {
    "outbound-gateway"("request-channel": "TransformedInbound",
                       "reply-channel": "ShippingCostChannel",
                        uri: "http://localhost:6980/ShippingCostWebService")
  }

  // AMQ/JMS stuff
  connectionFactory(org.apache.activemq.pool.PooledConnectionFactory) { bean ->
    bean.destroyMethod = "stop"
    connectionFactory = { org.apache.activemq.ActiveMQConnectionFactory cf ->
      brokerURL = "tcp://localhost:61616"
    }
  }

  "AL.REQUEST"(org.apache.activemq.command.ActiveMQQueue, "SI.AL.REQUEST")
  "MZ.REQUEST"(org.apache.activemq.command.ActiveMQQueue, "SI.MZ.REQUEST")
  "REPLY"(org.apache.activemq.command.ActiveMQTopic, "SI.REPLY.TOPIC")

  jms {
    "outbound-channel-adapter"(channel: "ALOutboundChannel",
                               destination: "AL.REQUEST")

    "outbound-channel-adapter"(channel: "MZOutboundChannel",
                               destination: "MZ.REQUEST")

    "message-driven-channel-adapter"(channel: "InboundTopic",
                                     destination: "REPLY")
   }

  // general componentry
  siGateway(org.springframework.integration.gateway.SimpleMessagingGateway) {
    requestChannel = ref("routerChannel")
    replyChannel = ref("ShippingCostChannel")
    replyTimeout = "10000"
  }

  highLowRouter(au.com.transentia.si.LookupRouter)

  csvStringTransformer(au.com.transentia.si.CsvTransformer) {
    homeBase = 'Brisbane'
  }
}

Listing 6: The Spring Integration configuration

As you read the following discussion, you may find it useful to refer back to Figure 1.

The configuration is structured into various sections, according to the needs of SI and of the various resources that are being integrated.

The si section (more precisely, those elements of SI configured via the XML namespace allocated the si prefix in this document) configures the various aspects of SI itself. This includes establishing the various channels that interlink the components, defining a few application-specific components (such as the router and transformer), and putting in place a default polling schedule for those parts of SI that need to poll.

It is worth examining the router and transformer components here. I have mentioned that the JMS services deal with a partitioned data space and that traffic is directed to an individual service based on the actual data message being processed. SI allows for the definition of an application-specific router to perform this type of task, in this case the method route of the highLowRouter instance. Listing 7 shows the code for the router.

package au.com.transentia.si

public class LookupRouter {

  public String route(String msg) {
    switch(msg) {
      case ~/(?i:[a-l].*)/: return 'ALOutboundChannel'
      case ~/(?i:[m-z].*)/: return 'MZOutboundChannel'
      default: return 'reject'
    }
  }
}

Listing 7: The router class

The router examines the incoming message data and determines the appropriate outbound path for the message. Groovy’s wonderfully versatile switch statement, combined with the ability to do case-independent pattern matching, makes for beautifully minimalistic code.

The transformer is equally simple, as Listing 8 shows.

package au.com.transentia.si

public class CsvTransformer {
  def homeBase

  public String transform(String csv) {
    def dest = csv.tokenize(',')[1]

    """
         $homeBase
         $dest
         1
      """
  }
}

Listing 8: The transformer class

This code is presented with an input CSV-formatted record, extracts the relevant field, and passes an XML-formatted string onwards.

Recall that the WebService’s API is not custom-built for the purpose of this application; for this use, arg2 (the nItems parameter) is always fixed at 1.

The stream section of the configuration merely allocates a generic channel listener so that the standard errorChannel is not silently ignored; this configuration will ensure that errors/exceptions, etc., are logged.

The ws section deals–unsurprisingly–with WebService integration. The section configures an SI outbound gateway that is capable of receiving a request on a specified channel, invoking the configured WebService and finally sending the result to the appropriate response channel. All this is specified declaratively: the actual mechanism of dealing with the WebService (retrieving WSDL documents, generating proxies and handling XML-based request/response messages) is completely hidden away.

The remaining configuration concerns SI’s JMS adapter and ActiveMQ integration.

The siGateway has been discussed previously.

By default, all SI’s JMS adapters reference a bean with the standard name of connectionFactory. It is the responsibility of this bean to ‘vend’ connections to the external JMS system. In this case, a connection pool to a local Apache ActiveMQ server running at the URL “tcp://localhost:61616″ is being created. Since the external connections are pooled for efficiency, by specifying destroyMethod=’stop’ we ensure that they are simply inactivated after each (re)use and are never actually discarded.

Note: I found the actual syntax used to declare the connectionFactory a little tricky. I had to ask for help on the excellent user@grails.codehaus.org mailing list to get it right! Thanks to the generous souls who pointed me in the right direction.

Following the definition of the the connectionFactory, we create two instances of ActiveMQ queues and an instance of an ActiveMQ topic. It is these that carry the messages to/from the external JMS resources.

The jms section configures two JMS-specific outbound channel adapters and one inbound channel adapter. As the names suggest, these are concerned with driving (or being driven by) the ActiveMQ instances declared above. The configured message-driven-channel-adapter instance defines a component that will be asynchronously “actively invoked” to handle an incoming message. It is possible to configure a polling-oriented component, but this is generally less flexible, and may waste CPU, and increase the latency in the system.

As with the WebService gateway, everything is declarative.

Admiring the Finished Puzzle

SI has enabled a deceptively simple solution to a fairly complex problem. There are no messy tracts of confusing mechanistic code; the system is essentially defined all in a single configuration; the individual components are simple and (generally) reusable; and the groundwork for coping with change set with very little effort.

SI is a simple but powerful toolkit; the example application I have discussed here has really just touched the surface of what is possible. SI provides much more, including facilities for secure channels, inbound/outbound mail handling, event handling, RMI integration and much better XML handling than I have touched upon here.

Remember that these are still early days for SI: for example, there are no debuggers or GUI editors as found in the big “name brand” systems integration tools. In time these tools may come but for now remember that SI is orders of magnitude simpler, and the SI/Groovy/Grails triumvirate makes the developer many times more productive than some other tools I have used.

I hope that I have stimulated your interest in SI and shown that a Grails application need not be limited to serving up “little web apps” but can actually service the needs of larger-scale systems integration tasks.

I hope that I have also been able to show that when the time comes to get out the scissors, Groovy is more than equal to the task of “slicing and dicing” until the pieces of the systems integration jigsaw puzzle fit nicely together.

Learn More

Gregor Hohpe and Bobby Woolf, Enterprise Integration Patterns : Designing, Building, and Deploying Messaging Solutions (ISBN 0321200683), also http://www.enterpriseintegrationpatterns.com/
Spring Integration Home, http://www.springsource.org/spring-integration
Apche ActiveMQ, http://activemq.apache.org
GroovyWS, http://docs.codehaus.org/display/GROOVY/GroovyWS. The exemplar application uses the 0.5 snapshot jars from http://snapshots.dist.codehaus.org/groovy/distributions/groovyws/

Watch out for my other Groovymag articles…coming soon to a browser near you.

Brisbane is “Groovy, Baby!”

bob — Tue, 29 Sep 2009 02:27:46 +0000

I was at yesterday’s Groovy/Grails Queensland meetup.

We started talking about the wealth of talent and Groovy/Grails ‘names’ that were clustered in and around Beautiful Brisbane, Australia. People with major, direct influence on the various Gr8 technologies. People like: Dr. Paul King, Lee Butts, Luke Daley, Bradley Beddoes (to name but a few). All movers and shakers in the Gr8 world. Many were in the room at the time.

Steve Dalton is maintaining a slightly bigger list.

We truly have a world-class, leading-edge group here.

To quote Lee: “We are the Australian HUB for Grails/Groovy :)”

(I’m so glad that I managed to avoid using the phrase ‘paradigm shifting’…but it was hard ;-))