This course is “Tiliting at Windmills 201.”

I’m your lecturer Don Quixote.

We will begin with a snap quiz.

You have two well-established computer systems A and B. In accordance with its original design system B makes use of a small but signifiant proportion of system A’s schema. It is proposed that system B should be enhanced, which will require accessing a few additional tables from system A’s schema.

Do you:

1. Go ahead and grab the data you need, along the lines of what is already being done?
2. First espouse the concept of a “common integration language” and then promote the idea of linking the two systems together via some to-be-announced-Real-Soon-Now “integration layer”?
3. Place a huge purchase order for the latest silver bullet product espoused in last month’s “What’s Next! A busy CIO’s guide to the latest trends in ICT” magazine, mandate that this product is now the “Enterprise Standard”, and require systems A & B should use this product for their new “Enterprise-level Interactions”?

I’ll give you a few seconds to think about it….

…time’s up.

Now: those of you who answered ‘A’…

Sigh. Did you learn nothing last year? Don’t you remember “Complification 101″? Don’t you remember the Three Principles: ‘Complicate’, ‘Obfuscate’, ‘Procrastinate’? I am disappointed in you…you’ll never get ahead in today’s enterprise with a can-do attitude like that.

OK. Who answered B?

That’s more like it! Glad to see you ‘B’s were paying attention in “Elementary Enterprise Operations 101″ last year. Remember that this alternative gives you the advantage of being able to wander around the whole organisation declaiming “Integration Layer” at every opportunity. This may not constitute a successful approach to building working software but it sure is a great way of building visibility with The Powers That Be…and what’s more important, after all, eh?

NO, Mr. “I answered ‘A’…I thought it was all about building software to do something useful”…it isn’t. Really! Maybe you should be rethinking your career path while the year is still young?

Did anybody answer C? Ah. you must be from the “Software and Sales” double degree. I see bright futures ahead for you guys. This is the way to go! Even you ‘B’s need to take notice here. Maybe even consider taking “Hype and Hysteria 204″? There’s always another silver bullet just around the corner and you don’t want to miss out on the opportunities.

Who’s left? You over there…you’re a “Software Law” student aren’t you? Kibitzing, I see…what are you looking so happy about? Wonderful, just wonderful: “Looking forward to all those failed implementations.” This, class, is what you should really be aspiring to if you really want to succeed! The software wrangling business is for mugs…why do you think that I am standing here in front of you and not sitting in front of a computer!

OK. With all this in mind, let’s begin, shall we?

Today’s lecture is entitled “A Picture Paints A Thousand Words. It Also Consumes Many Megabytes, Gives You RSI And Still Ends Up Less Useful Than A Single Line Of Code.” Please open your books to the chapter entitled “Oracle Service Bus”…

I’ve got to go now…some nice men in white coats have come to take me on holiday. How nice of them!

Seriously though…some 2004-vintage food for thought, entitled Jump off the bus, take a cab!:

…
A lot of companies have expressed the similarities between SOA and the concept of (hardware) Bus….I understand that a bus offers a “common interface infrastructure” and components can use the bus to communicate with each other, this is pretty much where the analogy stops for SOA. SOA requires very little in the middle (other than a standard communication infrastructure, that we already have for free) and is inherently point to point and asynchronous, unlike most bus architectures which require some centralized infrastructure and synchronous behavior. I would agree with the OASIS/BCM group that the only shared component in a SOA is a directory service.
…

Bear this quote in mind the next time some SOA-totin’ salesman attempts to bale up your checkbook with an order for their latest and greatest centralised server-based product.

Also worth an “honourable mention” is this oldie-but-a-goodie from InfoQ: ESB-Oriented Architectures considered harmful.

A final thought, as directed at the ‘real’ Don Quixote:

And you, Señor Don Quixote, your head is going to end up a stranger to your neck.

Concomitant with my attempts to be a good modern developer, one of the first entries on my Fings Wot I Need To Find Out More About list went something like:

Given that almost all of the actual ‘code’ in my system will be XQuery transformations, how do I unit-test those transformations with JUnit? It is obviously important that I use the weblogic API/Jars, not some external or third-party system.

Roughly six months later I am finally in a position to answer that question!

Took me a few goes (turns out that there are a number of different potential red-herring APIs to filter out and reject, including BEAs xdbc), a number of visits to other people‘s attempts and about 20 messages on the Oracle forum (gotta say I appreciated the comment “I just wanted to praise your discipline and devotion to research and investigation. Kudos!” that I picked up along the way…) but here I am.

This is the Groovy JUnit4 test class, just to prove that it can be done:

package transentia.test

import org.junit.*
import transentia.XQueryTransformer

class XQTest {
  def xqt

  @Before
   void setup() {
    xqt = new XQueryTransformer("xqueries/Messier.xq",
                                 src: new File("xml/Messier.xml"),
                                 str: 'hello, world',
                                 n: 42,
                                 b: true,
                                 dt: new Date(),
                                 f: Math.PI,
                                 nd: null as Date)
  }

  @Test
  void testTransform() {
    xqt.withString { str ->
      println str
    }
  }

  @Test
  void testSlurping() {
    xqt.withString { str ->
      def data = new XmlSlurper().parseText(str).declareNamespace(oth: "http://other")
      assert data.'oth:m'.size() == 5
      assert data.'oth:m'[3].text() == 'Scorpius'
    }
  }

  @Test
  void testParsing() {
    xqt.withString { str ->
      def data = new XmlParser().parseText(str)
      assert data.'oth:m'.size() == 5
      assert data.'oth:m'[4].text() == 'Serpens'
    }
  }
}

Pretty trivial.

This is the XQuery that is under test:

(Strangely, wordpress simply refuses to display this file without barfing. Can’t for the life of me work out why, hence the above is an image. Here’s the real file)

Notice the <destination:m?> bit? That trailing question mark is a BEA extension to the standard XQuery language: “Optional Indicator in Direct Element and Attribute Constructors.” Very useful but not widely advertised/supported (which basically accounts for why it took me 6 months to work out how to handle it).

Next is the XML file:

<?xml version="1.0" standalone="yes" ?>
<MESSIER xmlns="http://transentia">
  <M INDEX="1">
    <CONSTELLATION>Taurus</CONSTELLATION>
    <DESCRIPTION>Diffuse Nebula</DESCRIPTION>
  </M>
  <M INDEX="2">
    <CONSTELLATION>Aqarius</CONSTELLATION>
    <DESCRIPTION>Globular Cluster</DESCRIPTION>
  </M>
  <M INDEX="3">
    <CONSTELLATION>Canes Venatici</CONSTELLATION>
    <DESCRIPTION>Globular Cluster</DESCRIPTION>
  </M>
  <M INDEX="4">
    <CONSTELLATION>Scorpius</CONSTELLATION>
    <DESCRIPTION>Globular Cluster</DESCRIPTION>
  </M>
  <M INDEX="5">
    <CONSTELLATION>Serpens</CONSTELLATION>
    <DESCRIPTION>Globular Cluster</DESCRIPTION>
  </M>
  <M INDEX="6">
    <DESCRIPTION>ONLY FOR TESTING</DESCRIPTION>
  </M>
</MESSIER>

This file is just a tiny amount of fixture data for testing.

The missing <CONSTELLATION> element within the final <M> element should ‘trigger’ the XQuery engine to elide the empty <destination:m /> element that it would normally produce; thus the tests are all expecting 5 elements in the transformed XML, not 6 as in the source.

And now…without further ado…the pièce de résistance:

package transentia

import org.apache.xmlbeans.XmlObject
import org.apache.xmlbeans.XmlOptions
import org.apache.xmlbeans.XmlDateTime
import org.apache.xmlbeans.XmlBoolean
import org.apache.xmlbeans.XmlInteger
import org.apache.xmlbeans.XmlDouble
import org.apache.xmlbeans.XmlString

class XQueryTransformer {
  private final resultXML

  public XQueryTransformer(map, xQueryTransformSource) {

    XmlObject xmlObject = XmlObject.Factory.newInstance()
    XmlOptions options = new XmlOptions()

    options.setXqueryVariables(createOptions(map))

    XmlObject[] results = xmlObject.execQuery(new File(xQueryTransformSource).text, options)

    resultXML = results[0].xmlText(new XmlOptions().setSavePrettyPrint().setSavePrettyPrintIndent(2))
  }

  private createOptions(map) {
    def optionsMap = [:]
    map.each { k, v ->
      switch (v?.class) {
        case Date:
          XmlDateTime dt = XmlDateTime.Factory.newInstance();
          dt.setDateValue(v)
          optionsMap.put(k, dt)
          break
        case Boolean:
          XmlBoolean b = XmlBoolean.Factory.newInstance();
          b.set(v)
          optionsMap.put(k, b)
          break
        case [BigInteger, Integer, Long, Short, Byte, byte, short, int, long]:
          XmlInteger i = XmlInteger.Factory.newInstance();
          i.setBigIntegerValue(v as BigInteger)
          optionsMap.put(k, i)
          break
        case [BigDecimal, Float, Double, float, double]:
          XmlDouble xd = XmlDouble.Factory.newInstance();
          xd.setDoubleValue(v as double)
          optionsMap.put(k, xd)
          break
        case String:
          XmlString string = XmlString.Factory.newInstance();
          string.setStringValue(v);
          optionsMap.put(k, string)
          break
        case File:
          def txt = v.text

          // look for the prologue < ?xml ...?> processing instruction
          def m = txt =~ /(\< \?xml[^>]*\>)/
          if (m.count > 0) {
            def xmlPI = m[0][1]
            txt = """${xmlPI}
${txt[xmlPI.size()..-1]}
"""
          }
          else
            txt = "${txt}"

          // all the messing around with xml processing instruction/adding a new root above is because
          // selectChildren seems to be required. Don't know why...
          optionsMap.put(k, XmlObject.Factory.parse(txt).selectChildren("temp42", "root42")[0])
          break

        case null:
          XmlObject n = XmlObject.Factory.newInstance()
          n.setNil()
          optionsMap.put(k, n)
          break

        default:
          throw new Exception("Unhandled type: ${v.class}")
          break
      }
    }
    optionsMap
  }

  def withString(c) {
    c.call resultXML
  }
}

I’ve said it before and I’ll most likely say it again: love the Groovy switch statement!

A bit of clarification for the ‘File’ case is in order, I guess because it’s quite confusing! I found that I had to wrap a dummy root element around the original root element in order to immediately select only the single child element (the original )…this is why I need to isolate the processing instruction: the opening tag has to go between the PI and the tags. It seemed easiest to do this with simple string manipulation…

Here is the tricky bit…the various jars that need to be placed on the classpath:

oracle.nlsrtl_11.1.0\orai18n-collation.jar
oracle.xdk_11.1.0\xquery.jar
com.bea.core.xml.xmlbeans_2.0.0.0_2-5-1.jar
com.bea.core.xquery.xmlbeans-interop_1.3.0.0.jar
com.bea.core.binxml_1.3.0.0.jar
oracle.xdk_11.1.0\xml.jar
com.bea.core.xquery_1.3.0.0.jar
oracle.xdk_11.1.0\xmlparserv2.jar
com.bea.core.xquery.beaxmlbeans-interop_1.3.0.0.jar
weblogic.server.modules.xquery_10.3.3.0.jar

(Note the ordering: this is what IntelliJ X EAP worked out for me; who am I to argue…)

And to cut out any confusion, here’s what I have in my Intellij project:

C:\DEVELOPMENT\XQueryTester>tree /f /a
Folder PATH listing for volume Bugblatter
Volume serial number is 42E3-A91B
C:.
|   XQueryTester.iml
|
...
|
+---lib
|   |   com.bea.core.binxml_1.3.0.0.jar
|   |   com.bea.core.xml.xmlbeans_2.0.0.0_2-5-1.jar
|   |   com.bea.core.xquery.beaxmlbeans-interop_1.3.0.0.jar
|   |   com.bea.core.xquery.xmlbeans-interop_1.3.0.0.jar
|   |   com.bea.core.xquery_1.3.0.0.jar
|   |
|   +---modules
|   |   \---features
|   |           weblogic.server.modules.xquery_10.3.3.0.jar
|   |
|   +---oracle.nlsrtl_11.1.0
|   |       orai18n-collation.jar
|   |
|   \---oracle.xdk_11.1.0
|           xml.jar
|           xmlparserv2.jar
|           xquery.jar
|
...
|
+---src
|   \---transentia
|       |   XQueryTransformer.groovy
|       |
|       \---test
|               XQTest.groovy
|
+---xml
|       Messier.xml
|
\---xqueries
        Messier.xq

C:\DEVELOPMENT\XQueryTester>

All requisite jars can be found within the OSB 11g installation.

Update 2011 01 01; at least one person has read this and given me some feedback:

I needed to add com.bea.core.antlr_2.7.7.jar to the build path as
well. Maybe it’s because I use Eclipse and not IDEA.

Thanks, Morten!

The proof, as ‘they’ say is in the pudding…

Here’s the passing case:

Now for the failing case (with the ? taken out of the XQuery transformation):

Take a good look at the way the failed assertion is represented in the above screenshot. Something else to love about Groovy!

Hope this helps some other OSB user actually trying to embrace good software engineering practices.

The above isn’t perfect but it’s better than nothing. Use it; improve it; share it. Please let me know what you have done…

Restating something I said in the Oracle SOA Suite forums:

IMHO it is RIDICULOUS that what we are trying to achieve here is not clearly described and supported for OSB.

For something like Apache Camel or Spring Integration testing is a complete no-brainer. For OSB it seems almost impossible to do properly. For me this is a (/one of the) killer points AGAINST OSB.

Its a shame that this is not available from Oracle but then one can’t really expect them to go to all that trouble of carefully crafting a tool like OSB with maximum lock-in potential and then make the APIs openly accessible, can one?
(…and yes I do know the genesis of Apache XMLBeans…)

Ooohhhh…hope I’m not breaking any licensing rules here…don’t want to tangle with Oracle’s lawyers, after all!

I am currently working on a pretty simple systems integration task: read from a small bunch of database tables, munge the data and call a variety of webservices. Fundamentally a simpler integration problem than the exemplar for Spring Integration I worked up before. Pretty simple with Java and a framework like Spring Integration or Apache Camel; undoubtedly even simpler if I could add Groovy into the mix.

Organisation says: “Thou Shalt Use Oracle Service Bus.”

Ever heard the joke about regular expressions: Once upon a time a programmer had a problem. He decided to solve it using Regular Expressions. Then he had two problems.

Welcome to my world (with an OSB flavour) :-)

I’ll try and capture a few thoughts and experiences below.

It is very rare that I react so strongly against a technology….can’t really think of another example. I’m usually pretty gung-ho and even suffer from odd breakouts of evangelical fervour. I can even find a place for UML in the nerd-ish pantheon :-) For this product all bets are off. I’m struggling to find a silver lining in the cloud.

I hope that if I break things up into small doses, you won’t break down with sympathy tears…

Positioning

Why choose OSB vs a BPEL implementation? BPEL has an OASIS standard behind it, is the richer language, is more mature and more expressive, has better tooling, has multivendor support, can deal with state, timouts and long-running workflows…all the things that one needs in an integration setting. OSB does practically none of these things, so what’s it all about, Larry?

Ahhhh…perhaps that little ‘multivendor’ word gives us a clue? “So young to be so cynical”, I hear you say.

I have heard it said that OSB is “high level” and the alternatives like Apache Camel and Spring Integration are developers tools and as such are too “low level” to be true solutions. Sigh. I thought that old argument for CASE was put to bed long ago. It’s sad perhaps, but you still need developers to do complex development. Don’t believe me? Take a look at the average spreadsheet and then tell me how appropriate this “end-user engineering” idea is.

The task is not to find the most abstract or pretty tool but to put into the developers’ sweaty hands the toolset capable of getting into the gnarly nooks and crannies of a problem space and still come out on top. Next time you hire a carpenter or electrician or plumber, take a look into his/her toolbox. It won’t be pretty but it will be varied and capable. I’ll bet that it took the tradesperson quite a bit of training and experience to work effectively with everything in it too…there is no magic bullet.

Architectural

“Stored Procedures migrate to Session Beans” used to be the J2EE catch-call (for very good reasons, IMHO, YMMV). In my current OSB-oriented project, this seems to have become “Let’s do as much as possible in stored procedures because it’s too darned painful to even consider otherwise.”

This is really rubbing against the grain!

It’s interesting that these days, discussions regarding Systems Integration always bring up the “VETRO pattern“: Validate, Enhance, Transform, Route, Operate. This is great, as far as it goes. In my experience however VETRO is by far the easy side of SI. Prising and cajoling the requisite data out from the various niches, repositories, formats, access methods, security regimes, etc. tends to be much more tricky and is a necessary prerequisite. It’s sad, but I can’t think of a catchy management friendly phrase for all this nasty boring legwork. The term “Data Mining” is already taken, darn it! Regardless, I’m absolutely positive that no pretty picture is going to encapsulate the true reality of the SI task. I’m all for aspiring to do better but hey…even in Star Trek they sometimes need to reverse the polarity of their polaron fields in new and untested ways, and when they do they don’t draw pretty pictures but rip off the access panel to their nearest Jeffries tube, get down on their hands and knees and crawl.

In the integration community these days, it is common to tip the hat to Gregor and Hopke’s Enterprise Integration Patterns work. The book sits on every “Integration Architect”‘s shelf and provides a useful reference work. It’s very tempting to imagine architects sitting around designing systems using Star Trek-style dialog: “We’ll hook this Incoming Polling Consumer to this Aggregator, then to this Resequencer before Routing the incoming message to the outgoing Messaging Gateway.” Leaving aside the issue that this scenario and reality correspond to each other in roughly the same way that Star Trek corresponds to the NASA, let’s take a look at OSB’s support for patterns.

There’s a few: various species of polling consumer, a bit of routing, some basic message transformation and then…you’re on your own.

It is extremely cumbersome to do anything other than broad brush-stroke flows…I feel that I am being forced to spatter stored procedures, triggers, DBMS_ALERT calls and odd little support tables throughout my database, along with (as simple as possible) snippets of Java in the OSB application in a desperate attempt get OSB to do something useful.

I frequently repeat the following mantras: “OSB is just a dumb pipe” and “Let’s keep OSB as dumb as possible.”

I simply can’t bring myself contemplate the effort needed to deal with all the callouts to all the Java objects needed to deal with any sort of sophisticated API or complex processing.

I’m pretty sure that I could replace much of the mess that has grown up as a result of OSB’s pathetically limited capabilities with a couple of standard Java collection classes mixed in with a decent polling implementation such as in Apache Camel or Spring Integration.

Compare and contrast with Camel, which has direct support for about 50 of these patterns. That previous link also provides links to a nifty library of icons for them all, in various formats…very useful. Spring Integration also directly supports several of these patterns “out of the box.”

Seems to me (cynic that I am) that OSB successfully implements just two patterns: the “sell copies of Weblogic” pattern and the “extract lots of money from customers” pattern.

What goes into OSB, stays in OSB. For all but the simplest bit of work one typically needs to develop additional helper tools/applications/scripts, etc. Tools like JPA and Camel are by design flexible and make it possible to create these tools by reusing elements of the ‘main’ development path (a part of the object model, or a transformer, say). OSB does not…quite the reverse, in fact (it is very much [Oracle Weblogic] server-side-only, for example). Any ‘investment’/effort put into OSB is in reality not going towards infrastructure development but is instead just bringing into existence another closed, siloed system. An instant legacy problem. The only investment that one does with OSB is investing into Oracle’s share price. This is unacceptable. It need not be this way. We have tools available that can be used to enhance our infrastructure, not fragment it, that can work with our Objects and APIs, not make an all-out (but subtle) effort to supplant them.

Development/Engineering

OSB has an “everything is XML” approach (OK I acknowledge that it’s slowly divesting itself of this, but there’s a way to go yet before the alternatives are anything approaching useable). I like XML as much as the next guy, probably more; it’s a great interchange format, but requiring it for all internal channels as well is a bit much: one ends up touching the in-flight data (translating/extracting/recomposing) too many times. With JPA and JAX-RPC, one can in essence and after a little housekeeping do:

WebService.doSomething(DBEntity.findByPrimaryKey(pk));

The equivalent OSB (with Database JCA Adapters in place) is horrendous, simply horrendous!

Don’t get me started on the Database JCA Adapters! They are supported in OSB but cannot (yet) be created via OSB’s Eclipse-based tooling. This means that one is required to use JDeveloper 11g to create the adapter bindings appropriate for the underlying databases and then manually copy the resulting batch of files (a group of up to 5 files for each table being used) into the Eclipse project. Automate those build steps, I challenge you!

It gets worse: an OSB proxy created to service a JCA adaptor is extremely fragile: changes to a database table requires a regeneration of the adapter files (or more like a recreation from scratch…manual click, click, clickety, click). Any slight change to an adapter forces a regeneration of the associated OSB proxy…which blows away any business logic (== pretty drawing) associated with the message flow. One is then forced to recreate the flow from scratch by hand (more manual click, click, clickety, click). Ghastly. Tedious. Error prone. Nonsense. It’s in the nature of the systems integration task that changes happen often. One gets tired of (re)drawing OSB’s pretty pictures very quickly, let me testify.

One gets around this by splitting a flow into upper and lower levels: the upper level simply forwards to the lower level with as little effort as possible. The lower level flow does all the real work and is unaffected by changes to the adapter or upper level flow.

Some things might get cleaned up in the future (Oracle/BEA’s XQuery implementation is curently only partial [no module support], for instance) but I’m not holding my breath.

There’s even worse: OSB seems to confuse database polling frequency with database locking time: tell an adapter to poll every 10 secs and it seems to go away and lock all other processes out of the database for the next 10 secs, polling away in splendid isolation. This can lead to performance issues and effectively ties all adapters to a very short polling cycle. That’s a bit of a surprise and can lead to some subtle ‘issues’ arising, let me tell you…

I a great fan of these adapters…I simply love the way that they get ‘stuck’ every now and then (often when the underlying database is itself under some load). They often get ‘unstuck’ as well. Usually…sometimes…. I love going into production with systems that ‘usually’ work OK (that was sarcasm, in case you are sarcastically-challenged :-)).

OSB is completely stateless. To create something as simple as an “I have sent out a total of n messages” counter requires introducing a Java callout with an absurd amount of glue coding. Some things simply aren’t worth doing…

A good tool makes it easy to develop clear, relevant solutions close to the “problem space” without unnecessary obtuse ‘ceremony’ getting in the way. OSB is not such a tool, sadly.

You know the acronym DRY? Well, forget it!

One can try to apply a workaround and achieve a degree of DRY-ish-ness but the spaghetti flows that result aren’t worth the hassle. Here’s the issue: the process of munging a solution to fit in with OSB’s way of doing things results in a horrid unclear mess. Really. The cure ends up being worse than the disease: the resulting diagram becomes simply unintelligible…much less clear than the equivalent plain-old-text representation in Java.

Not convinced? OK, Mr./Mrs. Smarty-Pants: what values are being assigned and logged here?

To find out, you have to engage in a storm of clicking through the pretty icons and juggling Eclipe’s idiosyncratic idea of view focus control. After a while you will find that your wrists need to recover from their little workout, giving you time reflect on the fact that you’d be able to find out instantly when perusing a textual source, no clickety-click investigation needed. There’s a big difference between reading and scanning code; oftentimes the latter is what we need to do…and quickly.

On a related note, it’s hard to make the correspondence between trace logging output and the actual code. Consider, for illustration, that all service callouts have the same fixed label: “Service Callout.” It can be slightly awkward to pin down which “Service Callout” just logged something.

While it may be appealing to the naive or hopeful, Mouse-Hand Engineering…drawing pretty pictures…doesn’t really ease the development task. That’s just the nature of our complicated world…sad, but that’s life.

(You know, I was going to use the phrase “Right-Hand Engineering” but felt it was slightly too risque ;-))
(And don’t get me started [again] on UML, I’ve already thrown it a bone once in this posting :-))

Oh for the ability to “comment out” or bypass sections of a flow! It continually frustrates me that in most so-called modern languages, one cannot nest comments (something that Pascal had all worked out), but OSB drives me potty. Would the industry (yes, even this fad-ridden, silver-bullet-seeking, flavour-of-the-month driven IT industry) accept a textual language without a commenting convention, I wonder? This issue also relates to testing (see below).

Here’s a cute feature: every message that enters the system is given an unique message ID. Except that it isn’t actually unique, darn it! As a message is passed between services, each service allocates it a brand new ID. This is dumb, dumb, dumb! One is forced either to fiddle with and refer to header fields all the time, or do what I did and use a message’s database-allocated synthtic (surrogate) primary key as the message ID. Way to make it easy for us, OSB!

Error handling is broken. I know I’m being picky but something a little more than “something bad happened somewhere in some downstream flow or other” would be helpful. There are rumors of patches for this issue; good luck is all I can say.

One can’t get at system parameters/classpath resources, etc. without falling back to Java. To do anything slightly out of OSB’s (putative) “sweet spot”, one falls back to Java…so why not use a Java-based framework?

Testing. Ha ha haaaah!

How does one go about testing a drawing? This is probably the #1 question on the OSB forum.

Yes, the OSB console web application allows one to “test drive” the various componentry. Surely we can do better than hand-driven integration testing, however. Tools like citrus and SoapUI come close but don’t ring the bell for me…

Mark Smith gives an overview of the ridiculous process of creating testing stubs/proxies and the associated rewiring required when testing a proxy service’s flow. If a flow must be progressively rewired in order to be ‘tested’, is the flow actually being tested, I ask (rhetorically)?

Unit testing XQuery transformations should be simple, but isn’t…there’s no doco and a closed implementation. Writing testing flows in OSB seems ridiculous to me. Unit tests shouldn’t need a service bus and you shouldn’t be forced into integration-style testing for your atomic code units. Something smells quite wrong here.

I’ll probably pontificate on this some another time…for now, its in my “tried and failed to do something of value for the project” basket.

It may be a little churlish of me to point this out, but the XQuery editor always has some reason or other why it won’t let you do something…even dragging a dateTime onto a String:

The editor gets in the way many more times than it helps. As Roger van de Kimmenade also noted, it can get so completely messed up that it can’t do anything except show a blank screen. It can sometimes (usually when opening a file) even rewrite a transformation as it sees fit…blowing any hand-made changes away in the process. This is problematic ‘cos (as I have just pointed out) you have to do pretty much everything by hand. You Have Been Warned (hmmm…maybe Oracle should trademark this phrase. Hey: maybe I should…).

I particularly enjoy the fact that–although OSB is BEA Weblogic Server at its heart–OSB can’t readily create HTML. “Hang On a min., ” you say “isn’t HTML just a variant of XML at heart? Surely OSB should be happy with HTML!” Surely. After I asked roughly the same question on the OSB forums, Manoj Neelapu was kind enough to work up a way of doing it. The eventual solution requires a gnarly combination of two techniques, however.

Remember VETRO? In my current project, I have to process an incoming message which is defined by an XSD file included into an XSD file that is referenced by the service’s WSDL. If you followed that chain without getting confused you are doing better than OSB. Because of the include chain, OSB’s validate step won’t find the message type. The WSDL and XSDs are from a third-party vendor and so I’m loathe to touch/munge them.

This means that OSB is only capable of ‘…ETRO.’ Quality stuff, this!

Deployment is ‘strange’ as well. Took me a while to find the up-to-date doco telling you how to script the creation of an OSB domain from scratch. There are a few helpful posts out there telling you what to script after you get this beastie but so far this in in the same basket as XQuery unit testing. Seems to me that “Thou shalt not apply your existing deployment plan to a new domain.” Not if you want to keep your sanity, anyway.

Other

In no particular order:

• Eclipse IDE console’s “scroll lock” doesn’t work. You search through a long log trace looking for something and then–just as the line appears–the console refreshes, leaving you staring at the latest line to be added. Good grief but this is frustrating! I like Eclipse, have used it lots over the years (by choice, note!) but in this incarnation it is simply hateful, darn it!
• From the sublime to the ridiculous: sometimes the console doesn’t actually deign to show anything at all. Many times I hear: “Bob. A message has gone missing again.” I now know that 9 times out of 10 what has actually happened is that the Eclipse console is blocking the server as it is trying to send debugging to System.out (I’m not directly using System.out please note…I’m not that naive). A quick keypress into the window unfreezes everything and Lo! I have miraculously found the missing message. It’s strange but even miracles can get old, fast.
• JDeveloper periodically corrupts a project. One may work on a project for a week and then BAM! it fails to load up in the morning (this may be a local SOE issue but its still frustrating and I don’t get hassles from any other apps).
• There exists a fairly small user community. The Oracle forums are really not that active or useful. Apologies to those members out that that have tried their best to help me out…this comment is not intended to be an indictment of you guys (I AM grateful, really, I AM!). I am just trying to identify issues caused by community size & experience.
• OSB’s Eclipse tooling tries to deploy all the projects it can find…even if they are closed. This can lead to weird situations. You Have Been Warned…
• It’s not just the older versions, either. In the latest update, Eclipse seems to take ages to realise that a change has happened and that a deployment refresh is needed…and then sometimes fails to see that a redeploy has happened.
• …and what has happened to Weblogic? Even a simple change to a datasource’s configuration now requires a reboot. Shocking!
• The weblogic log browser is…’idiosyncratic.’ You can be eagerly following a chain of logging messages…clicking from one page to the next…homing in on the cause of all life’s woes…feeling the expectation of a debugging session drawing to a close welling up inside…and then bloody thing will decide to show page 0 again. Just for shits and giggles, it’ll be so confused that you will have to start a new web browser session. You quickly learn to munge the parameters displayed in the address bar, not that there’s anything wrong with that, of course…

My colleague David has introduced me to a term that sums all this up nicely: complification. To complify something is “To dramatically increase the difficulty of something for no good or apparent reason.”

Of course, I am an ingenious guy and I can pull some sort of solution to all of these things out of my expansive…err…mind. AFWSE is also my friend in many situations (but see my earlier comments regarding the size and vitality of the user community).

I am sure that you, dear browser, also pride yourself on your ingenuity and proven ability to wrestle a recalcitrant universe into shape.

This, Of course, is also the trouble! Ego comes riding into town…

My (rhetorical) question is: should we have to be so ingenious? Especially when many of the ‘issues’ are those that OSB in and of itself causes?

So here we are: back to the title…I can’t help but imagine that somewhere within Oracle there is a poor, downtrodden group afraid to tell King Larry that OSB just doesn’t make sense.

Guys, let me try. I’ll be the plucky little boy from the crowd (I hope that remember your Hans Christian Anderson fairy tale?) that doesn’t understand he should be following the party line…

Larry, listen up! In my (not-so-)humble-opinion there are much better ways to skin the integration cat that what I see here. OSB isn’t a worthy member of your product lineup. Don’t lead your customers down this particular garden path. The last decade has seen some good steady quiet progress in areas such as testing, interoperability and frameworks (to name but a few), please don’t set this industry back to the pre-2000s with this regressive, locked-down, locked-in, half-baked ‘thing.’ Please!

Phew! Said it. I Feel much better now!

Lest you think that I am losing it, I’m not the only person saying this sort of thing.

You know…the story doesn’t really say what happened to that plucky little boy from the crowd. Maybe he was showered in riches for saving the King’s honor. More likely, he was beheaded for embarrasing the high-and-mighty.

[edit]
Some almost immediate–and pretty much equally jaundiced and cynical (although slightly more generic)–reaction.

Thank you – a rather excellent way to start my day :-)

Now let me also share something. You may or may not agree. Oracle makes a lot (LOT) of money by sending in the consultants.

The consultants get called in when the local team comes up with zip because it’s so damned hard to get the Oracle tools to work.

Consultants are charged out at $2k / day and become an on-going income stream as the client needs lots of refresh (code change) activity.

It serves the likes of the big consulting houses to have internally trained specialists who know nothing else but these ghastly tools.

In fact some outside consultants cash in big time by spending all their time focusing on nothing but – discovering all the tips and tricks from the Oracle paid guys.

I think this has been going on since I first entered the industry and it’s a great model. Not for the industry nor the profession, but for money making monsters of which Oracle is one of the best.

There, my take…take it or leave it, but it’s ultimate raison d’etre of the Closed source movement and some clients just gravitate to it…

I’ll leave the sender anonymous…no sense in taking other people’s career down alongside my own ;-)

GroovyMag is very nice in that it allows the author to republish their work after a ‘decent’ interval has passed (either 60 or 90 days, can’t quite remember at the moment). That period is well and truly passed for this article so here it is in its full resplendency. Enjoy!

[edit]
Forgot to make the source available when I first republished this article. Silly me. All fixed now!

Solving the Enterprise Integration Puzzle
Getting Started with Spring Integration

Groovy is frequently promoted as “the scripting language for the JVM.” Grails is often described as being “great for smaller, quick-to-build web applications.” Neither of these pieces of perceived wisdom really gives the Groovy/Grails combination its due: a powerful tool in the Systems Integrator’s armory. This article will show how Groovy and Grails–in conjunction with SpringSource’s up-and-coming “Spring Integration” project–can make complex systems integration tasks as easy as building a standalone web application.

A Brief Overview of Spring Integration

Systems integration is frequently likened to attempting to piece together a badly made jigsaw puzzle. The end goal is often quite fuzzy; there may be several potential solutions to consider; there may be many conflicting approaches to evaluate; and when all is said and done, sometimes all that can be done is to say “To heck with the Grand Plan!” and ‘adapt’ the puzzle using a pair of nice sharp scissors.

Spring Integration (SI) is a recent project within the Spring stable that provides numerous tools to make the systems integration task less frustrating. SI builds on several existing Spring stablemates, but provides a unified way of configuring these components and linking them together. According to SI, all message endpoints (which act with and on behalf of application components) interact via messages sent over various flavors of messaging channels, perhaps undergoing various transformations along the way. In addition, external resources (files, JMS-based queueing systems, SOAP-based WebServices, etc.)  interact with these channels via simple-to-configure adapters.

SI imposes very little onto the developer; almost everything in the system can be implemented using simple POG/JOs (Plain Old Groovy/Java Objects) and configuration is typically via XML or Annotations.

SI provides out-of-the-box support for many of the patterns defined in Hohpe and Woolfe’s Enterprise Integration Patterns book and associated website and–as this article will show–is quite “Groovy friendly.”

A Small Example Application

In this article, I am going to take a very simple systems integration task and build a small example application. The application is a small Grails application that performs a simple function: given a customer number, it calculates the shipping cost for a single widget.

To show how Spring Integration can be used, I am assuming that a customer will enter their customer ID into a Grails-originated web page, which is then routed to one instance of a set of JMS servers. The associated address data is then retrieved, the city field extracted and then passed on to a SOAP-based WebService that calculates the cost of shipping widgets from Brisbane to the customer’s home city. To add a smidgen of realism, I have built the various services to be completely unrelated; in particular, they do not reference any form of standardized messaging schema. The integration pipeline needs to handle various simple message types, including strings, CSV-formatted data records, XML-formatted strings and floats.

Figure 1 provides a high-level view of the application.

Figure 1: The processing flow and major components comprising the example application

The various steps highlighted in this flow are:

1. The customer ID (a simple string) is entered into the Grails web application and passed off to a generic SI gateway. The gateway places the customer ID on an internal channel that leads to a message router.
2. The router retrieves the message from its inbound channel and determines which available channel to use to ensure that the message is handled by the appropriate JMS server instance.
3. The message is then handled by a SI JMS “channel adapter,” which deals with the mechanics of working with the underlying external JMS server.
4. The nominated JMS server instance produces a record (a CSV-formatted string) that is placed onto a shared reply channel.
5. The CSV record is removed from the shared channel and transformed. The result of the transformation is an XML-formatted string which is again placed on a ‘downstream’ queue.
6. The WebService channel adaptor removes the string from its request channel and invokes the external SOAP WebService for further processing.
7. The WebService response (a single float) is then passed back to the originating messaging gateway, which returns it to the Grails controller.

This is a very simple system and it may seem that this is overkill. There is a payoff, however: SI makes it easy to plug in disparate (often legacy) systems and thus produces an expandable framework that is capable of being built out to cover future, more complex, situations.

The Pieces of the Jigsaw Puzzle

Before starting out on the integration exercise proper, it is worth taking a quick look at the various pieces of the jigsaw puzzle that are going to be joined together.

You will quickly see that all the pieces are exceptionally simplistic. I haven’t even tried to make them correspond to a multitude of real-life issues such as error handling, logging, configuration, etc., which have been almost completely ignored. This is by design: for the purposes of this article, I want to focus mostly on SI, not on the various services.

The Grails Web Application

Figure 2 shows the Grails web interface “in action.” It is very simple, with only a single form and a single results page.

Figure 2: The deployed Grails web application

The ‘meat’ of the application (as far as this article is concerned) lies in the single Grails controller class shown in Listing 1.

public class SpringIntegrationController {

  def siGateway

  def index = { }

  def submit = {
    long start = System.currentTimeMillis()
    def msg = params.custid
    siGateway.send(msg)
    def got = siGateway.receive()
    flash.message =
      "Query took: ${System.currentTimeMillis() - start} millis."
    [shippingCost: got]
    }
  }

Listing 1: The Grails controller class

There are two points of interest in this simple class.

The siGateway instance injected from the Spring application context constitutes the sole point of contact between the Grails web application and the SI framework. (In theory, SI allows for an even less intrusive interface than this, based around dynamically constructed proxies, but a bug surfaced as I started writing the application…no matter, the hard way is not too onerous to use and I am sure that the SI guys will soon squash the bug.)

The submit closure is responsible for handling the POSTed form data and for driving the injected siGateway instance through its paces. It is pretty straightforward: all the action is happening over in “SI-land.”

The JMS Application

For the example application, I have written a very simple JMS service (using Apache ActiveMQ as the messaging provider), as shown in Listing 2.

import javax.jms.Session
import org.apache.activemq.ActiveMQConnectionFactory

public class AMQService
{
  private static factory =
	  new ActiveMQConnectionFactory("tcp://localhost:61616")

  public static void main(String[] args) throws Exception
  {
    def cKey = args[0]
    def cFile = args[1]

    println 'Starting...'
    println "Config key: $cKey"
    println "Config file: $cFile"

    def config =
      new ConfigSlurper(cKey).parse(new File(cFile).toURL())

    def database = config.database
    println "Database: $database"

    def inQ = config.inQ
    println "Incoming Queue: $inQ"

    def replyTopic = config.replyTopic
    println "Reply Topic: $replyTopic"

    def qConn = factory.createQueueConnection()
    def qSession =
      qConn.createQueueSession(false, Session.AUTO_ACKNOWLEDGE)
    def consumer =
      qSession.createConsumer(qSession.createQueue(inQ))

    def tConn = factory.createTopicConnection()
    def tSession =
      qConn.createTopicSession(false, Session.AUTO_ACKNOWLEDGE)
    def producer =
      tSession.createProducer(tSession.createTopic(replyTopic))

    qConn.start()
    tConn.start()

    for ( ; ; )
      process(database,
              tSession.createTextMessage(),
              consumer,
              producer)
  }

  private static process(database, replyMessage, consumer, producer)
  {
    try
    {
      def textMessage = consumer.receive()  // indefinite blocking
      replyMessage.with {
        setText(getResponse(database, textMessage.getText()))
        setJMSCorrelationID(textMessage.getJMSCorrelationID())
      }
      producer.send(replyMessage)
    }
    catch (Throwable t)
    {
      t.printStackTrace()
    }
  }

  private static getResponse(database, inMsg)
  {
    def resp = database[inMsg]
    println "Request: '$inMsg'; Response: '$resp'"
    resp
  }
}

Listing 2: The JMS server

This is a standard JMS service; no need to elaborate.

One point to note (that is not directly SI-related, but of interest nonetheless) is that the service is written to allow multiple instances to run at the same time, dealing with a partitioned data space. For the purposes of the example application, I have configured two instances. One instance is prepared to deal with customer IDs whose first character is in the range a-l and the second will deal with the rest of the alphabet.

The service accepts input via a uniquely-designated input queue, but sends its response to a shared topic. This structure simplifies things by ensuring that any downstream component merely has to handle a single publish/subscribe channel (which can carry data from multiple sources), rather than trying to deal with a multitude of point-to-point links.

Each server instance is driven by a pair of application parameters, defining:

• The location of the configuration file to use
• The appropriate environment section in the designated configuration file

The configuration file supplies:

• The name of the input queue to use
• The name of the output topic to receive the responses
• The actual static database to use (I did warn that these applications were extremely simple!)

The service makes use of ConfigSlurper’s ability to deal with different environment sections and so I can define a single configuration file (see Listing 3) to configure the different instances uniquely.

replyTopic="SI.REPLY.TOPIC"

replyTopic="SI.REPLY.TOPIC"

environments {
  al {
    inQ="SI.AL.REQUEST"
    database=['b9876':'123 Nowhere St,Brisbane',
              'f0234':'42 Imaginary Place,Darwin']
    }
  mz {
    inQ="SI.MZ.REQUEST"
    database=['x5555':'123 Fake St,Hobart',
              'w8888':'000 Talkfest Lane,Canberra']
    }
  }

Listing 3: JMS server configuration file

This service accepts a string, but returns a CSV-formatted record.

The WebService

The final part in the jigsaw puzzle is a SOAP WebService built on top of the facilities provided by GroovyWS.

The service’s raison-d’être is to retrieve the cost of shipping n widgets between major cities in Australia; see Listing 4 and Listing 5.

public class ShippingCostWebService {

  static costs = [
    Brisbane: [ Brisbane:0.0F,
                Darwin:44.44F,
                Canberra:22.22F,
                Hobart:88.88F ],
    Darwin: [ Brisbane:44.44F,
              Darwin:0.0F,
              Canberra: 33.33F,
              Hobart: 99.99F ],
    Canberra: [ Brisbane:22.22F,
                Darwin:33.33F,
                Canberra: 0.0F,
                Hobart: 77.77F ],
    Hobart: [ Brisbane:88.88F,
              Darwin:99.99F,
              Canberra: 77.77F,
              Hobart: 0.0F ],
  ]

  Float calculateShippingCost(String fromLoc,
                              String toLoc,
                              Integer nItem) {
    def cost = costs[fromLoc][toLoc] * nItem
    println "calculateShippingCost($fromLoc,$toLoc,$nItem) => $cost"
    cost
    }
}

Listing 4: The shipping cost WebService

println 'Starting...'

def ws = new groovyx.net.ws.WSServer()

ws.setNode("ShippingCostWebService",
           "http://localhost:6980/ShippingCostWebService")

ws.start()

println '...Started.'

Listing 5: The GroovyWS WebService server harness

I can’t lie! This is just an old WebService that I had lying around that I created for a conference presentation, and which I have adopted for this application. It doesn’t have a customized API and isn’t necessarily a perfect fit to the task. In the systems integration world, improvisation is often required to squeeze data out of the numerous ‘sub-optimal’ nooks and crannies that are found all over an enterprise.

Joining the Pieces Together

Now that you are familiar with the basic componentry, it is time to look at SI proper.

As with any Spring project, SI is configured separately to the actual code in an application. Since this application is based on Grails, I have chosen to use the nice resources.groovy Grails Spring Beans DSL configuration file, rather than the ‘traditional’ XML-based configuration.

Listing 6 shows the complete configuration.

beans = {
  xmlns si:"http://www.springframework.org/schema/integration"
  xmlns jms:"http://www.springframework.org/schema/integration/jms"
  xmlns stream:"http://www.springframework.org/schema/integration/stream"
  xmlns ws:"http://www.springframework.org/schema/integration/ws"

  // SI componentry and plumbing
  si {
    poller(default: true) {
      "interval-trigger"(interval: 1, "time-unit": "SECONDS")
    }
    router("input-channel": "routerChannel",
           ref: "highLowRouter",
           method: "route")
    transformer("input-channel": "InboundTopic",
                "output-channel": "TransformedInbound",
                ref: "csvStringTransformer",
                method: "transform")
    channel(id: "routerChannel")
    channel(id: "ALOutboundChannel")
    channel(id: "MZOutboundChannel")
    channel(id: "RoutingRejectChannel") {
      queue(capacity: "256")
    }
    channel(id: "TransformedInbound") {
      queue(capacity: "16")
    }
    channel(id: "InboundTopic") {
      queue(capacity: "16")
    }
    channel(id: "ShippingCostChannel",
            dataType: "java.lang.Float") {
      queue(capacity: "16")
    }
  }

  stream {
    "stderr-channel-adapter"(channel:"errorChannel",
                             "append-newline": true)
  }

  // WebService stuff
  ws {
    "outbound-gateway"("request-channel": "TransformedInbound",
                       "reply-channel": "ShippingCostChannel",
                        uri: "http://localhost:6980/ShippingCostWebService")
  }

  // AMQ/JMS stuff
  connectionFactory(org.apache.activemq.pool.PooledConnectionFactory) { bean ->
    bean.destroyMethod = "stop"
    connectionFactory = { org.apache.activemq.ActiveMQConnectionFactory cf ->
      brokerURL = "tcp://localhost:61616"
    }
  }

  "AL.REQUEST"(org.apache.activemq.command.ActiveMQQueue, "SI.AL.REQUEST")
  "MZ.REQUEST"(org.apache.activemq.command.ActiveMQQueue, "SI.MZ.REQUEST")
  "REPLY"(org.apache.activemq.command.ActiveMQTopic, "SI.REPLY.TOPIC")

  jms {
    "outbound-channel-adapter"(channel: "ALOutboundChannel",
                               destination: "AL.REQUEST")

    "outbound-channel-adapter"(channel: "MZOutboundChannel",
                               destination: "MZ.REQUEST")

    "message-driven-channel-adapter"(channel: "InboundTopic",
                                     destination: "REPLY")
   }

  // general componentry
  siGateway(org.springframework.integration.gateway.SimpleMessagingGateway) {
    requestChannel = ref("routerChannel")
    replyChannel = ref("ShippingCostChannel")
    replyTimeout = "10000"
  }

  highLowRouter(au.com.transentia.si.LookupRouter)

  csvStringTransformer(au.com.transentia.si.CsvTransformer) {
    homeBase = 'Brisbane'
  }
}

Listing 6: The Spring Integration configuration

As you read the following discussion, you may find it useful to refer back to Figure 1.

The configuration is structured into various sections, according to the needs of SI and of the various resources that are being integrated.

The si section (more precisely, those elements of SI configured via the XML namespace allocated the si prefix in this document) configures the various aspects of SI itself. This includes establishing the various channels that interlink the components, defining a few application-specific components (such as the router and transformer), and putting in place a default polling schedule for those parts of SI that need to poll.

It is worth examining the router and transformer components here. I have mentioned that the JMS services deal with a partitioned data space and that traffic is directed to an individual service based on the actual data message being processed. SI allows for the definition of an application-specific router to perform this type of task, in this case the method route of the highLowRouter instance. Listing 7 shows the code for the router.

package au.com.transentia.si

public class LookupRouter {

  public String route(String msg) {
    switch(msg) {
      case ~/(?i:[a-l].*)/: return 'ALOutboundChannel'
      case ~/(?i:[m-z].*)/: return 'MZOutboundChannel'
      default: return 'reject'
    }
  }
}

Listing 7: The router class

The router examines the incoming message data and determines the appropriate outbound path for the message. Groovy’s wonderfully versatile switch statement, combined with the ability to do case-independent pattern matching, makes for beautifully minimalistic code.

The transformer is equally simple, as Listing 8 shows.

package au.com.transentia.si

public class CsvTransformer {
  def homeBase

  public String transform(String csv) {
    def dest = csv.tokenize(',')[1]

    """
         $homeBase
         $dest
         1
      """
  }
}

Listing 8: The transformer class

This code is presented with an input CSV-formatted record, extracts the relevant field, and passes an XML-formatted string onwards.

Recall that the WebService’s API is not custom-built for the purpose of this application; for this use, arg2 (the nItems parameter) is always fixed at 1.

The stream section of the configuration merely allocates a generic channel listener so that the standard errorChannel is not silently ignored; this configuration will ensure that errors/exceptions, etc., are logged.

The ws section deals–unsurprisingly–with WebService integration. The section configures an SI outbound gateway that is capable of receiving a request on a specified channel, invoking the configured WebService and finally sending the result to the appropriate response channel. All this is specified declaratively: the actual mechanism of dealing with the WebService (retrieving WSDL documents, generating proxies and handling XML-based request/response messages) is completely hidden away.

The remaining configuration concerns SI’s JMS adapter and ActiveMQ integration.

The siGateway has been discussed previously.

By default, all SI’s JMS adapters reference a bean with the standard name of connectionFactory. It is the responsibility of this bean to ‘vend’ connections to the external JMS system. In this case, a connection pool to a local Apache ActiveMQ server running at the URL “tcp://localhost:61616″ is being created. Since the external connections are pooled for efficiency, by specifying destroyMethod=’stop’ we ensure that they are simply inactivated after each (re)use and are never actually discarded.

Note: I found the actual syntax used to declare the connectionFactory a little tricky. I had to ask for help on the excellent user@grails.codehaus.org mailing list to get it right! Thanks to the generous souls who pointed me in the right direction.

Following the definition of the the connectionFactory, we create two instances of ActiveMQ queues and an instance of an ActiveMQ topic. It is these that carry the messages to/from the external JMS resources.

The jms section configures two JMS-specific outbound channel adapters and one inbound channel adapter. As the names suggest, these are concerned with driving (or being driven by) the ActiveMQ instances declared above. The configured message-driven-channel-adapter instance defines a component that will be asynchronously “actively invoked” to handle an incoming message. It is possible to configure a polling-oriented component, but this is generally less flexible, and may waste CPU, and increase the latency in the system.

As with the WebService gateway, everything is declarative.

Admiring the Finished Puzzle

SI has enabled a deceptively simple solution to a fairly complex problem. There are no messy tracts of confusing mechanistic code; the system is essentially defined all in a single configuration; the individual components are simple and (generally) reusable; and the groundwork for coping with change set with very little effort.

SI is a simple but powerful toolkit; the example application I have discussed here has really just touched the surface of what is possible. SI provides much more, including facilities for secure channels, inbound/outbound mail handling, event handling, RMI integration and much better XML handling than I have touched upon here.

Remember that these are still early days for SI: for example, there are no debuggers or GUI editors as found in the big “name brand” systems integration tools. In time these tools may come but for now remember that SI is orders of magnitude simpler, and the SI/Groovy/Grails triumvirate makes the developer many times more productive than some other tools I have used.

I hope that I have stimulated your interest in SI and shown that a Grails application need not be limited to serving up “little web apps” but can actually service the needs of larger-scale systems integration tasks.

I hope that I have also been able to show that when the time comes to get out the scissors, Groovy is more than equal to the task of “slicing and dicing” until the pieces of the systems integration jigsaw puzzle fit nicely together.

Learn More

• Gregor Hohpe and Bobby Woolf, Enterprise Integration Patterns : Designing, Building, and Deploying Messaging Solutions (ISBN 0321200683), also http://www.enterpriseintegrationpatterns.com/
• Spring Integration Home, http://www.springsource.org/spring-integration
• Apche ActiveMQ, http://activemq.apache.org
• GroovyWS, http://docs.codehaus.org/display/GROOVY/GroovyWS. The exemplar application uses the 0.5 snapshot jars from http://snapshots.dist.codehaus.org/groovy/distributions/groovyws/

Bob Brown is the director and owner of Transentia Pty. Ltd.. Based in beautiful Brisbane, Australia, Bob is a specialist in Enterprise Java and has found his niche identifying and applying leading-edge technologies and techniques to customers problems.

I am opening up old courseware to the world on a “pay if you like it” or have it free basis.

My fourth offering is SOAP/WebServices Overview.
(You will find other offerings in this site’s DonationWare Category.)

This course was created in mid-2003.

It was a quick, one-day overview/review of the current “state of the art.”

How things change!

I remember going down to Sydney to give this course on behalf of “A Famous Database Company” for a group of South Korean V.I.P.s…who (it turned out) did not speak ANY English. The poor translator almost had a nervous breakdown. It was a very strange session, almost surreal!

Here’s the ‘blurb’:

The course is now well and truly at End of Life.

I figure that it would be such a pity for it to end as a set of bits decaying away on my hard disk so I am opening it up to the world on as “as-is” basis:

sws-courseware-donationware.04.july.2009.zip
(MD5: 150f88d70d5a86d1129af4e21059594c; size: 6,692,573 bytes)

Some (unfortunately necessary) legalese:

• This content is provided “as-is”, with no guarantees.
• Feel free to use it, but not to abuse it (to give a couple of examples: don’t make hundreds of copies for friends; don’t claim it as your own work).
• I retain copyright, so “all rights reserved.”

Enjoy!

If you like it, or have any questions/comments, send me an email ().

If you find this material useful, please consider paying me a small amount: via PayPal.

A few gems:

So having vendors claiming “we are still shifting products” while analysts who told people to go and buy those products are now complaining that they haven’t seen the value really is just a farce… SOA hasn’t failed, what has failed is the next generation lipstick on the pig, buy my technology it will save your problems, do a big project it will fix everything style of IT that has always failed.

And:

…those companies that have not yet started a SOA initiative did so because they could not articulate and demonstrate its business value.

And even:

…you should probably avoid using the word “SOA” and instead focus your efforts on building “services” that deliver measurable value to the business.

What a horrifying idea: “deliver measurable value to the business”! What will they think of next ;-)

Improvements include reporting and upgraded Groovy support.

Useful grist for the mill…

I came across it when I was trying to remind myself what VETRO stood for (”Validate, Enrich, Transform, Route, Operate”…I simply could not remember what the ‘O’ was for…) .

Here’s a taster:

While rank and file developers go to conferences to soak in deep technical content, their peripherally technical managers (the ones who wrote some rockin’ good Cobol code back in the day, but now they make decisions about modern enterprise architecture) go to different conferences in Palm Springs. At those conferences, they have a 2-hour morning session, run by a big tool vendor, then play golf for the balance of the afternoon. And what the vendors show them is poison.

I just love the term “doodleware” the author introduces!

It’s a curious thing: just yesterday I was pontificating on a similar theme to a colleague; it’s nice to know that my cynical world-view is not completely in my own head, after all…

Perhaps the author wanted something along the lines of grailsflow?