SOA is Green, Not.

I read last week in an InfoWorld email blast an article talking about SOA being green. It caught my eye primarily because I work with SOA and wondered if the author had seen something in the approach that I had not.

Well, he hadn't, which is unfortunate.

For those of you familiar with the common SOA technologies, you undoubtably know that nearly all data manipulations are done via XML. Oracle's SOA suite components BPEL and ESB certainly do that including translating more traditional record-structured data to XML. And, of course, web service SOAP messages are XML.

Therein lies the problem. Data processing in XML is inherently slower than for a traditional record representation due to translation overhead. The same data represented in XML requires greater storage. I use the Oracle SOA Suite to put a web service interface on a database stored procedure which gives me the same functionality accessible either as a direct call through a db link or as a web service call. I typically see slowdowns on the order of 10 fold or more when using the web service call vs. the direct call.

What this means is that large volumes of data such as a batch file load requiring preprocessing using a service, either must take a lot longer or need more horsepower. The latter is where most of my customers go because batch processing windows are fixed at night and they generally don't have the option to extend processing. More horsepower requires more electricity and AC. The volume of data in XML format often requires substantially greater disk space for both data and log files. This is not green at all.

Granted, SOA is ideal for asynchronous low volume messaging between applications or application orchestration, again with low volume. It is only the high volume batch activity that is inappropriate.

There are different middleware tools emerging for handing higher volumes. Oracle has a product called Oracle Data Integrator that does this. I have not used it yet, but it has the specifications for higher throughput.

I believe this will be an emerging area in SOA tool development. There are advantages to translating data from a record form to a canonical XML form, especially when dealing with integration problems where the sources are different from each other yet the core processing is the same. And, batch files are not going away.

There is also the option to not use SOA, but use other, often existing methods to move and load data that do not translate to XML but nonetheless perform well and get the load job done. In my case, I generally use the direct package procedure call through the dblink to gain batch performance and then use the same package procedure now enabled as a web service when called from a UI. The flexibility gets my job done for me.

The lesson here is flexibility. Until the tool vendors come up with approaches that run light without overbite, as the old Dr. Dobb's motto said it best, choice in approach and mindful of performance is the best way currently to stay green.

++B