Cicero Rautenbach

After some recent projects I did and looking at the performance of other projects I am beginning to form a definite opinion about using an ESB for batch processing.

What do I consider to be batch processing? Gathering up data for a period of time, daily, weekly, monthly, quarterly or yearly and then submitting it all at once for processing by the ESB. I am definitely not talking about running the ESB in the background.

Firstly it must be stated that there is normally no difference in processing data in batch or in real-time, except for the timing and the volume. The ESB does not have any concept of batch or real-time and can not tell the difference. by definition the ESB is loosely coupled to consumers of its services.

I often hear the comment from customers and consultants that the ESB is adding no value. “I can do everything that the ESB does with a simple Java object or Java Servlet. This is often true but also frequently misguided. The idea behind the ESB is not compete with Java technologies for the job. Using an ESB allows you to configure rather than develop the processes. This allows for easier debugging, maintenance and flexibility in deployment architectures. These benefits are seldom apparent up-front but will become apparent over time. Unfortunately this configuration to tie loosely coupled components together in a flow/sequence or orchestration is not as performant as a tightly coupled Java object or Servlet.

This fact have servere implications for batch processing. If each data element takes 50% longer to process through the ESB than through a Java object, this can add hours to the processing of the batch. Handling errors also becomes a bigger challenge when there is a potential for a high volume of errors.

For these reasons I have become of the opinion that that the enterprise is much better served by executing ESB processes in real-time or near real-time