Debatching Messages Into Smaller Batches In An Orchestration

It is a common request. How can I debatch a message into smaller batches rather than single messages inside BizTalk 2004?

I have tried many times to get this accomplished natively inside the pipeline. One would think it would be as easy as setting the group property to 10 if you wanted a batch size of 10, but that does not seem to work. The only way I have been able to break up a message into smaller batches has been inside custom code.

I wrote a .net component that can be called using an atomic scope shape from inside an Orchestration. This component will return smaller batches of any size you specify inside the Orchestration and wrap the smaller batches inside a specified root node. This .net component uses the XmlTextReader to stream the message along and debatch it.

This .net component is called StreamHelper. It illustrates the following:

• Using XmlTextReader
  
• Passing an Xlang Message into a .net component
  
• Returning an Xlang Message from a .net component
  
• Setting context properties on a message returned from a .net component

Download: Debatch Streaming Sample

Setup: Download and extract the solution. Build and GAC the StreamHelper assembly then Build and Deploy the DebatchProperties and DebatchOrch assemblies. I have included a basic sample file to test with. For more detailed information on the sample see the comments throughout the solution.

A better way to do it?

I’m sure there is. I just put this together in a short amount of time. I would guess the most efficient place for this type of work would be inside a custom pipeline component. But, working with custom pipelines can be difficult.

How does this debatching approach perform compared to the others?

Performance was almost identical to that of the Xml Node List inside an atomic scope. In this sample, I used the XmlTextReader and passed Xlang messages in and out of the component to try to increase performance. That did not seem to accomplish anything. As expected, as the total number of messages inside the atomic scope increased the performance decreased.

For more debatching information please see my past post:
Debatching Options and Performance Considerations in BizTalk 2004