In a past edition of The BizTalker, I talked about how easy it is to use custom XSLT inside the BizTalk Mapper to sort data. Sorting data inside the map can be doing using inline XSLT like this:
<xsl:for-each select=”Record”>
<xsl:sort select=”Id” data-type=”number” order=”ascending”/>
<xsl:element name=”Customers”>
<xsl:element name=”Id”><xsl:value-of select=”Id” /></xsl:element>
</xsl:element>
</xsl:for-each>
Sometimes sorting just is not enough. What if you also needed to group your data and break it up into single messages? This can be done several different ways including using xpath, .net components, or something external to BizTalk all together; with none of them really being ideal.
Another non-ideal way to accomplish grouping and debatching is to use pure BizTalk Messaging! Yes, pure messaging can group and debatch your data. Now, it is a little strange and takes a few hops through the message box, but it is a relatively simple solution to implement in a short amount of time.
So, how does it work?
Pass 1 will apply a map on the Receive Port to sort the data. The Send Port will apply another map to group the data.
The grouping map using two Functoids. One determines if the current record and the past record have the same Id. The second does the grouping.
The code in these Scripting Functoids looks like this:
True / False
string newNode = “”;
string pastValue = “X”;
public string MakeNewNode(string currentValue)
{
if (pastValue != currentValue)
{
pastValue = currentValue;
newNode = “true”;
}
else
newNode = “false”;
return newNode;
}
Grouping
<xsl:template name=”GroupByID”>
<xsl:param name=”param1″/>
<xsl:param name=”param2″/>
<xsl:if test=”$param1=’true'”>
<xsl:element name=”Group”>
<xsl:for-each select=”//Customers[Id=$param2]”>
<xsl:element name=”Customer”>
<xsl:element name=”Id”><xsl:value-of select=”Id”/></xsl:element>
<xsl:element name=”Name”><xsl:value-of select=”Name”/></xsl:element>
</xsl:element>
</xsl:for-each>
</xsl:element>
</xsl:if>
</xsl:template>
Pass 2 will apply an envelope to split the data inside the Receive Port and publish single messages to the message box.
Download the Sorting, Grouping, & DebatchingSample Code.
See the ReadMe.txt for set up information. Also note that I needed two projects for this solution. That is because the envelope schema needs to be in a separate project due to the way pipelines read the schema collections.
Have you tried using functoid + xslt to do de-duping?
More or less. A similar approach to the debatching should work for debuping. You just need to send a False to the node you want to remove.
Best of luck.
Is it possible to do this but only pull data that is unique? I am doing EDI and we have some customers that send a repeating file, meaning everything is the same except fo the item that is sent for example the Invoice number may be the same on 12 items but I only want one instance of the invoice number
I think so. I did something like that in a past project but the schema was very simple. I grouped the items in order and then created an output node for each unique items using custom Xslt and passing forward False to nodes I wanted to hide.
I%u2019m sure there is a better way%u2026 you might want to post this to the forum. Some of the guys are expert Xslt people and could answer this better than I could.
This is something that I am looking for but heres my problem, I have a file that is structured soemthing like this.
<10-09-06><001><10-10-06><1><5><1.25>
The Date PONum and Delivery date all repeat the same information until the next PO starts. so it would look like this
<10-09-0><001><10-10-06><2><5><1.30>
<10-09-0><001><10-10-06><3><6><.95>
<10-09-0><001><10-10-06><4><7><.50>
<10-09-0><001><10-10-06><5><1><2.50>
<10-09-0><002><10-10-06><1><10><1.25>
<10-09-0><002><10-10-06><2><10><1.30>
What I need is one line for the
and all the items that are asscociated with the PONum to be grouped together and a seperate file for the next PO.
Any Suggestions?