I have previously written about how to generate a types set of WCF services to achieve end-to-end BAM based tracking – making BAM logging possible from outside potential firewalls and/or from non .NET based clients.
In that post I reference a open source tool to generate the services based on the BAM definition files to be able to make types BAM API calls. I called the tool BAM Service Generator and it’s published on CodePlex.
The first version of the tool only made it possible to make simple activity tracking calls, even missing the option to defining a custom activity id. All advanced scenarios like continuation, activity and data reference or even updating a activity wasn’t possible. That has now been fixed and the second version of the tool enables the following operations.
LogCompleteActivity
Creates a simple logging line and is the most efficient option if one only want's to begin, update and and end an activity logging as soon as possible.
BeginActivity
Start a new activity with a custom activity id
UpdateActivity
Updates a already started activity and accepts a types object as a parameter. The typed object is of course based on what's has been defined in the BAM definition file. Multiple updates can be made until EndActivity is called.
EnableContinuation
Enables continuation. This is used if you have multiple activity loggings that belong together but you can not use the same activity id. This enables you to tie those related activities together using some other set of id (could for example be the invoice id).
AddActivivityReference
Add a reference to an another activity and enables you to for example display a link or read that whole activity when reading the main one.
AddReference
Adds a custom reference to something that isn’t necessarily an activity. Could for example be a link to a document of some sort. Limited to 1024 characters.
AddLongReference
Similar to AddReference but enables to also store a blog of data using the LongRefernceData parameter – limited to 512 KB.
EndActivity
Ends a started activity.
Let me know if you think that something is missing, if something isn’t working as would expect etc.
An integration between systems can often be view upon as a chain of system and services working together to move a message to its final destination.
One problem with this loosely-coupled way of dealing with message transfers is that its hard to see if and where something has gone wrong. Usually all we know is that the sending system has send the message, but the final system never received it. The problem could then be within any of the components in between. Tracking messages within BizTalk is one thing but achieving a end-to-end tracking within the whole “chain” is much harder.
One way of solving the end-to-end tracking problem is to use BAM.aspx). BAM is optimized for these kind of scenarios were we might have to deal with huge amount of messages and data – several of the potential problems around these issues are solved out-of-the-box (write optimized tables, partitioning of tables, aggregated views, archiving jobs and so on).
And even though BAM is a product that in theory isn’t tied to BizTalk its still a product that is easier to use in the context of BizTalk than outside of it due tools like the BizTalk Tracking Profile Editor.aspx) and the built in BAM-interceptor patterns within BizTalk. It is however fully possible to track BAM data and take advantage of the BAM infrastructure even outside of BizTalk using the BAM API.aspx).
The BAM API is a .NET based API used for writing tracking data to the BAM infrastructure from any .NET based application. There are however a few issues with this approach.
The application sending the tracking data has to be .NET based and use the loose string based API. So for example writing to the “ApprovedInvoice” activity below would look something like this – lots of untyped strings.
public void Log(ApprovedInvoicesServiceType value)
{
string approvedInvoicesServiceActivityID = Guid.NewGuid().ToString();
var es = new DirectEventStream("Integrated Security=SSPI;Data Source=.;Initial Catalog=BAMPrimaryImportw", 1);
es.BeginActivity("ApprovedInvoices", approvedInvoicesServiceActivityID);
es.UpdateActivity("ApprovedInvoices", approvedInvoicesServiceActivityID,
"ApprovedDate", "2011-01-18 12:02",
"ApprovedBy", "Richard",
"Amount", 122.34,
"InvoiceId", "Invoic123");
es.EndActivity("ApprovedInvoices", approvedInvoicesServiceActivityID);
}
One way of getting around problem with the usage of strings is to use the “Generate Typed BAM API tool”. The tool reads the BAM definition file and generates a dll containing strong types that corresponds to the fields in the definition. By referencing the dll in the sending application we can get a strong typed .NET call. The fact that this still however requires a .NET based application remains.
The obvious way to solve the limitation of a .NET based client - and to get one step closer to a end-to-end tracking scenario - is of course to wrap the call to the API in a service.
BAM Service Generator
BAM Service Generator is very similar to the Generate Typed BAM API tool mentioned above with the difference that instead of generation a .NET dll it generates a WCF service for each activity.
c:\Tools\bmsrvgen.exe /help
BAM Service Generator Version 1.0
Generates a WCF service based on a BizTalk BAM definition file.
-defintionfile: Sets path to BAM definition file.
-output: Sets path to output folder.
-namespace: Sets namespace to use.
Example: bmsrvgen.exe -defintionfile:c:\MyFiles\MyActivityDef.xml
-output:c:\tempservices -nampespace:MyCompanyNamespace
The BAM Service Generator is a command line tool that will read the BAM definition file and generate a compiled .NET 4.0 WCF service. The service is configured with a default basicHttp endpoint and is ready to go straight into AppFabric or similar hosting.
This service-approach makes it possible to take advantage of the BAM infrastructure from all different types of system, and even in cases when they aren’t behind the same firewall! As shown in figure below this could take us one step closer to the end-to-end tracking scenario.
“BAM Service Generator” is open-source and can be found here.
I’ve doing a lot of EDI related work in BizTalk lately and I have to say that I’ve really enjoyed it! EDI takes a while to get used to (see example below), but once one started to understand it I’ve found it to be a real nice, strict standard - with some cool features built into BizTalk!
UNB+IATB:1+6XPPC+LHPPC+940101:0950+1'
UNH+1+PAORES:93:1:IA'
MSG+1:45'
IFT+3+XYZCOMPANY AVAILABILITY'
ERC+A7V:1:AMD'
IFT+3+NO MORE FLIGHTS'
ODI'
TVL+240493:1000::1220+FRA+JFK+DL+400+C'
...
There are however some things that doesn't work as expected …
Promoting values
According to the MSDN documentation.aspx) the EDI Disassembler by default promotes the following EDI fields: UNB2.1, UNB2.3, UNB3.1, UNB11; UNG1, UNG2.1, UNG3.1; UNH2.1, UNH2.2, UNH2.3.
There are however situation where one would like other values promoted.
I my case I wanted the C002/1001 value in the BGM segment. This is a value identifying the purpose of the document and I needed to route the incoming message based on the value.
The short version is that creating a property schema, promoting the field in the schema and having the EDI Disassembler promoting the value will not work (as with the XML Disassembler). To do this you’ll need to use a custom pipeline component to promote the value. Rikard Alard seem to have come to the same conclusion here.
Promote pipeline component to use
If you don’t want to spend time on writing your own pipeline component to do this yourself you can find a nice “promote component” on CodePlex here by Jan Eliasen.
If you however expect to receive lots and lots of big messages you might want to look into changing the component to use XPathReader and custom stream implementations in the Microsoft.BizTalk.Streaming.dll. You can find more detailed information on how to do that in this MSDN article.aspx).
Update 2010-04-13: Grant Samuels commented and made me aware of the fact that inline scripts might in some cases cause memory leaks. He has some further information here and you’ll find a kb-article here.
I’ve posted a few times before on how powerful I think it is in complex mapping to be able to replace the BizTalk Mapper with a custom XSLT script (here’s how to.aspx)). The BizTalk Mapper is nice and productive in simpler scenarios but in my experience it break down in more complex ones and maintaining a good overview is hard. I’m however looking forward to the new version of the tool in BizTalk 2010 – but until then I’m using custom XSLT when things gets complicated.
Custom XSLT however lacks a few things once has gotten used to have - such as scripting blocks, clever functoids etc. In some previously post (here and here) I’ve talked about using EXSLT as a way to extend the capabilities of custom XSLT when used in BizTalk.
Bye, bye external libraries – heeeello inline scripts ;)
Another way to achieve much of the same functionality even easier is to use embedded scripting that’s supported by the XslTransform class. Using a script block in XSLT is easy and is also the way the BizTalk Mapper makes it possible to include C# snippets right into your maps.
Have a look at the following XSLT sample:
<?xml version="1.0" encoding="utf-8"?>
<xsl:stylesheet
version="1.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:msxsl="urn:schemas-microsoft-com:xslt"
xmlns:code="http://richardhallgren.com/Sample/XsltCode"
exclude-result-prefixes="msxsl code"
>
<xsl:output method="xml" indent="yes"/>
<xsl:template match="@* | node()">
<Test>
<UniqueNumber>
<xsl:value-of select="code:GetUniqueId()" />
</UniqueNumber>
<SpecialDateFormat>
<xsl:value-of select="code:GetInternationalDateFormat('11/16/2003')" />
</SpecialDateFormat>
<IncludesBizTalk>
<xsl:value-of select="code:IncludesSpecialWord('This is a text with BizTalk in it', 'BizTalk')" />
</IncludesBizTalk>
</Test>
</xsl:template>
<msxsl:script language="CSharp" implements-prefix="code">
//Gets a unique id based on a guid
public string GetUniqueId()
{
return Guid.NewGuid().ToString();
}
//Formats US based dates to standard international
public string GetInternationalDateFormat(String date)
{
return DateTime.Parse(date, new System.Globalization.CultureInfo("en-US")).ToString("yyyy-MM-dd");
}
//Use regular expression to look for a pattern in a string
public bool IncludesSpecialWord(String s, String pattern)
{
Regex rx = new Regex(pattern);
return rx.Match(s).Success;
}
</msxsl:script>
</xsl:stylesheet>
All one has to do is to define a code block, reference the xml-namespace used and start coding! Say goodbye to all those external library dlls!
It’s possible to use a few core namespaces without the full .NET namespace path but all namespaces are available as long as they are fully qualified. MSDN has a great page with all the details here.aspx).
Update 2012-03-28: This bug was fixed in the CU2 update for BizTalk 2006 R2 SP 1. Further details can be found here and here.
Update 2010-04-12: Seems like there is a patch coming that should fix all the bugs in SP1 … I’ve been told it should be public within a week or two. I’ll make sure to update the post as I know more. Our problem is still unsolved.
Late Thursday night last week we decided to upgrade one of our largest BizTalk 2006 R2 environment to recently released Service Pack 1.aspx). The installation went fine and everything looked good.
… But after a while we started see loads of error messages looking like below.
Unable to cast COM object of type 'System.__ComObject' to interface type
'Microsoft.BizTalk.PipelineOM.IInterceptor'. This operation failed because the
QueryInterface call on the COM component for the interface with IID
'{24394515-91A3-4CF7-96A6-0891C6FB1360}' failed due to the following error: Interface not
registered (Exception from HRESULT: 0x80040155).
After lots of investigation we found out that we got the errors on ports with the follow criteria:
Send port
Uses the SQL Server adapter
Has a mapping on the port
Has a BAM tracking profile associated with the port
In our environment the tracking on the port is on “SendDateTime” from the “Messaging Property Schema”. We haven’t looked further into if just any BAM tracking associated with port causes the error or if only has to do with some specific properties.
Reproduce it to prove it!
I’ve setup a really simple sample solution to reproduce the problem. Download it here.
The sample receives a XML file, maps it on the send port to schema made to match the store procedure. It also uses a dead simple tracking definition and profile to track a milestone on the send port.
Sample solution installation instructions
Create a database called “Test”
Run the two SQL scripts (“TBL_CreateIds.sql” and “SP_CreateAddID.sql”) in the solution to create the necessary table and store procedure
Deploy the BizTalk solution just using simple deploy from Visual Studio
Apply the binding file (“Binding.xml”) found in the solution
Run the BM.exe tool to deploy the BAM tracking defintion.
Should look something like:
bm.exe deploy-all -definitionfile:\BAM\SimpleTestTrackingDefinition.xml
Start the tracking Profile editor and open the “SimpleTestTrackingProfile.btt“ that you’ll find in the solution and apply the profile
Drop the test file in the receive folder (“InSchema_output.xml”)
The sample solution fails on a environment with SP 1 but works just fine on a “clean” BizTalk 2006 R2 environment.
What about you?
I haven’t had time to test this on a BizTalk 2009 environment but I’ll update the post as soon as I get around to it.
We also currently have a support case with Microsoft on this and I’ll make sure to let you as soon as something comes out of that. But until then I’d be really grateful to hear from you if any of you have the same behavior in your BizTalk 2006 R2 SP1 environment.
