Back in February I and my college Robin did an presentation on ALM and BizTalk for the BizTalk User Group in Stockholm. In the two hour presentation we talked about things like:
Pros and cons on distributed version control handling systems and BizTalk (more specially Git).
Identifying the right level of automated test – what are the differences between integration and unit tests and how and what should one use when in a BizTalk context.
Using NuGet and BizTalk to handle dependencies and packages. How can we use the NuGet infrastructure to handle all BizTalk dependencies and also distribute artifacts like pipeline components within our company and teams?
How, in our opinion, are Continuous Integration and Delivery best handled when developing with BizTalk. In the presentation we’re looking at everything from build servers to how to handle BizTalk automated builds and packing with minimal amount of work.
So, now we doing the presentation all over again for the newly started BizTalk User Group in Gothenburg the 24 of Mars. The presentation will be in Swedish and packed with demos.
Hope to see you there!
Getting a full Continuous Integration (CI) process working with BizTalk Server is hard!
One of the big advantages in a working CI process is to always have tested and verified artifacts from the build server to deploy into test and production. Packaging these build resources into a deployable unit is however notorious hard in BizTalk Server as a Visual Studio build will not provide a deployable artifact (only raw dlls). The only way to get a deployable MSI package for BizTalk Server is to first install everything into the server and then export – until now.
Why Continuous Integration?
Continuous Integration is a concept first described by Martin Fowler back in 2006. At its core its about team communication and fast feedback but also often leads to better quality software and more efficient processes.
A CI process usually works something like the above picture.
A developer checks in code to the source control server.
The build server detects that a check in has occurred, gets all the new code and initiates a new build while also running all the relevant unit tests.
The result from the build and the tests are sent back to the team of developers and provides them with a up to date view of the “health” in the project.
If the build and all the test are successful the built and tested resources are written to a deploy area.
As one can see the CI build server acts as another developer on the team but always builds everything on a fresh machine and bases everything on what is actually checked in to source control – guaranteeing that nothing is build using artifacts that for some reasons is not in source control or that some special setting etc is required to achieve a successful build.
In step 4 above the CI server also writes everything to a deploy area. A golden rule for a CI workflow is to use artifacts and packages from this area for further deployment to test and production environments – and never directly build and move artifacts from developer machines!
As all resources from each successful build is stored safely and labeled one automatically achieves versioning and the possibility to roll back to previous versions and packages if needed.
What is the problem with CI and BizTalk?
It is important to have the build and feedback process as efficient as possible to enable frequent checkins and to catch possible errors and mistake directly. As mentioned it is equally as important that the resources are written to the deploy area are the ones used to deploy to test and production so one gets all the advantages with versioning and roll back possibilities etc.
The problem with BizTalk Server is however that only building a project in Visual Studio does not gives us a deployable package (only raw dlls)!
There are a number of different ways to get around this. One popular option is to automate the whole installation of the dlls generated in the build. This not only requires a whole lot of scripting and work, it also requires a full BizTalk Server installation on the build server. The automated process of installation also takes time and slows down the feedback loop back to development team. There are however great frameworks as for example the BizTalk Deployment Framework to help with this (this solution of course also enables integration testing using BizUnit and other framework).
Some people would also argue that the whole script package and the raw dlls could be moved onto test and production and viewed on as a deployment package. But MSI is a powerful packaging tool and BizTalk Server has a number of specialized features around MSI. As MSI also is so simple and flexible it usually the preferred solution by IT operations.
A final possibility is of course to directly add the resources one by one using BizTalk Server Administration console. In more complex solutions this however takes time and requires deeper knowledge into the solution as one manually has to know in what order the different resources should be added.
Another option in BtsMsiTask
Another option is then to use BtsMsiTask to directly generate a BizTalk Server MSI from the Visual Studio build and MsBuild.
The BtsMsiTask uses same approach and tools as the MSI export process implemented into BizTalk Server but extracts it into a MSBuild task that can be directly executed as part of the build process.
BtsMsiTask enables the CI server to generate a deployable MSI package directly from the Visual Studio based build without having to first install into BizTalk Server!
I’ve talked a lot about efficient documentation previously on this blog, both when in comes to splitting the documentation up in different layers as here, but also in relation to tools BizTalk Documenter and BizTalk Web Documenter for automatically generation technical documentation as here.
Last week I however had the pleasure to present on the BizTalk User Group Sweden meeting and got to talk all documentation for a whole hour! You can find the PPT from the meeting here.
I ended the presentation by presenting the ten commandments for efficient system integration documentation:
I. Thou shall not manually document anything that can be automatized!
II. Thou shall keep it simple & make it look nice
III. Thou shall use a wiki based platform
IV. Thou shall use pictures whenever appropriate
V. Thou shall have well defined guidelines for your documentation
VI. Thou shall have a well defined target audience for your documentation
VII. Thou shall document continuously in your project
VIII. Thou shall have a common vocabulary and common icons defined
IX.Thou shall test your documentation with target audience
X. Thou as the developer of an integration should document it
About a year ago I published a project that helps automating documentation for BizTalk Server – much like BizTalk Documenter but with the difference that this tools instead generates a dynamic web site. I’ve called in Config Explorer, a name I knew was sh*t from the start but the best I could come up with at the time.
After a while however, when enough people told me that they also thought the name was crap I decided to change it to BizTalk Web Documenter. I’ve also moved the whole project to GitHub.
I'll try and spend a bit more time on the project than I have the last year and will update the documentation, wiki and roadmap as well as implement a few new features.
I hope you’ll help me – fork the project today!
2012-09-30: I’ve renamed the whole project and moved it to GitHub. Hope to see you there!
As most BizTalk developers/architects you have probably been in a situation were documentation of your integration solution has been discussed. And you probably also been just as happy as me when you first found BizTalk Documenter. BizTalk Documenter is a great tool for extracting all that configuration data that is so vital to BizTalk and create a nice technical documentation of it.
There are of course several levels to documentation in BizTalk Server based solution were the technical information is at the lowest level. Above that you would then have you integration process descriptions and then possible another level above that where you fit the integration processes into bigger business processes, and so on – but let’s stick to the technical documentation for now.
BizTalk Documenter does a nice job of automating the creation of technical documentation, instead of basically having to all that configuration job a second time when documenting. Doing technical documentation by hand usually also breaks down after a while as solution setup and configuration changes a lot and the work is really time consuming. And as documentation that isn’t up to date frankly is completely worthless people soon looses interest in both reading and maintaining the technical documentation.
Using BizTalk Documenter however has a few problems …
The tool generates either a CHM file or a word file with documentation. As more and more organizations however move their documentation online new output formats are needed.
The latest 3.4 versions of the tools throws OutOfMemory Exceptions on bigger solutions.
Config Explorer Web Documenter
As we started trying to find ways to solve the issues above we had a few ideas we wanted the new solution to handle better.
Web based documentation
A web based documentation is both much easier to distribute and more accessible to read – sending a link and reading a web page rather then sending around a CHM file.
Easy way of see changes in versions
Sometimes one wants to see how things have changed and how things used to be configured. The tools should support “jumping back in time” and see previous versions.
Check out an an example of a Config Explorer yourself.
Config Explorer is a open source tool published on
CodePlex GitHub - a good place to start to understanding the workflow of creating documentation and to download the tool itself.
This release is a 1.0 release and there will are many features we haven’t had time for, but a few of them we’ve planned for future releases. Unfortunately there will however also be bugs and obvious things we’ve missed. Please use the
CodePlex GutHub Issue Tracker to let us know what you like us to add and what possible issues you’ve found.
I’ll of course also check for any comments here! Winking smile