Introduction

As I alluded to at the very end of 2011, I’ve been spending more and more time with BizTalk 2010. It’s been a little while since I’ve had to support/build/maintain BizTalk artefacts, but it doesn’t take long to reacquire “the groove”.

To gently thrust you into the dark underbelly of the land of BizTalk development, I thought I’d start off by exploring some fairly basic BizTalk 101 type experiences (and solutions!).  This article focuses on generic issues with XML schemas, with an eye to BizTalk’s usage of said schema.

Sadly, some of what you might read here (and later) might smack a tad of ‘What?  You didn’t know that?”, but I’m willing to swallow some pride if it helps out other Architects and Programmers.

What you need (to follow along)

This article is (IMHO) decent enough for the generalist (non-BizTalk specialist) and especially may be handy for those unfamiliar in working with XML schemas.

There is a big part where I’m playing with BizTalk’s Developer Tools (in Visual Studio 2010), so if you’d like to give it a go, I’d suggest you pick up a copy of BizTalk Server 2010 Developer Edition (click here for more info on this edition of BizTalk), which – I believe – you need to have to install the developer tools.

If I write subsequent article on BizTalk, the BizTalk developer tools will likely play a significant role, so it might be worth investigating.  As always, it’s worth bookmarking the BizTalk Development Center link.

The Joys of Schema

If you know anything about BizTalk, you’ll know that it’s all about the schemas. Everything ends up mapping to a document schema type whether it’s inbound or outbound. Schemas (XSD) can be an acquired taste, especially if you’ve been off doing something more exciting.

Last month, handwritten schemas tripped me up quite a bit. I was used to using Wizards and Adapters to generate a lot of the schemas for me, so when I went to design some basic input and output schemas, I encountered some “fun” issues.

Something funny happened…

Now, I was putting together some very basic sample schema to use as an example for demonstrating a really neat technique with maps (transforms).

Just in case someone else encounters this (or, like me, found it mildly amusing) I thought I’d document some cause and effect of why you might get some head scratching error messages when designing in Visual Studio 2010 with the BizTalk designer.

Given the following (flawed) schema, which I built within Visual Studio:

<?xml version="1.0" encoding="utf-16"?>

<xs:schema xmlns="http://Sandbox.InputSchema" xmlns:b="http://schemas.microsoft.com/BizTalk/2003" targetNamespace="http://Sandbox.InputSchema" xmlns:xs="http://www.w3.org/2001/XMLSchema">

  <xs:element name="Root">

    <xs:complexType>

      <xs:sequence>

        <xs:element minOccurs="2" maxOccurs="3" name="Customer">

          <xs:complexType>

            <xs:sequence>

              <xs:element name="Address">

                <xs:complexType>

                  <xs:attribute name="Street" type="xs:string" />

                  <xs:attribute name="State" type="xs:string" />

                  <xs:attribute name="Country" type="xs:string" />

                </xs:complexType>

              </xs:element>

            </xs:sequence>

            <xs:attribute name="ID" type="xs:integer" />

            <xs:attribute name="FirstName" type="xs:string" />

            <xs:attribute name="Surname" type="xs:string" />

          </xs:complexType>

        </xs:element>

      </xs:sequence>

    </xs:complexType>

  </xs:element>

</xs:schema>

Which, when loaded in the BizTalk Schema Designer looks like this:

Has an unfortunately obvious flaw if you consider I was expecting the schema to facilitate the following format of XML data:

<ns0:Root xmlns:ns0="http://Sandbox.InputSchema"> 

  <Customer ID="100" FirstName="FirstName_1" Surname="Surname_2">

    <Address Street="12 Jump Street" State="NSW" Country="Australia" />

  </Customer>

  <Customer ID="101" FirstName="FirstName_1" Surname="Surname_2">

    <Address Street="22 Fish Street" State="QLD" Country="Australia" />

  </Customer> 

</ns0:Root>

In other words, I was expecting to store multiple “Customer” elements in my XML document.

Debugging

When I loaded the document and tested it in Visual Studio 2010, I received the following, very weird error message: “The element ‘Root’ in namespace ‘http://Sandbox.InputSchema’ has invalid child element ‘Customer’.” – huh?

The problem, I thought, might be related to not expressly prefixing the namespace directive to the child element, so I made some edits.

This resulted in an even more ambiguous error message, funny even:

“The element ‘Root’ in namespace ‘http://Sandbox.InputSchema’ has invalid child element ‘Customer’ in namespace ‘http://Sandbox.InputSchema’. List of possible elements expected: ‘Customer’.”

or, in generic terms:

“The element ‘<element>’ in namespace ‘<namespace>’ has invalid child element ‘<child element>’ in namespace ‘<namespace>’. List of possible elements expected: ‘<child element>’.”

WTF?

You may have to squint to read the text…

At this point, I felt like I was going backwards. So I undid my changes and took a look at the schema definition – which, upon second inspection was clearly wrong.

The Solution

See if you can spot the difference?

My mistake was in not declaring a sequence after the root element. Thus rectified, my XML now matched the fixed schema, and I was able to continue on my merry way.

Filtering in a Transform/Map

This all brings us back to the original intention of my article – to highlight the use of functoids for filtering multiple nodes in a transform.

Given that we want to transform our input schema (from before) into another format, with filtered data, here’s the schema for the output:

The Mapping

We can construct a map which assigns the fields we want per the following example:

A basic map

Now, if we wanted to only take Customers who live in the state of New South Wales, we need to validate each element in the input XML recursively.

A Functoid for all occasions

The easiest means is to use a equality functoid, in this example I’ve used an equals functoid, as per below. You must still map the field directly to the output schema, but also to the functoid – which is, in turn, mapped to the logical root of the target (per the example above).

When the functoid returns a true value, the element is copied to the output. If the result is fale, the entire element (children etc) are skipped. Thus, from the following input XML:

<ns0:Root xmlns:ns0="http://Sandbox.InputSchema"> 

  <Customer ID="100" FirstName="FirstName_1" Surname="Surname_2">

    <Address Street="12 Jump Street" State="NSW" Country="Australia" />

  </Customer>

  <Customer ID="101" FirstName="FirstName_1" Surname="Surname_2">

    <Address Street="22 Fish Street" State="QLD" Country="Australia" />

  </Customer> 

</ns0:Root>

When run through this map (using trusty ‘Test Map’ functionality), will generate the following output, filtering out that pesky Queensland (QLD) customer:

<ns0:Root xmlns:ns0="http://Sandbox.OutputSchema">

  <Address Street="12 Jump Street" State="NSW" Country="Australia" CustomerID="100" />

</ns0:Root>

Helpful Support in Visual Studio 2010

Before you start despairing about schema development and BizTalk maps, there are a bunch of very helpful tools built into Visual Studio 2010 (and prior versions).

Generating Schema Instances

Given you have Schema files in your project, you can right click them and select “Generate Instance”. This will create an XML file using intelligent default values, according to the schema definition. These files can be modified and used to test out other artefacts who use this schema.

Validating Schema Instances

If you want to test if a particular XML file is valid, you can do the reverse – by clicking on the schema properties, you can set the “input XML” for a given schema. Then, when right clicking on the Schema in Solution Explorer, you can select “Validate Instance”. This will highlight any issues with your schema.

Testing a Map/Transform

Once you have some valid XML for your schema, you can then use it as test input data for various things including maps.

The following shows how to establish test data parameters for a map within Visual Studio 2010:

When you have configured a map to use a specified XML input, you can then right click on the map (in Solution Explorer) and select “Test Map”.

Other Options – Xml Schema/Data Type Utility

A handy note for future reference – if you have an XML structure in mind, but (like me) you are rusty with your XSD designing skillz, there’s another way forward which may prove to be better.

There is a tool which helps you work with XML and XSD document formats, and it ships by default with a number of Microsoft products and SDKs, it is called “Xml Schemas/DataTypes support utility” or XSD.exe.

Important Disclaimer: In nearly almost all circumstances, it is considered a better practice to produce or design the XML schema first. Generating a schema from XML data can be fraught with peril, especially if your XML sample doesn’t accurately reflect the full spectrum of supported formats.

For example, min/max occurrences, type lengths and other constraints, vague data types (where the type or precision is “guessed”), elements supported by the schema which are not required – and thus missing from your sample XML – to name but a few.

Where Do I Find This Tool?

Chances are that you have it already. I’ve located it installed in the following locations (check the (x86) folder as well if you are on a 64 bit machine):

Directory of C:\Program Files\Microsoft SDKs\Windows\v6.0A\bin

Directory of C:\Program Files\Microsoft SDKs\Windows\v7.0A\Bin

Directory of C:\Program Files\Microsoft SDKs\Windows\v7.0A\Bin\NETFX 4.0 Tools

Directory of C:\Program Files\Microsoft SDKs\Windows\v7.0A\Bin\x64

What does it do?

Keeping in mind that this utility was really meant for the days of evil DataSets, it still can be quite useful.

Using the following (poor) sample XML structure:

<?xml version=”1.0” encoding=”utf-8”?>
<Country Name="Australia">
     <State Name="NSW">
           <Capital Name="Sydney"/>
     </State>
     <State Name="QLD">
           <Capital Name="Brisbane"/>
     </State>
     <State Name="WA">
           <Capital Name="Perth"/>
     </State>
     <State Name="TAS">
           <Capital Name="Hobart"/>
     </State>
     <State Name="VIC">
           <Capital Name="Melbourne"/>
     </State>
</Country>

You can use the XSD.exe utility to generate an approximation of the schema, like so:

C:\Program Files (x86)\Microsoft SDKs\Windows\v7.0A\Bin>xsd Input.xml

Microsoft (R) Xml Schemas/DataTypes support utility

[Microsoft (R) .NET Framework, Version 2.0.50727.3038]

Copyright (C) Microsoft Corporation. All rights reserved.

Writing file ‘C:\Program Files (x86)\Microsoft SDKs\Windows\v7.0A\Bin\Input.xsd’.

Which would produce the following output:

<?xml version="1.0" encoding="utf-8"?>

<xs:schema id="NewDataSet" xmlns="" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:msdata="urn:schemas-microsoft-com:xml-msdata">

  <xs:element name="Country">

    <xs:complexType>

      <xs:sequence>

        <xs:element name="State" minOccurs="0" maxOccurs="unbounded">

          <xs:complexType>

            <xs:sequence>

              <xs:element name="Capital" minOccurs="0" maxOccurs="unbounded">

                <xs:complexType>

                  <xs:attribute name="Name" type="xs:string" />

                </xs:complexType>

              </xs:element>

            </xs:sequence>

            <xs:attribute name="Name" type="xs:string" />

          </xs:complexType>

        </xs:element>

      </xs:sequence>

      <xs:attribute name="Name" type="xs:string" />

    </xs:complexType>

  </xs:element>

  <xs:element name="NewDataSet" msdata:IsDataSet="true" msdata:UseCurrentLocale="true">

    <xs:complexType>

      <xs:choice minOccurs="0" maxOccurs="unbounded">

        <xs:element ref="Country" />

      </xs:choice>

    </xs:complexType>

  </xs:element>

</xs:schema>

Now, given I mentioned that bit about the DataSet support.. You’ll need to remove the element called “NewDataSet”. This is included because the utility assumes you want to generate a schema for use with DataSets.

Removing the element, renders a nice XSD in Visual Studio:

But wait.. There’s more

Given a valid XSD schema, you can now use the same utility (isn’t it wonderful?) to generate valid C# classes by using the following syntax:

C:\Program Files (x86)\Microsoft SDKs\Windows\v7.0A\Bin>xsd Input.xsd /c

Microsoft (R) Xml Schemas/DataTypes support utility

[Microsoft (R) .NET Framework, Version 2.0.50727.3038]

Copyright (C) Microsoft Corporation. All rights reserved.

Writing file ‘C:\Program Files (x86)\Microsoft SDKs\Windows\v7.0A\Bin\Input.cs’.

The result of executing this command produces the following classes (which you could now serialize into and out of XML):

Summary

You may not be able to teach a dog new tricks, but I’m willing to bet he remembers a bunch of old ones!

Although I’m a bit rusty, I’m able to overcome obstacles reasonably quickly. This article might not make me look like Einstein, but I thought I’d write up some of my experiences in the hope it helps others.

We’ve only just touched the tip of the iceberg with BizTalk 2010. The development environment is challenging and prickly (which makes for great sources of inspiration for blog articles) so you can be sure there will be more here soon, so check back!