Message Transformations in BizTalk Server 2004

essage transformations have long been an integral part of EAI projects. Disparate systems participating in an integration project need transformations as dictated by their respective standards. As an EAI tool, BizTalk provides excellent support for message transformations.

BizTalk carries out transformations primarily with the BizTalk Mapper, which has evolved significantly from its first version in BizTalk Server 2000. A map in BizTalk can be created graphically by defining a set of relationships between two or more XML messages. Much like its predecessor, BizTalk 2004 Mapper allows users to define relationships by simply drawing a line between the two schemas. More complex transformations can be established using functoids, which also should be familiar to users who’ve used the Mapper.

This article demonstrates the ability of BizTalk Server to carry out message transformations with amazing ease. It walks through the processes of simple message transformations with the Mapper and extends them to some complex scenarios that are common in EAI projects. The intricacies of functoids also are explored, along with a demonstration of how to create a brand new functoid. Finally, the article looks at some lesser-known aspects, such as transformations with multiple-source and destination messages.

Creating the Map File

The message schema is a prerequisite for any BizTalk project. An XML schema must be defined for each message used in a BizTalk project. Creating a BizTalk map involves specifying the source and destination schemas that need to be mapped together.

Figure 1. Construct Message Scope: The transform shape must be within a Construct Message Shape. This takes care of the fact that messages are immutable by nature.

In a BizTalk project, a map can be created in two ways:

1. Within the orchestration ? Once you have added the schemas and created the messages, simply drop a transform shape on the orchestration flow. This will automatically add a Construct Message shape that will contain the transform shape (see Figure 1). Visual Studio adds the construct message shapes as messages, which are supposed to be immutable. Hence, to transform a message, you need to actually create a new one. The old message stays the same so the transformations can be tracked throughout the scope of the message.

To begin the process of mapping the values, just double click on the transform shape. On the popup dialog, select the “Create a new map” option and specify the messages that you want as input and output (see Figure 2). The input and output schemas for the map will be decided accordingly. Make sure the “When I click OK, launch the BizTalk Mapper” option is checked before you click OK. This will create a new map and open it up for you. You can work with this simple map by dragging fields from the source schema to the target schema.

Figure 2. Transform Configuration: BizTalk provides functionality to create a map with schemas derived from the messages being used. This functionality also helps in creating maps with multiple input or output messages.

2. Independently by adding a map file to the project ? To create a map independent of the orchestration, use the regular files add method as you would in any Visual Studio project. Once you’ve added the schemas to the project, simply right-click on the project and select Add New Items. In the popup dialog, select the BizTalk Map file (see Figure 3). This will create a new BizTalk Map file where you will need to specify the source and the target schemas.

Working with the Map File

The BizTalk Mapper provides an easy and efficient interface to interact with the maps. Primarily, the Mapper provides sections for the Source Schema, the Mapping Area, and the Destination Schema (see Figure 4).

Figure 3. Add a Blank Map: Add a blank map to the project using the “Add New Item” context menu on the project.

The Source Schema section allows you to configure the map’s source schema, which you use as the input to the map. The Destination Schema section allows you to configure the map’s destination schema, which the Mapper will use to generate the output.

The mapping Area is divided into a number of pages. Each page can contain its own functoids and links, which you normally wouldn’t use unless you had a really complex map that required multiple pages. The pages can also act as partitions in case you have multiple input or output schemas.

Figure 4. The Mapper: The BizTalk Mapper contains sections for Source schema(s) to map from, Destination schema(s) to map to, and the mapping area that does the actual processing. The Functoid toolbox provides access to the built-in, as well as the custom, functoids.

The functoid Toolbox is an integral part of BizTalk mapping. It is a library of functoids that can be used to perform various operations on the schemas. A functoid is basically a .NET assembly that takes inputs from the source (schema element or another functoid), processes them, and delivers the output either to the target schema or simply to another functoid. The functoids normally fall into the following categories:

  1. Mathematical
  2. Logical
  3. Date/Time
  4. Conversion
  5. Scientific
  6. Cumulative
  7. Database
  8. Advanced

In addition to these, users can add custom functoids in the Toolbox. Custom functoids normally are used when dealing with more complex conversions that cannot be accomplished easily with the normal set of functoids. An example could be something like encryption/decryption or Fahrenheit-to-Celsius conversions.

What If I Need to Lookup in the Database?

BizTalk users commonly need to extract values from the database to assist in transformations. A rich set of database functoids assists in doing just that. Consider an order-processing scenario wherein you generate an order directly to the supplier from an order request that you obtained from the requisition system. You would need to get all the supplier details, including the name, address, and contact information, from the database based on the supplier ID in the requisition system.

The value extractor functoid, in conjunction with the database lookup functoid, would help you achieve this. Both are in the database functoids category. The database lookup functoid actually connects to the database and queries using the lookup value. The lookup value is generally determined by the input parameter from the source schema, whereas the connection string can be stored in the BizTalk configuration file (btsntsvc.exe.config) and retrieved using advanced functoids. This allows the flexibility to change the data source without recompiling the orchestration.

The database lookup functoid requires four parameters in a specific order:

  1. The lookup value, which is normally taken from the source schema
  2. The database connection string with which the functoid will connect to the database
  3. The parameter that specifies the table / view name
  4. The parameter that determines the column name where the functoid will make the lookup for the value

Once connected, the database lookup functoid returns a recordset that holds the records matching the specified criteria (see Figure 5). A value extractor functoid must be used in conjunction with this to retrieve the values from the recordset. The value extractor functoid takes in two parameters: the first is the link from the database lookup functoid and the next is the column name from where the value is to be extracted. The value extractor returns the value for the specific column matching the criteria specified in the database lookup functoid.

Figure 5. Database Functoids: Use the Database Lookup in conjunction with the Value Extractor functoid to extract values from the database.

These Functoids Are No Help. I Need My Own.

Despite the flexibility and power provided by built-in functoids, some business scenarios demand more. Consider a scenario where you need to convert temperatures from Celsius to Fahrenheit or vice-versa based on parameters from a source schema. Doing this with the normal set of functoids would involve using advanced functoids to determine the type of conversion and then a series of mathematical functoids for the actual conversion process. This scenario provides a nice example in which to build a custom functoid.

A custom functoid builds on a .NET assembly, which inherits features of a functoid from the Microsoft.Biztalk.BaseFunctoids.BaseFunctoid class. For a functoid, you can just get down to the code without many preliminary steps (they just don’t have much!). A custom functoid starts with a Class library project in the Visual studio environment. This project needs to reference the Microsoft.Biztalk.BaseFunctoids.dll file located in the BizTalk developer tools folder under the BizTalk installation folder.

The Class library compiles in an assembly that you need to register in the GAC. So be sure to generate the snk file using the “sn” utility and refer it in your AssemblyInfo.cs. Furthermore, you can have a resource file that defines attractive icons for your new functoid. So with all that done, you are ready to code the example:

using System;using Microsoft.BizTalk.BaseFunctoids;using System.Reflection;namespace MyFunctoids{	public class ConvertTemperatureFunctoid: BaseFunctoid	{		public ConvertTemperatureFunctoid()		{			//Functoid Unique Id. Start from 6000 for your own functoids			this.ID = 6003; 					SetupResourceAssembly("MyFunctoids.Resources",_
Assembly.GetExecutingAssembly()); SetName("MY_FUNCTOID_NAME"); //Functoid Name from Resource SetTooltip("MY_FUNCTOID_TOOLTIP"); //Functoid Tooltip from Resource SetDescription("MY_FUNCTOID_DESCRIPTION"); //Functoid Description from Resource SetBitmap("MY_FUNCTOID_BITMAP"); //Sets the Bitmap from resources //Functoid will accept 2 parameters. //One for Base type - Fahrenheit or Celsius //The other for the actual value of temperature this.SetMinParams(2); //Sets the Minimum number of parameters this.SetMaxParams(2); //Sets the Maximum number of parameters //Specify the function that will be called when the functoid executes SetExternalFunctionName(GetType().Assembly.FullName,_
"MyFunctoids.ConvertTemperatureFunctoid", "ConvertTemperature"); this.Category = FunctoidCategory.String; this.OutputConnectionType = ConnectionType.AllExceptRecord; // Parameter 1 AddInputConnectionType(ConnectionType.AllExceptRecord); // Parameter 2 AddInputConnectionType(ConnectionType.AllExceptRecord); } public string ConvertTemperature(string val1, string val2) { string result = ""; //Here goes all the logic required for the conversion return result; } }}

In the above sample, note several things that differentiate the assembly as a functoid:

  1. The class is inherited from the BaseFunctoid class, which provides the required functionality for building over a functoid. Functionality and properties specific to your functoid need to be coded within the constructor.
  2. Within the constructor, the first thing you do is set the ID of the functoid. This ID must be unique to the functoid. Normally, you should start with an ID value of more than 6,000 so that the IDs do not conflict with those of the basic functoids.
  3. After setting the ID, you can set further properties of the functoid, including the name, tooltip text, description, and even the icon that is displayed in the toolbox.
  4. The setMaxParams() and setMinParams() methods allow you to define the maximum and minimum parameters for a functoid. These values will be used for the restrictions while compiling a map.
  5. Next to the parameters is the setting of the parameter types. This basically allows you to specify the type of values that each parameter will expect. In this case, you can allow everything (except the records) as input as well as output parameters because the functoid expects a single value and not a record, which contains multiple nodes.
  6. The next thing to do is set up the functoid category that determines where the functoid will fit in the toolbox.
  7. That leaves only the most important part: specifying the method that will do the actual job for the functoid. This is specified by the SetExternalFunctionName method. The method expects the complete path to the function, including the assembly name, the namespace and class, and the actual function name.

To use the functoid in the BizTalk Mapper, just GAC the compiled assembly and copy it to the Mapper extensions folder under the developer tools folder. This allows Visual Studio to “see” the assembly. After that, just add it to the toolbox by right clicking on toolbox and selecting add/remove items. With this, you should be able to see the functoid in the toolbox and use it just like any other.

In similar ways, a custom functoid can assist in various ways while doing the transformations. This can include Web service calls, logging functionality, database interactions, and several other things that would otherwise be quite complicated with the normal set of functoids.

Mapping Multiple Schemas

Most of the transformations in BizTalk are of a simple type wherein a source schema is transformed directly to the destination schema. Many times they use certain functoids to do minor processing during the transformations. However, sometimes you need to have multiple schemas converge into a single one or a single schema output two different messages or even multiple schemas generate multiple output messages.

Consider an example wherein you have a message that is an order request to the purchase department. As a company policy, every purchase has to be approved by the purchase manager before being sent to the supplier. In this case, after you get the purchase request, an approval message will be generated and sent to an approval system. The orchestration will wait for the approval and then correlate the approval response to the order request. Before raising it to the supplier, the orchestration will need to merge the two messages in order to send over the actual request to the supplier. This solution can be accomplished by using multiple transform shapes, but an easier way is using two different source messages.

Figure 6. Mapping Multiple Schemas: The Transform configuration toolbox allows you to define multiple input and output schemas, thus easing complexities in the design and development of the orchestrations.

BizTalk allows you to perform many-to-many message transformations by specifying multiple-source or destination schemas. However, this is possible only when the map is created within an orchestration by specifying more than one schema at the source or destination (see Figure 6). In this case, the Mapper shows the multiple schemas under one with different nodes, and the transformation can be performed as a regular one-to-one.

This helpful approach can be a good design consideration. It helps reduce the complexities?and hence the processing?during the execution of the orchestration.

To Sum Up…

You can cater to most of the message transformations required for any business scenario by using the flexible and powerful tools provided by BizTalk 2004. A rich set of built-in functoids allows you to handle most of the requirements with ease. Custom functoids allow you the flexibility to unleash the power of .NET assemblies within the Mapper tool. Last but not the least, the ability to support many-to-many transformations helps ease the complexities that the orchestration would otherwise require.

Happy BizTalking!

Share the Post:
Share on facebook
Share on twitter
Share on linkedin

Overview

Recent Articles: