Total Pageviews

Sunday, June 26, 2016

What is canonical Schema?Why Canonical Schema required?

CANONICAL SCHEMA

What is canonical Schema?
Sometime situation comes where a developer need to create an internal schema to provide lose coupling, to increase performance factor.  Such internal schema which is created at BizTalk level (e.g. none of the receiver or sender uses this schema, it is for purely internal purpose of BizTalk) is called canonical schema.

Why Canonical Schema required?
In order to provide lose coupling between different orchestrations, to increase performance, to get less rework if input schema format changes, Canonical Schema is required.

Let us say we are receiving purchase order from a trading partner and this purchase order (input message) has to be sent to 3 different partners after applying some business logic in transforms (map). So we have to create 3 maps one for each partners. In this case source and destination system is tightly coupled. At later point of time suppose some fields in input message gets changed, so we need to change all 3 maps which leads more rework.

Now if we create an internal schema (canonical schema) and map the input message to this canonical schema. This canonical schema can be published to message box and from there other artifacts can subscribe it. There we can create maps from 3 different partners. Doing so we achieved the lose coupling.e.g. Source and destination systems are not tightly coupled. At later point of time if suppose some fields in input message is changed then we need to do changes in only one map. I.e. from input message to canonical which leads less rework. Having less rework and lose coupling the performance is also gets increased. Thus canonical schemas are helpful and if possible we should try to develop the solution creating the canonical schema.


Saturday, June 25, 2016

What are the different types of BizTalk Schema

SCHEMA IN BIZTALK

What is BizTalk Schema?

BizTalk is an integration technology and it basically connects different partner. Now different partner can send the message in different format. E.g. one partner can send the message in JSON format and receiver can accept the flat file. However BizTalk understand only xml format and if partners are using another format than first it needs to be converted to xml format. 
                In order to understand the xml format by BizTalk, we need to define the structure of the xml message. E.g. what will be the root node name, what will be the elements name, its data type, attributes name , structure of the message (which element will fall under which node) etc.  Schema does this work for BizTalk. Schema defines the structure of the messages which are being received in BizTalk and sent to partner. BizTalk uses XML Schema Definitions (XSDs) to describe the xml message. There are 4 types of schema in BizTalk.

1)      XML Schema
2)      Flat file Schema
3)      Envelope Schema
4)      Property Schema

XML Schema:
 If BizTalk has to receive an xml message then the schema which defines the structure of this xml message is called as xml schema.

e.g. below is the xml messages





Corresponding xml schema in BizTalk looks like this.



We can see here the schema is defining the xml messages, name of the elements their data types.

There are several properties of the schema which needs to be set like targetnamespace etc.

Flat file Schema: 
If BizTalk has to receive or send flat files then schema generated to define the structure of the flat file is called Flat file schema.

e.g below is the  flat file which contains only first name and last name of student which is comma separated.



Corresponding Flat File Schema generated in BizTalk looks like this.



Annotation tag is being added in schema to define the flat file schema. Legacy system uses the flat file schema and now a day’s most of the system uses either xml or JSON or EDI format for business. The flat file specification should be thoroughly agreed between businesses partners before developing the schema, small change in flat file can lead to large rework from development prospective.

Envelope Schema:
If partner wants to send more than one message having same structure in one shot e.g. partner wants to send the multiple student details in one message then we call that message as Envelope. In BizTalk we can de batch the Envelope message and send one message at a time to BizTalk.This can be achieved by envelop schema.  In order to create the envelope schema we need to set some schema property. Envelope property needs to be set to true and in Body xpath expression, need to provide body path of the document which is repeating structure. Both Flat file and Xml Schema can be treated as envelop schema and we can set property once double click on the schema and see property.




Generally the repeating node is Record.


Property Schema:
To understand the property schema, first we should know what property promotion is. Suppose we are receiving and xml message having student percentage and based on percentage we want to route the message to different department. In such cases we can promote the age from the message. The Age coming from xml instance messages is coming as content of the message. In order to route the message based on age, this property should be present in context of the message. Moving the content of message to context is called the property promotion and when we do so the promoted element is added to one Schema which takes care to move the values from content to context. This schema is called property schema.







Apart from these Schemas there is one more type of schema which is called as EDI Schema. Actually it is same as Flat File Schema , but considering the complexity of the EDI messages ,BizTalk provides around 8000+ EDI schemas and we can use them instead of generating on own.


To Read about Canonical Schema click here.




Wednesday, June 1, 2016

Dynamic Header while consuming Rest Service in BizTalk

Scenario:
We have to consume a Rest Service (Or it can be WCF\Web service) in BizTalk with some headers, which will change per message. e.g. for each message the header values will be different. Lets Say we have to pass Username, Password and Signature in header. Signature is to be calculated as combination of UserName and GUID.
Solution:
Approach1: Using custom pipeline.
Simply we can create a Custom Pipeline and write the code but there is some challenge in that.
If we use custom pipeline, first message will go successfully, but from second message onwards, messages will not process.
It is because of the design of WCF-WebHttp adapter, it will not generate the headers per messages, and adapter uses the properties by port and not by messageSo we cannot use it for these purpose.

Approach2: Using Dynamic Send port.
This approach works fine, but could not try. You can go through below link to get it.
Also it is tightly coupled and for any minor changes, it  requires modification in Orchestration.

Approach3: Using wcf behavior extension.
This approach suits best for such scenario and it works 100% perfect.
Let’s first go to little description of the WCF Behavior, if you directly want to jump on code, Go to Code Section.
Implementating WCF Custom Behavior extension
Extension is something which you see in Behavior tab. So basically we have to create custom behavior and we bind it with extension.
Creating custom behavior
To create custom behavior we have to implement some interfaces. Below are the 2 interface required to implement.
  1. IEndpointBehavior
  2. IClientMessageInspector
Let’s go in little depth on how we can implement these interface.
  1. IEndpointBehavior
This interface implements the methods which we can use to extend the run-time behavior for and endpoint.
This interface contains four methods
  1. ApplyDispatchBehavior
  2. ApplyClientBehavior
  3. AddBindingParameters
  4. Validate
Let’s have little description of above 4 methods.
  1. ApplyDispatchBehavior
This method is used to create the object of context inspector class (which is implementing the interface IClientMEssageInspector) and bind that object to DispatchRuntime.MessageInspectors as shown in code below.

public void ApplyDispatchBehavior(ServiceEndpoint endpoint, System.ServiceModel.Dispatcher.EndpointDispatcher endpointDispatcher)
        {
            AddHttpHeaderContextInspector inspector = new                          AddHttpHeaderContextInspector();
            endpointDispatcher.DispatchRuntime.MessageInspectors.Add(inspector);
        }
  1. ApplyClientBehavior
This method is used to create object of the context inspector class (which is implementing the interface IClientMEssageInspector) and bind that object to clientRuntime.ClientMessageInspectors, as shown in code below:
public void ApplyClientBehavior(ServiceEndpoint endpoint, Client Runtime clientRuntime)
        {
            AddHttpHeaderContextInspector inspector = new AddHttpHeaderContextInspector(_interface Name);
            clientRuntime.ClientMessageInspectors.Add(inspector);
            return;
        }
  1. AddBindingParameters
Implementation of this method can be as below
public void AddBindingParameters(ServiceEndpoint endpoint, System.ServiceModel.Channels.BindingParameterCollection bindingParameters)
        {
            return;
        }


  1. Validate
Implementation of this method can be as below
public void Validate (ServiceEndpoint endpoint)
        {
            return;
        }

  1. IClientMessageInspector
This is the interface which implements the methods, which are used to inspect the message. Message can be inspected before sending the request to any service and after receiving response from the service Or message can be inspected after receiving request at server and before sending the response back to client.
By Inspection of message we mean that, we can tweak the message, add headers to message and many more things can be done.
This interface requires to implements the below four methods.

  1. AfterReceiveRequest
  2. BeforeSendReply
  3. AfterReceiveReply
  4. BeforeSendRequest
We can implement the methods as below
  1. AfterReceiveRequest

public object AfterReceiveRequest(ref System.ServiceModel.Channels.Message request, System.ServiceModel.IClientChannel channel, System.ServiceModel.InstanceContext instanceContext)
        {

        }

  1. BeforeSendReply

public void BeforeSendReply(ref System.ServiceModel.Channels.Message reply, object correlationState)
        {

        }
  1. AfterReceiveReply

public void AfterReceiveReply(ref Message reply, object correlationState)
        {
            string str = “”;
        }
  1. BeforeSendRequest

public object BeforeSendRequest(ref Message request, System.ServiceModel.IClientChannel channel)
        {
}

Apart from these we have to create one more class which inherits the  BehaviorExtensionElement  class to implement the CreateBehavior method.
So over all need to create 2 classes as below:
public class AddHttpHeaderContextBehaviorElement : BehaviorExtensionElement
public class AddHttpHeaderContextBehavior : IEndpointBehavior
public class AddHttpHeaderContextInspector  : IClientMessageInspector

Code
Let’s come back to our topic “How to add dynamic header to consume any service from BizTalk”
Let’s go  step by step:
  • Create a class library project and above mentioned 3 classes
  • Add below in namespaces for all 3 classes
System.ServiceModel.Description
System.ServiceModel.Channels
System.ServiceModel.Dispatcher
System.Runtime.Serialization
System.ServiceModel;
  • Add references for below dll
System.ServiceModel
System.Runtime.Serialization
  • Code for AddHttpHeaderContextBehaviorElement class
You can remove the unnecessary namespaces
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.ServiceModel.Configuration;
using System.Configuration;

namespace HttpHeaderExample.Component
{
public class AddHttpHeaderContextBehaviorElement : BehaviorExtensionElement
    {
        public override Type BehaviorType
        {
            get {return typeof(AddHttpHeaderContextBehavior); }
        }

        protected override object CreateBehavior()
        {
            return new AddHttpHeaderContextBehavior(INTERFACENAME);
        }
// here we are creating the property of the extension which will appear in admin console. As we can use same extension to apply on different send ports. Now we can send different type of messages to different send port so to distinguish them in component we created this property. See picture below where we can see it in admin console. We can use biztalk message promoted properties as well inside the inspector.        

        [ConfigurationProperty(“interfaceName”)]
        Public string INTERFACENAME
        {
            get { return (string) base[“interfaceName”]; }
            set { base[“interfaceName”] = value; }
        }
}
}
//End of class






  • Code for AddHttpHeaderContextBehavior class

You can remove the unnecessary namespaces

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.ServiceModel.Dispatcher;
using System.ServiceModel.Description;

namespace HttpHeaderExample.Component
{
public class AddHttpHeaderContextBehavior : IEndpointBehavior
{

public string _interfaceName;

public AddHttpHeaderContextBehavior(string interfaceName)
{
_interfaceName = interfaceName;
}

public void ApplyDispatchBehavior(ServiceEndpoint endpoint,      System.ServiceModel.Dispatcher.EndpointDispatcher endpointDispatcher)
{ //leave below commented
//AddHttpHeaderContextInspector inspector = new AddHttpHeaderContextInspector();
//endpointDispatcher.DispatchRuntime.MessageInspectors.Add(inspector);
}


public void AddBindingParameters(ServiceEndpoint endpoint, System.ServiceModel.Channels.BindingParameterCollection bindingParameters)
{
return;
}

public void ApplyClientBehavior(ServiceEndpoint endpoint, ClientRuntime clientRuntime)
{
AddHttpHeaderContextInspector inspector = new AddHttpHeaderContextInspector(_interfaceName);
clientRuntime.ClientMessageInspectors.Add(inspector);
return;
}

public void Validate(ServiceEndpoint endpoint)
{
return;
}
}
}


  • Code for AddHttpHeaderContextInspector class

You can remove the unnecessary namespaces

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.ServiceModel.Description;
using System.ServiceModel.Channels;
using System.ServiceModel.Dispatcher;
using System.Net;
using Newtonsoft.Json.Linq;
using System.Xml;
using System.Xml.Linq;
using System.IO;
using System.Runtime.Serialization;
using System.ServiceModel;
using System.ServiceModel.Web;
using Newtonsoft.Json;


namespace HttpHeaderExample.Component
{
public class AddHttpHeaderContextInspector : IClientMessageInspector
{

public string _httpMethod;
public string _uri;
public string _basicAuthUserName;
public string _basicAuthPassword;
public string _interfaceName;
public string content;
public string signatureKey;
public string operationName;




public AddHttpHeaderContextInspector(string interfaceName)
{
_interfaceName = interfaceName;
}


// Below two methods not reuired to implement for this example
// you can  use these if you host some service on server and want manipulation in //the message after receive request and before send reply back to client.


//public object AfterReceiveRequest(ref System.ServiceModel.Channels.Message request, System.ServiceModel.IClientChannel channel, System.ServiceModel.InstanceContext instanceContext)
//{

//}

//public void BeforeSendReply(ref System.ServiceModel.Channels.Message reply, object correlationState)
//{
//
//}




public void AfterReceiveReply(ref Message reply, object correlationState)
{
// not mainupulating the message after receiving reply from zycus
string str = “”;
}


public object BeforeSendRequest(ref Message request, System.ServiceModel.IClientChannel channel)
{
Message newMessage = null;
try
{
// Get the http request and method

HttpRequestMessageProperty httpRequest = null;

if (request.Properties.ContainsKey(HttpRequestMessageProperty.Name))
{
httpRequest = request.Properties[HttpRequestMessageProperty.Name] as HttpRequestMessageProperty;

}


// Fetch the URI and method , you can fetch biztalk message property here which is promoted.

_httpMethod = httpRequest.Method.ToLower();
_uri = request.Headers.To.AbsoluteUri;
// fetch the operation name


#region Extract the xml message

// Extract the message only for methods other than get
if (!(_httpMethod.Equals(“get”)))
{
// the data coming here is in Raw format so used below
// methods, even the message content is json but in this method it //was received as raw message. At the time of debugging you can see which type of data is //coming and accordingly you can use ReadContent method.

MessageBuffer msgbuf = request.CreateBufferedCopy(int.MaxValue);
Message tmpMessage = msgbuf.CreateMessage();
XmlDictionaryReader xdr = tmpMessage.GetReaderAtBodyContents();

xdr.ReadStartElement(“Binary”);
byte[] bodyBytes = xdr.ReadContentAsBase64();
string messageBody = Encoding.UTF8.GetString(bodyBytes);

// Here we are calling the Rest service which accepts the json //message so receving json string here.if you call normal wcf service then it will be xml //message.
// copy json message body to content

content = messageBody;

// we can tweak here the message content
Content = content +”Add some string if we want to add”
// Using newton soft json library we can read the json object and //mainupulate that. Also we can use here interfaceName property to distinguished the //different messages and tweak them. Lets say there are two different services and we //have to use same component for them. But  the message that should go have different //string to be added , so at admin console configuration provide different name for //property interfaceName. That property value will be available here and we can tweak //accordinglty.
//e.g if (interfaceName == “A”)
//content = content+A;
//else
//content = content+B;

// Now to recreate the message
// after mainupulating the message it needs to be send back to //stream

bodyBytes = Encoding.UTF8.GetBytes(content);
MemoryStream ms = new MemoryStream();
XmlDictionaryWriter writer = XmlDictionaryWriter.CreateBinaryWriter(ms);
writer.WriteStartElement(“Binary”);
writer.WriteBase64(bodyBytes, 0, bodyBytes.Length);
writer.WriteEndElement();
writer.Flush();
ms.Position = 0;
XmlDictionaryReader reader = XmlDictionaryReader.CreateBinaryReader(ms, XmlDictionaryReaderQuotas.Max);
newMessage = Message.CreateMessage(reader, int.MaxValue, request.Version);
newMessage.Properties.CopyProperties(request.Properties);
newMessage.Headers.CopyHeadersFrom(request);
request = newMessage;

msgbuf.Close();
}
//Message manipulation is completed here

#endregion

// Now suppose we have to pass the user name password and signature in header.
// and signature has to create here using username+new guid;
//So User name and password directly can be taken from sso config or bre or as per //requirement
//Signature can be calculated here
#region Fetch UserName, Password,

_basicAuthUserName = “Some User Name”
_basicAuthPassword = “Some password”

signatureKey =  _basicAuthUserName+”New guid”(it will change permessage);



#endregion




#region Add Headers to Message

WebHeaderCollection headers = httpRequest.Headers;
headers.Add(“username”, _basicAuthUserName);
headers.Add(“signature”, signatureKey);
headers.Add(HttpRequestHeader.ContentType, “application/json”);

#endregion

#endregion

}
catch (Exception ex)
{
throw ex;
}

return null;
}
}
}



  • Coding part complete here. Now build the project and add it go GAC.
  • In order this newly created behavior to be appear in BizTalk admin console, we need to add the component in machine.config file
  • You can find machine.config file at below location and this file need to be updated for both 32 bit and 64 bit folder.

C:\Windows\Microsoft.NET\Framework\v4.0.30319\Config – this is the location for 32bit machine.config
C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Config – this is the location for 64 bit machine.config

V4.0.30319 is the latest framework of the dot net installed in machine.


First take back up of machine.config files, it is very critical file and can down entire system.


Make sure that below entry is present in machine.config with siebleAdapter.

  • Once this completes, you can see the behavior in BizTalk admin console, WCF-WebHttpAdatper. Once restart the host instance and open fresh instance of admin console.
  • Go to send port à ConfigureàBehavioràRight Click on Endpoint behavioràAdd Extension à(here you will see the behavior you have created) click ok to add it.
  • Click on the behavior element, on right side window you see the property name interfaceName. You can give value here.

This way we can achieve sending dynamic headers to consume rest\wcf service in biztalk.