MSIT: Real World Migration Story from Gentran 5.1 to BizTalk 2010

Author: Amit Kumar Dua, Microsoft | Banupriya Thirumaran, Microsoft

Reviewed By: Mandi Ohlinger, Microsoft | Nitin Mehrotra, Microsoft

Contributors: Nikhil Tayal, Microsoft | Anil Chandra Lingam, Microsoft

Published: October 2014

Applies to: BizTalk Server, Gentran Server

Summary

The Microsoft IT (MSIT) integration platform uses BizTalk Server and Gentran. Until recently, Microsoft IT had a good presence of Gentran footprints. With BizTalk Server supporting cloud computing and AS2/EDI capabilities, there is an opportunity to replace Gentran with Microsoft BizTalk Server 2010. Gentran supports Windows Server 2003 and SQL server 2005; which are in extended support that is soon ending. There is no roadmap for Gentran to support newer platforms, including Windows Server 2008/2012 and SQL Server 2008/2014.

MSIT viewed this as an opportunity to:

Get to the newer technology platforms Remove the supportability constraints Simplify the overall architecture of the integration platform Make the integration platform more cost effective and robust

BizTalk Server 2010 is a proven integration platform, has numerous capabilities and features, and supports the new platform technologies.

This white paper discusses the strategy and steps taken to migrate from Gentran 5.1 to BizTalk Server.

2

Copyright

This document is provided “as-is”. Information and views expressed in this document, including URL and other Internet Web site references, may change without notice. You bear the risk of using it.

Some examples depicted herein are provided for illustration only and are fictitious. No real association or connection is intended or should be inferred.

This document does not provide you with any legal rights to any intellectual property in any Microsoft product. You may copy and use this document for your internal, reference purposes.

© 2013 Microsoft. All rights reserved.

3

Contents1 Introduction..................................................................................................................................................................5

2 Motivation.....................................................................................................................................................................5

2.1. Features of BizTalk Server 2010.............................................................................................................................6

3. Microsoft IT (MSIT) BizTalk 2010 Migration Story.........................................................................................................8

3.1. Problem Statement...............................................................................................................................................8

3.2. Objectives..............................................................................................................................................................8

3.3. Migration Strategy.................................................................................................................................................9

3.4. Development.......................................................................................................................................................10

Routing........................................................................................................................................................................10

Map Development.......................................................................................................................................................11

Partner Profile Configuration......................................................................................................................................11

3.5. Testing.................................................................................................................................................................12

6 Automation.................................................................................................................................................................15

6.1 Gentran Migration Tool.......................................................................................................................................15

6.2 Test Automation Framework (TAF).....................................................................................................................17

6.3 Data Segregation Tool.........................................................................................................................................18

7 Challenges faced during Migration..............................................................................................................................20

8 Best Practices..............................................................................................................................................................22

9 Appendix.....................................................................................................................................................................23

9.1 Glossary...............................................................................................................................................................23

10 Conclusion...............................................................................................................................................................23

11 Acknowledgements.................................................................................................................................................23

4

1 IntroductionSuccessfully delivering a solution for any Enterprise Product Migration as large as Gentran is always a challenging task. This white paper is an attempt to help the teams working on similar migrations. It also includes information on possible solutions and the challenges involved.

This paper addresses two critical questions that commonly occur when migrating from Gentran. These questions become even more critical when the existing solution has been running for years and supports billions of dollars of business. These questions include:

a. What are the motivations behind migrating from Gentran?b. How to plan, design, and implement a migration?

This paper covers MSIT’s Gentran migration approach, lists the primary reasons to migrate, and describes the solution.

2 MotivationDuring our previous migration to BizTalk Server 2010, we found that BizTalk Server 2010 is an excellent option for integrating small to medium scale businesses due to its remarkable features, implementation and maintenance costs, performance, security, and other aspects. When we compare BizTalk Server 2010 to Gentran, here are some of the gains we expect:

BizTalk Server Features Benefits

a. AS2 support is not available in Gentran 5.1; BizTalk includes this capability.b. Platform support to host Gentran 5.1 on Windows Server 2003 and SQL server 2005 is ending. Migration in such

a case, especially for companies heavily dependent on Microsoft technology stack, can leverage BizTalk because it has great Azure (cloud) support and a smooth, successful migration story.

c. BizTalk Server supports the newer Microsoft platform technologies, including Windows Server 2008 R2/2012, SQL Server 2008 R2/2014 and the .NET Framework 4/4.5.

d. BizTalk Server includes support for Unicode character set.e. Performance benefits of BizTalk over Gentran.

5

2.1. Features of BizTalk Server 2010Easier Integration of Enterprise Applications

BizTalk accelerates time-to-solution with better and more accessible tools:

BizTalk includes easy to use tools that integrate Line-of-Business applications with Windows Server AppFabric Workflows. This includes Workflow activities that provide access to the BizTalk transformation engine and mapper, and the BizTalk Line-of-Business adapters.

The BizTalk mapper is updated with a new user interface that improves development of large transformations, and provides new search and predictive matching functionality. The new mapper also improves productivity by adding cut, copy, paste, move, and undo functions. It also includes stronger support for documenting maps and readability.

BizTalk RFID includes new event handlers and event delivery capabilities. The new event handlers enable removal of duplicate events and filters based on Electronic Product Codes (EPC) and visibility requirements. The new delivery capabilities send events directly to .NET applications, BizTalk Server, and EPC Information Services (EPCIS).

Excellent support for industry-standard web services. The BizTalk SOAP adapter enables sending and receiving messages by using SOAP over HTTP, thus enabling it to interact with the universe of web services. BizTalk also includes seven adapters and easy-to-use “wizards” that enable communicate to and from BizTalk Server and Web services-based applications using Windows Communication Foundation (WCF).

Platform Support for Latest Microsoft Technologies

BizTalk supports the newer Microsoft platform technologies, including Windows Server 2008/2012, SQL Server 2008/2014, Visual Studio 2010/2013, and the .NET Framework 4/4.5:

BizTalk support for Windows Server 2008 R2 offers modular, minimal installation, improved network performance and control. It also includes improved high availability features, enhanced administration, and management with Server Manager, Windows PowerShell command-line, and enhanced virtualization with Hyper-V. By utilizing Windows Server 2008 R2 clustering, BizTalk Server can be deployed in multisite cluster scenarios, where cluster nodes can reside on separate IP subnets and avoid complicated VLANs.

BizTalk takes advantage of the virtualization improvements included in Windows Server 2008 R2 Hyper-V. These improvements lead to reduced costs through lower hardware, energy, and management overhead; plus the creation of a more dynamic IT infrastructure.

BizTalk Server support for SQL Server 2008 and SQL Server 2008 R2 offers better manageability and scalability, an optimized virtual SQL Server implementation, and improved performance, especially in a 64-bit environment.

Support for Visual Studio and NET Framework (v4) introduces a number of improvements to the underlying Visual Studio-based BizTalk project system. Improvements include debugging support for artifacts and BizTalk maps (XSLT), enhanced BizTalk artifact property pages integrated into Project Designer tabs, the Visual Studio Project Update Wizard, and support for release and debug builds.

6

Simplify Solution Manageability for IT Pros

A new Settings Dashboard consolidates all settings into a single place; providing granular settings at the group, host, or host instance level. The Settings Dashboard supports automated settings deployment with scriptable export/import operations.

The new System Center Operation Manager Pack includes a new model with separate application and deployment views, new alerts and diagnostics, and improved multi-machine monitoring and cluster awareness.

Enhanced Enterprise Interoperability

Enhanced Trading Partner Management provides easy on-boarding and lifecycle management of trading partners. A new partner management model reflects typical business-to-business (B2B) relationships and is easier to manage large-scale TPM deployments. An improved user interface includes agreement templates and business profiles for rapid configuration and a new TPM operator role that provides access to users other than administrators.

New and enhanced adapters provide higher levels of security, better performance, and support for the latest releases of line-of-business applications.

7

3. Microsoft IT (MSIT) BizTalk 2010 Migration Story

3.1. Problem Statement

The Microsoft IT Integration team provides integration services to enable business integration within Microsoft and with external partners. MSIT uses “Gentran”; which is a 3rd party EDI and flat file translation engine that competes directly with BizTalk Server and Microsoft Azure BizTalk Services. Gentran version 5.1 is running on Windows Server 2003 and SQL Server 2005; which Microsoft support is ending around 2015. To ensure business continuity, it is mandatory to migrate all maps from Gentran before 2015.

3.2. Objectives

The main objectives of the migration from Gentran to BizTalk Server are:a) Retire message streams from the Gentran system.b) Simplify and consolidate partner flows on a single BizTalk Server 2010 platform.c) Reduce the cost involved in supporting the Gentran infrastructure.d) Upgrade the integration solution to the new Microsoft technology stack.

8

3.3. Migration StrategyThe migration is a strategic initiative that involves migrating an existing platform that supports billions of dollars of business. The following key objectives are identified as highly critical:

a. Least impact to the ongoing business during the migration.b. Least cost to be incurred for the migration.c. Least time to be taken for the migration.d. Great supportability of the solution in long term.

With these given objectives, there is lot of emphasis on the following:

a. Phased delivery modelb. Agile rollback processc. Swift and cost-effective development and testing process

Phased Delivery Model

There is a large scale of transactions running through the Microsoft integration infrastructure; Gentran is one of the critical components. It is very important to have a sound migration strategy. Microsoft IT followed a simple approach of ‘divide and conquer’. There are nearly 1500+ streams running through the Gentran platform; all need migrated to BizTalk Server. These streams are categorized by business processes and migrated in different phases. Each business process is also prioritized so that the less-simple and less-critical business processes are migrated before more critical and more complex business process. This strategy helps mature the migration process gradually without impacting the critical business processes.

Agile Rollback Process

A sound and agile rollback process is required in case of any failures during the migration. Strategically, there should be the least adverse impact (if any) on the business once transactions migrate from Gentran to BizTalk. If a situation arises, it should be mitigated quickly. The Development section in this whitepaper describes the rollback process.

Swift and Cost Effective Development and Testing Process

In the MSIT integration solution, there are about 1500+ streams running with around 300 maps. There is a high cost involved if these maps and streams are migrated and developed manually. As a result, the team designed and developed some productivity tools that enable map development and EDI partner configuration on BizTalk Server very quickly and in consequence, reduce costs.

Also, a sound testing process is required to meet the quality bar. To achieve the ‘least cost’ goal, productivity tools are developed with an automation framework that reduces the time and effort dedicated to testing.

9

3.4. DevelopmentWhen moving to a new non-Gentran platform, you need to rewrite the maps, and reconfigure the partner profiles and their transactions. We need a seamless way to migrate the workflows presently going through the Gentran platform. The MSIT development team has three focus areas:

a. Routing of streamsb. Map development c. Partner profile configuration

RoutingIn the integration scenario, there are thousands of message streams going through a gateway and then processed through different transformations on a translator. These streams need routed to the new translator/platform. You can specify routing on the gateway or after the messages have passed through the gateway.

A phase-wise delivery model and a sound rollback process are two requirements. With these requirements, it’s very important to have a sophisticated mechanism in place to easily redirect traffic to the old and new translators. To implement this kind of mechanism, we need a streams-routing component. This component parses and redirects the message based on a set of redirection rules.

In this migration story, there are several EDI streams being received on the gateway that were going to Gentran. To migrate these streams to BizTalk, the routing component is introduced. The routing component uses an EDI parser that parses the EDI stream and filters the Partner Key, Transaction, and Transaction Version details. There are redirection rules defined based on the same information and using those rules, streams are routed either to Gentran or to BizTalk.

The following image explains the routing:

10

As seen in the image, the traffic is initiated from EDI channel partners. Messages go through the Integration Gateway and then go through Gentran flows or BizTalk flows. The routing component routes the traffic based on a set of routing rules. Every rule typically defines that Partner X with Transaction Y and Version Z routes to a particular location. The location has a Gentran or BizTalk listener that pulls their messages respectively.

The routing component essentially parses the EDI message coming from the partner and determines the Partner Key, Transaction, and Transaction Version from the message. This information is further matched against the rule definitions. Then the messages are dropped to the location defined in the matching rule definition.

Map DevelopmentMap development is generally one of the biggest activities in any migration initiative in an integration scenario. MSIT has around 300 Gentran maps to redevelop. These maps vary in complexity. The team expected a huge amount of effort in developing those maps. As a result, the team created the Gentran Migration Tool to automate this process and reduce the effort involved.

The Gentran Migration Tool takes the Gentran map text file as an input and produces BizTalk schemas and map artifacts. The Automation section in this whitepaper describes this tool.

Partner Profile ConfigurationPartner profile configuration is one of the significant development activities done when EDI streams are on-boarded to a platform. The scope of this migration includes about 1500 streams to be migrated. The team added another feature to the Gentran Migration Tool. The new feature takes the Partner Configuration information from Gentran and creates partner profiles and agreements into Trading Partner Management in BizTalk Server 2010. The Automation section in this whitepaper describes this tool.

11

3.5. Testing

Test Approach:For the migration, over a year of production Gentran archive data is collected. The testing scenarios covered new changes like map, pipelines, regression of the as-is functionality, data comparison, negative testing, end-to-end testing with partners, and deployment verification.

All the active transactions are tested for all the partners. For end-to-end testing, the business partners and external customers are selected based on production volume, encoding, location, their availability, and so on.

PlanningTest Environments & Licenses

o Creating the test environments is a sizeable effort. Gentran servers are required immediately from the data segregation phase until SIT, and UAT.

o Gentran test licenses are used for setting up production Gentran environment replicas.

Non Functional requirements

o Capacity of the BizTalk 2010 infrastructure was previously evaluated as part of the Migration from BizTalk 2006 / R2 to BizTalk 2010 (See Whitepaper on Real world migration from BizTalk 2006 / R2 to BizTalk 2010). As a result, there is no explicit performance testing done to evaluate the scalability and throughput for the migrated Gentran transactions.

o For guidance on the BizTalk 2010 performance testing, refer to Performance section in the BizTalk Whitepaper Gallery.

Resources

o The test resources and timelines are planned ahead of the project’s baseline. Setting clear milestones ahead of the projects helped in getting an insight into the lead time and getting the right resources.

Test Data Generationo Data is collected from production archives of Gentran servers and internal File Movement servers. The

data is received in bulk and is categorized based on Gentran maps and partners. o For transactions coming from partners (which are usually of EDI formats, including X12 and EDIFACT), a

highly efficient tool based on EDI Parsing (X12/EDIFACT) segregates and validates the test data. For transactions coming from internal Line of Business applications (which vary from custom flat-file formats to XML), we use Gentran’s capability to parse and associate the message to a particular map.

Early data segregation helped identify and eliminate existing Gentran issues related to invalid data, removing inactive partners from the system, and test-driven development. For HBI (High Business Impact) data, sensitive information was scrubbed before being used for testing.

12

Test CoverageTest cases are created for multiple phases of testing, including: o BVT, IVT, and EVT test cases covering the binding file checks, configuration files, and assembly

verificationso Functional positive and negative test cases include adapter, map logic, and pipelineso Bulk data comparison per streamo End-to-end test cases per streamo Rollback and deployment testing

Test Phases

Testing Phase Areas Covered CriteriaIsolated Environment Testing (ITE)

Covers testing the Internal logic of the BizTalk 2010 application code, including the newly migrated Gentran map, pipelines, adapters, ports, parties, and agreements.

Automated ITE tests are run through Continuous Integration builds.

BizTalk output of each transaction must match Gentran’s output.

Functional Acks created must match Gentran’s Acks.

All the active partners should be validated for any transaction.

BAM tracking and promoted properties are validated.

Volume Testing Compared bulk data between Gentran and BizTalk.

Internal tools are used for the bulk comparison; which provide a visual report of the test results, and highlights errors and output discrepancies.

At least 30% of one year’s data was tested when the data availability was more than 20,000 messages.

For < 20000 messages, all the data was used for volume testing.

System Integration Testing (SIT)

BizTalk 2010 is integrated with upstream and downstream systems.

Partners are simulated (if required) using HTTP posts and local AS2 sites.

Connectivity tests with internal partner systems

All the active transactions are validated end-to-end.

Minimum of 3 files per partnerUser Acceptance Test (UAT)

Testing the solution with key external partners selected based on availability, high impact, and volume.

Collaboration with external customers starts early to ensure their test environments are available.

End-to-end test for transactions; Transport and Functional acknowledgements with key external partners

If the environments are new, due to network or firewall issues, connecting to external partner’s UAT URLs are validated.

Deployment & Rollback Testing

MSIT BizTalk infrastructure supports around 14 million transactions per month. So deploying any artifacts in such an environment needs a clean rollback back to Gentran environment, in case of any issues.

Phase wise rollback and deployment testing to simulate production phased deployment.

Internal business systems, such as SAP, are also involved as ports like WCF-custom are impacted.

At least one transaction is tested per-stream during rollback and deployment.

13

Pre-Production Validation

Deployment scripts, assemblies, and artifacts are validated and smoke tested before the Go-Live event.

Checklist is prepared with all the validations required post-production deployment.

Post Production SupportUsing System Center Operations Manager (SCOM) 2010, the following events are monitored:

Receive location and send port availability BizTalk-to-BizTalk server services Suspended queue monitoring Database availability Any disconnection between the database and processing nodes

Monitoring is automated. Alerts are sent for failed messages and SLAs are created based on the application criticality. The 24/7 support team from Operations and Engineering ensures that any critical issue is resolved based on the SLA.

The post-production support for applications is extended for a month from the Go-Live date. Any suspended messages in BizTalk or any issues raised by customers are attended to immediately with the 24/7 support model. There are no major issues during the post-production support period.

14

6 Automation

6.1 Gentran Migration ToolThe MSIT integration team created this tool to automate the process of developing BizTalk artifacts and partner configuration. The tool focuses on two development activities:

a. BizTalk map and schemas developmentb. Partner profiles migration

BizTalk Map and Schemas Development

Map and schema development for BizTalk is done using the Gentran TXT file. This file is generated from the Gentran map file using the Gentran Client Application. The Gentran Migration Tool takes the Gentran TXT file as input and makes BizTalk schemas and map as output. The automation varies from 40% to 60% with respect to the complexity of the schemas and map. Using this tool, a map with average complexity can be developed in 16 to 24 hours with little manual work. When the tool creates the new map, it organizes the functoids in multiple pages for easier readability.

There is some manual work needed, including removing the compilation errors and checking if mapping is not done. When the tool generates the new map, an extensive report logs all areas where the conversion is ambiguous or not possible; making the manual steps easier.

All schema types (EDIFACT, X12, Flat File, IDOC, and so on) can be converted to BizTalk schemas using the tool. It sets all the properties of various elements of the schemas and maps based on the Gentran TXT file.

The following lists summarizes the features this tool provides:

1. Generates BizTalk schemas and maps.2. Creates a report on the number of errors during map generation.3. Provides flexibility to standardize the generated schemas with one of the standard BizTalk available

schemas.

15

Map and Schemas Development Feature

Partner Profiles Migration

The tool takes the partner profiles and transaction metadata in the Gentran database and creates parties and agreements in BizTalk Server 2010. This automation completely eliminates the effort required to recreate party agreements. With thousands of streams, the tool recreates the parties and agreements in several hours.

Generally, the metadata required for EDI streams are Sender Qualifier, Sender ID, Receiver Qualifier, Receiver ID, Transaction, Transaction Version, Acknowledgement Settings, and Delimiters. All of this metadata is obtained from Gentran Metadata and Transactional tables and put into a table; which can later be used by the tool to create the TPM parties and agreements in BizTalk Server 2010.

Partner Profiles Migration Feature

16

6.2 Test Automation Framework (TAF)Map development is one of the key activities during Gentran Migration and is of paramount importance to make sure the downstream systems are receiving the messages as expected. The team must confirm that the new maps are tested comprehensively and the Gentran and BizTalk outputs match completely. It is important to use test files that cover all execution paths for the schema and map. For some maps, this test data may go up to thousands of files. The team looked for opportunities to help save time and effort.

The Test Automation Framework (TAF) was created to meet these requirements. This framework is designed and developed to provide automated comparison testing with a high volume of input test files.

Test Automation Framework Events (TAF Events)

Sequence of Events

As depicted in the previous figure, the following steps list the sequence of events that happen when the Test Automation Framework automates the bulk file testing:

1. Data Segregation Tool – It is essentially a provider of test files. It provides test files for a specific stream to be tested through the TAF. The tool is described at Data Segregation Tool.

The flow depicted with 1 shows this tool sourcing the test file to a location.

2. Map X Test Files – It is a location of files collected for a specific stream, like a map for a specific partner. This is used for dropping to locations where Gentran and BizTalk servers are listening, for example Location1 and

17

Location2 in the diagram. This location is configured in the TAF; which is listened to by its components. The files located here are dropped to BizTalk or Gentran.

The flow depicted with 2 shows the files sourced to TAF.

3. File Listener Component – It is a TAF Component that moves the files from one location to another. This component uses a set of file movement rules defined in a configuration file. It typically performs the following file movements from one location to another:a. Move the input test files for a specific map to the locations where configured translators services are

listening for the files.b. Move the input test files to the test files archive for a specific map. This archive has a dual purpose:

Acts as a reference location for input files when a comparison report is looked at and when to determine the input file for a given output. The report provides a reference to this location for the input files.

Acts as a test data provider for the next iteration of a test run and becomes an alternative source of test data (other than the location where Data Segregation Tool sources the data).

c. Move the files from the output location of given translators to three output locations. These locations are required to compare the files; as done by the File Comparison Component. There are three output locations used by this component: Outputs coming from Location X, namely the output location of BizTalk Server Outputs coming from Location Y, namely the output location of Gentran Server Archive Location for the outputs from Location Y. This archive provides a flexibility to turn off Gentran

processing for the files that are already processed. This is important when you are testing a set of files repeatedly through BizTalk to check for bug fixes.

These flows are depicted with 3, 4, 5, 6, 7.1, 7.2.

4. File Comparison Component – It does the comparison between two different files. The files are typically the outputs coming from two different workflows. It reports the comparison results in the form of an excel worksheet that lists the test input file location, its corresponding outputs, and the difference in the outputs, if any exist.

This component does the following:

a. EDI File Outputs comparison – Compares the EDI format files including X12 and EDIFACT.b. Flat File Outputs comparison –Compares the flat files, such as IDOCs and other custom flat files format.c. Fields Masking – Masks the fields not to be compared.

6.3 Data Segregation ToolVolume Comparison Testing is one of the important activities for testing the new BizTalk maps. To test the maps, production data and manually-created Unit Test data is used.

Production data is generally archived in production and encrypted while stored. The Gentran integration architecture uses a common archive location for all the streams that pass through Gentran. A single location stores the data for all the maps. The data for every map needs separated. With thousands of files, manually going through each file is time consuming.

18

There is an opportunity to automatically separate all the data from one folder into a different folder for every map. Note that this tool was created due to constraints within the Microsoft integration platform design. If the data is archived separate for every partner and their transactions, then this tool may not be needed.

The Data Segregation Tool essentially has a parser component that refers to the metadata table for segregating the files. The metadata table stores the mapping between the map name and a set of key information (Partner Key, Transaction, and Transaction Version). In the following image, there is parser component parsing the incoming file (EDI/Flat) and determining the required key information from the file. Based on this, the map name is determined from the metadata table. After this, a folder is created (if it doesn’t already exist) by map name and copies the file from the archived file location to that folder:

19

7 Challenges faced during Migration One of the fundamental differences between BizTalk and Gentran is configuring EDI streams. Specifically:

a. The BizTalk TPM is designed so that the EDI agreement between two parties has a unique combination of Sender Qualifier, Sender ID, Receiver Qualifier, and Receiver ID. This limits EDIFACT transactions as we cannot define a single agreement with multiple transactions in the BizTalk TPM. This is not a limitation for X12 transactions.

b. The BizTalk TPM is designed so that the EDI agreement is defined with a common delimiter set for all the transactions configured in that agreement. This is a limitation if the transactions configured in the same agreement are using different delimiter sets.

c. While configuring Functional Acknowledgement (FA/997) in TPM, it cannot be done in isolation for the incoming transactions configured in the agreement. When set to “YES”, then it applies to all incoming transactions configured. This poses a problem for migration as not every transaction may need to be acknowledged on BizTalk. This behavior complicates the issue as BizTalk ends up sending unexpected FA/997 to the partner who originally sent the transactions.

d. While configuring the X12 transactions to be sent to the trading partner, if the ISA11 value is not same for those transactions, then those transactions cannot be configured in the same agreement. This causes a problem as BizTalk does not allow configuring those transactions separately. These transactions are using the same combination of Sender ID, Sender Qualifier, Receiver ID, and Receiver Qualifier.

There are multiple ways to deal with these constraints; while not impacting the upstream/downstream partner. To deal with these constraints, a custom pipeline component is used. Specifically:

Issue 1

An agreement needs to be defined between Partner A and Partner B for transaction ORDERS (D93A) and ORDERS (D97A). But, BizTalk TPM does not support defining these two transactions in the same agreement. So, we need two different agreements that cannot be defined with same Sender Qualifier, Sender ID, Receiver Qualifier, and Receiver ID combination.

Solution

Create the two separate agreements for ORDERS (D93A) and ORDERS (D97A). The BizTalk TPM does not allow two agreements with same Sender Qualifier, Sender ID, Receiver Qualifier, and Receiver ID combination. So, additional agreements are created. One agreement continues to use the original combination. Any additional agreements use an internal representation of Sender ID/Receive ID.

When there are multiple agreements between two partners, it is important to have the right agreement resolution for the transactions. Agreement resolution happens using the UNB header for EDIFACT. The UNB header undergoes changes for Sender ID/Receiver ID in case it needs to be resolved to the agreements having internal IDs. This is done using a pipeline component and a mapping table that defines an internal Sender ID/Receiver ID for the original Sender ID/Receiver ID. The pipeline component uses the mapping table to determine the internal ID based on the

20

transaction in the EDI document. This component parses the EDI document, determines the transaction, and determines Sender ID, Sender Qualifier, Receiver ID, Receiver Qualifier, and EDI Code (for X12). The change in IDs occurs before the “Disassemble” stage in the EDI receive pipeline.

Issue 2

An agreement needs to be defined between Partner A and Partner B for transaction 810 and 855 with a different set of delimiters for 810 and 855. The BizTalk TPM does not support defining these two transactions in the same agreement. This requires creating two separate agreements; which is not possible with same Sender Qualifier, Sender ID, Receiver Qualifier, and Receiver ID combination.

Solution

The same solution is used; which is to create separate outbound agreements for 810 and 855. The 810 agreement uses the original Sender ID and the 855 agreement uses the new Sender ID. While passing the message through the EDI send pipeline to produce the EDI message, a custom pipeline component replaces the new ID with the original ID. This occurs after the “Assemble” stage in the pipeline. The original ID is determined by the pipeline component using the metadata table based on the transaction in the message.

21

8 Best Practices This paper covers the learnings and practices the MSIT integration team followed during the Gentran migration, including:

a. Divide and conquer approachb. Leverage automation framework and toolsc. Reinforce known best practices of BizTalk Server 2010

Divide and Conquer Approach

This approach has proven very effective with the MSIT Integration Team for handling the Gentran migration. We mitigated the risks involved; which impact a business worth billions of dollars. The phase-wise delivery and agile rollback process are the key considerations in this approach. The Migration Strategy section in this white paper provides more details.

Leverage Automation Framework and Tools

MSIT integration team looked for various opportunities to reduce the development and testing cost of BizTalk artifacts. As a result, the following tools/framework were created:

a. Gentran Migration Toolb. Test Automation Frameworkc. Data Segregation Tool

For details on the tools and framework, see the Automation section in this whitepaper.

Reinforce known best practices of BizTalk Server 2010

The following lists some of the best practices taken from this BizTalk Server 2010 whitepaper:

Ensure consistent monitoring and tracking of messages is present across all applications. Avoid overloading individual hosts. Instead, use the appropriate number of BizTalk 2010 servers and decide how

many applications to deploy on each of them in the design phase. Validate your message volume per host and modify the default settings as appropriate. Ensure that the BizTalk SQL Agents Jobs run successfully in SQL Server to release all the resources. Ensure that you have a backup and replication strategy in place. When possible, avoid large schemas with mostly optional fields. Ensure that appropriate permissions exist for SSO, SQL, and BizTalk. Modify default retries and timeouts as appropriate.

The following lists the performance improvements implemented during the migration process:

Utilized the SQL Server Backup Compression feature in BizTalk Server; which reduces the size of the BizTalk backups and the time consumed for backing up the databases.

In BizTalk Server 2010 Management Pack for System Center 2010, set the discovery interval to lower levels. This resolved the performance issues related to resource utilization in previous BizTalk versions.

To achieve higher performance, the Max Receive Interval is tuned at the host Level in BizTalk 2010. Performance was further improved by tuning the Max Polling Interval and Orchestration polling interval. See this

22

(http://blogs.msdn.com/b/prethishkumar/archive/2012/02/04/high-performing-biztalk-server-2009-biztalk-server-2010-server-setup.aspx) blog for more information.

Upgraded the hardware (for example, increasing the RAM and disk space) to reduce disk I/O Latency.

9 Appendix

9.1 Glossary

Terms DefinitionStream A transaction that runs for one specific partner. For example, there are two partners (X, Y)

and two X12 transactions (850, 852) running for both partners. The total number of streams is four.

10 Conclusion BizTalk 2010 is implemented by many organizations worldwide today for their mission-critical integration. BizTalk 2010 is for high volume, high availability, and high scalability; which are required for business process solutions. Migrating to BizTalk 2010 gives many benefits to the organization and to customers.

11 AcknowledgementsMany thanks for the technical information and input provided by Nikhil Tayal, Anil Chandra Lingam. A special thanks to Mandi Ohlinger and Nitin Mehrotra for the technical review and valuable insights on this white paper.

Did this paper help you? Please give us your feedback here. Tell us on a scale of 1 (poor) to 5 (excellent), how would you rate this paper and why have you given it this rating? For example:

Are you rating it high due to having good examples, excellent screen shots, clear writing, or another reason?

Are you rating it low due to poor examples, fuzzy screen shots, or unclear writing?

This feedback helps us improve the quality of white papers released by MSIT.

23