Upload
btugbe
View
1.134
Download
0
Embed Size (px)
DESCRIPTION
Toon Vanhoutte s
Citation preview
BTUG – BizTalk TrackingToon Vanhoutte – Integration Architect
2
Configuration
Under the hood
Best Practices
Demo Visualization
Tools
Agenda
3
SetupDemo
Specify unique reference for messages #search
Specify simulated response times of backend systems #sla
Specify customer and transporter names #inspiration
4
Configuration
5
| Port Tracking Configuration
Configuration
Only track what’s really required #performance
6
| Pipeline Tracking Configuration
Configuration
Bug BTS2010 CU2 CU5 #solved #CU6 #kb2728652
7
| Orchestration Tracking Configuration
Configuration
Avoid Shape start and end #performance #orchestrationdebug
8
| Property Schema Tracking Configuration
Configuration
Make context properties searchable #troubleshooting
Global + specific property schema #generic #consistent
9
| Disable Global BizTalk Tracking
Configuration
10
Under the hood
11
| BizTalk Database Structure
Under the hood
11
SQL Agent
Host Instance processes message
12
| BizTalk Database Structure
Under the hood
12
SQL Agent
Host Instance processes message
Message stored in MessageBox
13
| BizTalk Database Structure
Under the hood
13
SQL Agent
Host Instance processes message
Message stored in MessageBox
TDDS copies events to DTA + BAM
14
| BizTalk Database Structure
Under the hood
14
SQL Agent
Host Instance processes message
Message stored in MessageBox
TDDS copies events to DTA + BAM
SQL Agent copies bodies to DTA
15
| Store messages in MessageBox database| Physical runtime message bodies are stored in:
| Spool| MessageParts| Parts| Fragments
Under the hood
15
Fragmentation controlled by large message size
16
| Process messages in MessageBox database| Per Host, these tables are created:
| HostNameQ: Main queue: each host instance polls this queue table to collect messages that are pending processing
| HostNameQ_Suspended: Suspended queue: queue table that holds supended messages
| HostNameQ_Scheduled: Scheduled queue: not used| InstanceStateMessageReferences_Host: State queue:
holds list of messages that have been processed but are required later
Under the hood
16
Stores only reference to physical message in Spool table
17
| Store messages in MessageBox database
Under the hood
17
#probts2006
18
| Store tracking data in MessageBox database| Host Instance serializes data and stores it in the
MessageBox| Message events:
| TrackingData_0_x [x: 0 - 3]| TrackingData_1_x [x: 0 - 3]
| Message bodies:| Spool| MessageParts| Parts| Fragments
Under the hood
18
#probts2006
19
| Copy Events from MessageBox to DTA database
| Tracked Message Events copied by TDDS Service:| Tracking Data Decode Service| Runs in each host instance with tracking enabled
Under the hood
19
Create a dedicated tracking host
20
| Copy Events from MessageBox to DTA database
Under the hood
20
BizTalk MessageBoxTrackingData_0_0TrackingData_0_1TrackingData_0_2TrackingData_0_3
TrackingData_1_0TrackingData_1_1TrackingData_1_2TrackingData_1_3
BAM Primary ImportActivity_ActiveActivity_ActiveRelationshipsActivity_CompletedActivity_CompletedRelantionshipsActivity_Continuations
BizTalk DTA DatabaseDta_MessageInOutEventsDta_ServiceInstances...
TDDS
21
| Copy Events from MessageBox to DTA database
Under the hood
21
BizTalk MessageBoxTrackingData_x_x
BAM Primary ImportTDDS_StreamStatus TDDS_FailedTrackingData
BizTalk DTA DatabaseTDDS_StreamStatus TDDS_FailedTrackingData
TDDS
TDDS_StreamStatus contains last copied sequence n°
TDDS moves events with > sequence n° from TrackingData_x_x to DTA
TDDS_FailedTrackingData contains tracking failures
22
| Copy Bodies from MessageBox to DTA database
| Tracked Message Bodies copied by SQL Agent Job:| TrackedMessages_Copy_BizTalkMsgBoxDb| Scheduled by default each minute (development: 10
seconds)
Under the hood
22
Monitor the SQL Agent Jobs #SCOM #blog
23
| Copy Bodies from MessageBox to DTA database
Under the hood
23
BizTalk MessageBoxSpoolMessagePartsPartsFragments
BizTalk DTA DatabaseTracking_Spoolx Tracking_Partsx Tracking_Fragmentsx
SQLAgent
[x: 1 - 2]:Always one online / one offline
24
| Remove data from MessageBox| This is performed by SQL Agent Jobs:
| MessageBox_Message_ManageRefCountLog_BizTalkMsgBoxDb:| Runs continuously (due to former SQL limitation of 1 min job
schedule)| This jobs checks weather a message is still referenced by instances
| MessageBox_Message_Cleanup_BizTalkMsgBoxDb| Is disabled (keep it like this)| It’s called from with the infinite loop of the ManageRefCount Job| This job deletes physical messages from MessageBox that have no
outstanding references anymore
Under the hood
24
25
| Remove data from DTA| Enable + Configure the DTA Purge and Archive Job
#avoidlargeDTA
| Remove data from BAM| SQL Agent Job BAM_DM_Activity moves data to BAMArchive
Under the hood
25
26
| BizTalkDTADb Database model| No foreign keys #performance
| Designed for fast insert| Results in slower query
| No single chain ID| Difficult to reproduce the “route” within BizTalk
Under the hood
26
27
| BizTalkDTADb Database model| No foreign keys #performance
| Designed for fast insert| Results in slower query
| No single chain ID| Difficult to reproduce the route within BizTalk
Under the hood
27
| Create a chain of messages
Under the hood
29
| Create a chain of messages
Under the hood
29
SI = 1MI = A
SI = 1MI = B
SI = 2MI = B
SI = 2MI = C
SI = 3MI = C
SI = 3MI = D
dta_MessageInOutEvents Service Instance ID Message Instance ID
30
Tools & SDK
30
31
| BizTalk Administration Console #technical #notsexy
Tools & SDK
31
32
| BizTalk MessageBox Viewer #troubleshooting
Tools & SDK
32
33
| BizTalk Terminator #troubleshooting
Tools & SDK
33
Import MBV scripts
34
| Performance Counters #troubleshooting| Throttling state find reason on MSDN| TDDS
Tools & SDK
34
35
| Retrieve tracked message body #3ways| Microsoft.BizTalk.Operations.dll
Tools & SDK
35
36
Visualization
36
37
Visualization
37
| Bad practice| Develop a dashboard directly on BizTalkDTADb
#performance| No control on user queries| Impacts directly the BizTalk runtime| Bad dashboard performance
38
Visualization
38
| Bad practice| Develop custom tracking mechanism
39
Visualization
39
| Good practice| Use default BizTalk Tracking #CID
40
Visualization
40
| Good practice| Use BizTalk Tracking #CID
41
Visualization
41
| Business Activity Monitoring? #notideal| Development impact
| Tracking Profile Editor is limited| BAM API is rather complex
| SQL Server Analysis Services| 2008 R2 required, when using SQL 2012
| Deprecated BAM portal| Impact on operations| Not ideal OLAP cube structure| Termination of activities not straightforward
42
Visualization
42
| Use case #1: Message Search| BizTalk Administration Console
| Codit Integration Dashboard
43
Visualization
43
| Use case #1: Message Search| Search on context properties:
| Application| Flow| Sender| Receiver| Reference
Demo
44
Visualization
44
| Use case #1: Message Search
Demo
45
Visualization
45
| Use case #2: Operational reports | Message Load| Message Size| Database Size
#managedservices
46
Visualization
46
| Use case #3: Pro-Active Monitoring| Custom SCOM alert: monitoring non-events #vbs
| E.g.: Just-In-Time ordering| Daily check on expected amount of received orders| Query on BizTAlkDTADb| Automated email alert / support ticket
#managedservices
47
Visualization
47
| Use case #4: SLA Reporting| Processing times of web services
| Total processing time| Processing time of every backend system| Processing time in BizTalk
48
Visualization
48
| Use case #4: SLA Reporting
Demo
rp_MasterDataService.WCF
sp_CRM_GetCustomer.WCF
sp_SAP_GetItem.WCF
sp_AX_GetEmployee.WCF
ORCH
GetCustomerGetCustomerItemList
GetItemGetEmployee
49
Visualization
49
| Use case #4: SLA Reporting
Demo
rp_MasterDataService.WCF sp_SAP_GetItem.WCF
Tota
l Pro
cess
ing
Tim
e SAP Processing Time
BizTalk Processing Time = Total Processing Time – SAP Processing Time
1 2 3 4
8 7 6 5
Same MessageID
Same MessageID
50
Visualization
50
| Use case #4: SLA Reporting| SQL Agent Job
| BizTalkDTADB Custom database #chaining
Demo
51
Visualization
51
| Use case #4: SLA Reporting
Demo
52
Visualization
52
| Use case #4: SLA Reporting
Demo
53
Visualization
53
| Use case #5: Process Overview| Visualization of related integration flows
54
Visualization
54
| Use case #5: Process Overview| Example: EDI Order Processing
Demo
55
Visualization
55
| Use case #5: Process Overview
Demo
56
Visualization
56
| Use case #6: Dashboards| BizTalk 360: Operational focus| AIMS for BizTalk: Behavior / trend analysis| Codit Integration Dashboard: Advanced message
handling| Minotaur| Frends Helium| Moesion| …
57
Visualization
57
| Use case #6: Dashboards| Codit Integration Dashboard
| Friendly Naming of technical artifacts| Message Search| Message Resume / Resubmit
| Editing| Multi selection
Demo
58
Visualization
58
| Use case #6: Dashboards| Codit Integration Dashboard: resubmit
Demo
59
Questions?
59
You
BTUG
Microsoft