DAT 332

SQL Server 2000 Data Transformation Services (DTS) Best PracticesEuan GardenProduct Unit ManagerSQL Server DevelopmentMicrosoft Corporation

Technologies Covered

Data Transformation Services

Analysis Services (Decision Support Objects)

Transact SQL

ADO

ActiveX Scripting

“Best Practices” Agenda

Package Design

Data Loading

Advanced Techniques

Auditing and Error Handling

Analysis Services Management

Package Design Practices

Drive Packages through MetadataDynamic Properties Task

Access to Property of any object

Sources values from many locales

Global VariablesBuilding blocks for communicating metadata

Populated by the Dynamic Properties Task

UDL FilesConfigurable File for storing Connection Information

Package Design Practices

Designing Modular Parent/Child Packages

Execute Package TaskAllows joining of Transaction

Supports passing of Globals

Execution through COM AutomationProgrammatic access to Package objects

Eliminates Version Affinity

Parent Child Packages

Child Package is Invoked by ActiveX Script

Can also be done with Execute Package Task

DTS Parent

DTS Child – Error Handler Example

DemoDemo

Parent/Child Package(s) And UDL Files

Design Practices

“Best Practices” Agenda

Package Design

Data Loading

Advanced Techniques

Auditing and Error Handling

Analysis Services Management

Data Loading Practices

Leverage Fast Load Interface(s)Bulk Insert (Not always possible)

Data Pump (For More Complex Logic)

Characteristics of this ApproachBulk Insert Task

Execute SQL Task Rowset Parameter

ADO Disconnected Recordsets

DTS Multiphase Data Pump

Data Loading Practices

Benefits to this ApproachCache Rowset on Client (fast lookups)

Hold updates on Client

Conditional Inserts/Updates without Data Driven Query

DemoDemo

Conditional Inserts And Updates With The Multiphase Data Pump

Design Loading

“Best Practices” Agenda

Package Design

Data Loading

Advanced Techniques

Auditing and Error Handling

Analysis Services Management

Advanced Practices

Multi-Threaded Data LoadingRequires Multiple Connections

Data is Loaded in Parallel

Avoid Contention by Loading into Separate Entities

Advanced Practices

Task LoopingRequires at least two tasks (Executive and Trigger)

Facilitated by Manipulating Workflow

<Step>.ExecutionStatus = DTSStepExecStat_Waiting

Maintain State in Global Variables

Advanced Practices

Communication through Global Variables

Only Means for Communicating among Tasks

ExamplesPointer to Position within Global Array

Status of a Stored Procedure Call

Flag to Shutdown Parallel Load Processes

Advanced Practices – Benefits

Higher Utilization during Data Load Compressed Data Loading Window Easy Orchestration within DTS

Multi-Threaded Loading

Task MarshallTask Marshall

Thread 1 Thread 1 Loading TaskLoading Task

Initialization TaskInitialization Task

Thread n Thread n Loading TaskLoading Task

Thread 1 Thread 1 Restart TaskRestart Task

Thread n Thread n Restart TaskRestart Task

DemoDemo

Multi-Threaded Data File Loading

Advanced Practices

“Best Practices” Agenda

Package Design

Data Loading

Advanced Techniques

Auditing and Error Handling

Analysis Services Management

Auditing And Error Handling

Logging OptionsLogging to MSDB

Logging to Text File

Logging to Event Log

Enabled through Package PropertiesCapture Step/Package Errors

Enabled through Data Pump PropertiesCapture Erroneous Data

Auditing And Error Handling

DTSPackageLog Object WriteStringToLog()

WriteTaskRecord()

DTS Lineage Global VariablesDTSLineage_Full

DTSLineage_Short

DTSErrorRecords ObjectAdd()

Clear()

Auditing And Error Handling

Monitoring Execution – Simple as Writing SQL

select

package.name, package.description, step.stepname,

step.errorcode, step.errordescription

from

msdb..sysdtspackagelog package, msdb..sysdtssteplog step

where

package.lineagefull = step.lineagefull and

step.errorcode <> 0

DemoDemo

Custom Logging Using Intrinsic DTS Options

Error Handling

“Best Practices” Agenda

Package Design

Data Loading

Advanced Techniques

Auditing and Error Handling

Analysis Services Management

Analysis Services And DTS

Analysis Services Processing TaskSupports Processing of all OLAP Objects

Customizable Processing through TreeKey and other properties

Prominent PropertiesTreeKey ServerName\DBName\CubeFolder\CubeName

ItemType 1=OLAP DB, 4=Cube, 7=Partition and 9=Dimension

ProcessingOption 0=Full Process, 1=Refresh Data and 2=Incremental Update

Analysis Services And DTS

Decision Support ObjectsProvide access to the AS Management Interfaces

Automated via AxScript

Supports Partition ManagementCreating

Cloning

Dropping

Merging of partitions

Should be an OLAP Administrator to Use

DSO Hierarchy

Analysis Services And DTS

BenefitsOpportunity to Dynamically Manage Partitions and other Objects

Support for Dynamic Processing

Inclusion in the BI Solution Load

DemoDemo

Dynamic Partition Creation And Processing

Analysis Services

Community ResourcesCommunity Resources

DTS Whitepaper – Search on MSDN for “Using DTS for Business Intelligence”

Microsoft Sitewww.microsoft.com/sql

SQLDTS Site www.sqldts.com

SQL Server Developers Network www.sqldev.net

SQLServer Centralwww.sqlservercentral.com

Ask The ExpertsGet Your Questions Answered

I’ll be in the Experts booth right after this session

euang@microsoft.comeuang@microsoft.com

Questions?Questions?

Community Resources

Community Resourceshttp://www.microsoft.com/communities/default.mspx

Most Valuable Professional (MVP)http://www.mvp.support.microsoft.com/

NewsgroupsConverse online with Microsoft Newsgroups, including Worldwidehttp://www.microsoft.com/communities/newsgroups/default.mspx

User GroupsMeet and learn with your peershttp://www.microsoft.com/communities/usergroups/default.mspx

evaluationsevaluations

© 2003 Microsoft Corporation. All rights reserved.© 2003 Microsoft Corporation. All rights reserved.This presentation is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY.This presentation is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY.