TM Deployment Guide

7/30/2019 TM Deployment Guide

1/79

Real Data Integration Solutions

TM DeploymentSeptember 2010

ETL Solutions Limited 2010 ETL Solutions Ltd.


2/79

ETL Transformation Manager TM Deployment

i

Part Number: TM418-TMD

Doc. Issue: 02 (September 2010)

Version Information

This edition ofTM Deploymentis subject to change without notice and is supplied without warranty ofany kind.

Copyright Notice

All information contained in this document is the property of ETL Solutions Limited. The informationcontained in this document is subject to change without notice and does not constitute a commitmenton the part of ETL Solutions Limited. No part of this document may be reproduced in any manner,including storage in a retrieval system, transmission via electronic means or other reproductionmedium or method (electronic, mechanical, photocopying, recording or otherwise) without the priorwritten permission of ETL Solutions Limited.

2010 ETL Solutions Limited. All rights reserved.

All trade marks acknowledged.

All trade marks mentioned herein belong to their respective owners.


3/79


ii

Preface

About TM Deployment

This is a user guide for users wishing to deploy TM transforms, (a) using the TM Deployment Tool, (b)from a command prompt or within a script or batch file, or (c) by using pure Java code.

Documentation Set

TMs documentation set, shown in the diagram below, is structured hierarchically, with documentsaimed at specific types of user.

The product distribution includes,

For the User:

TM Basics which contains step-by-step instructions for developing transformationprojects that illustrate both general principles and specific techniques;

TM Featureswhich describes a number of individual features to be found in TM. It isa self-contained tutorial-style introduction to some of the more advanced features and

is intended to supplement TM Basics.

TM Deployment which explains how to deploy transforms generated by TM.

These three documents constitute the primary user guides to TM, and it is stronglyadvised that the user begin with these documents in order to gain a sound knowledge

of TM and its use.

TM Upgrade Tool which explains how repositories created by previous versions ofTM may be upgraded so that they may be used by the current version.

TM Release Noteswhich outline any new features and known issues in the currentrelease are also included.

For the Administrator:

Advanced User

TM

Deployment

TM

FeaturesTM Upgrade

Tool

Database

Guide

Using and

Creating

Transform

Packs

Patterns

Guide

Key

User Administrator Data modeller Databaseexpert

TM

BASICS

Installation

Guide

Properties

Editor

Guide

TM Release

Notes

Building

TM

Projects

Program

matically

TM

Runtime

Error

Handling


4/79


iii

I nstallation Guidewhich provides Windows and UNIX installation instructions forTM.

Properties Editor Guide which describes how users may edit and extend theproperties stored in the lax files generated when installing TM.

For the Data Modeller:

Patterns Guide which is an advanced guide describing patterns of use for solvingcommon transformation problems.

For the Database Expert:

Database Guidewhich provides an introduction on how to use TM to transform datato a database efficiently.

For the Advanced User:

Buil ding TM Projects Programmaticall ywhich describes how to build TM projectsprogrammatically.

Using and Creating TransformPacks which explains what a TransformPack is, howto run one and how to create one.

TM Runtime Error H andling which describes the behaviour of the TM runtimewhen errors are thrown.

Audience

This guide is intended for those who wish to deploy TM transforms.


5/79


iv

Contents

PREFACE .................................................................................................. ............................................ IICONTENTS ....................................................... ............................................................. ..................... IVTABLE OF FIGURES ........................................................................................................................ VIINTRODUCTION ....................................................... ........................................................... ................ 1TM DEPLOYMENT TOOL................................................ ............................................................. ..... 2

INTRODUCTION..................................................................................................................................... 2USING TMDEPLOYMENT TOOL .............................................................. ............................................. 2

Available Tasks ............................................................... ........................................................... ..... 3Scheduled Tasks ................................................................................. ........................................... 12Output ...................................................................................... ..................................................... 16

COMMAND LINE DEPLOYMENT ....................................................... ........................................... 17INTRODUCTION................................................................................................................................... 17INSTRUCTIONS.................................................................................................................................... 17

Step 1 ............................................................................................................................................ 17Step 2 ............................................................................................................................................ 17Step 3 ............................................................................................................................................ 17

LICENSING INFORMATION............................................................... .................................................... 17JNDIPASSWORDS .............................................................................................................................. 18

PURE JAVA DEPLOYMENT ....................................................... ..................................................... 18INTRODUCTION................................................................................................................................... 18LICENSING INFORMATION............................................................... .................................................... 18GENERATION OF JAVA CODE ......................................................... ..................................................... 19TMTRANSFORMER/TMTRANSFORMEXECUTOR:OVERVIEW OF OPERATION.................................... 19

Initialising TMTransformer / TMTransformExecutor.............. ..................................................... 19Setting Adaptors .............................................................. ........................................................... ... 19

Running the Transforms and Getting Results ............................................................................ ... 20JNDI Passwords ............................................................. ........................................................... ... 21

TMTRANSFORMEXECUTOR-MAIN LOOP .......................................................... ................................ 22Basic pattern ........................................................ ........................................................... .............. 22Variations ..................................................................................................................................... 22

SESSIONS DIVISIONS AND PARTITIONS .................................................... ........................................... 25"Partitions" and "Divisions" .............................................................. ........................................... 25Transformation Architecture with Partitions and Division ..... ..................................................... 25Stalled Handles .................................................................................. ........................................... 26

Built-in Adapters and Sessions .......................................................... ........................................... 26CLEARING A PROJECT......................................................................................................................... 26TRANSFORMS FROM $DOCUMENT,$PREDOCUMENT AND $POSTDOCUMENT .................................... 26RUNNING THE SAMPLES ..................................................................................................................... 27SHORT-LIVED DEPLOYMENT EXAMPLES ............................................................ ................................ 28

Simple Example ............................................................... ........................................................... ... 28Java to Java Example ............................................................... .................................................... 37Java to XML Example ( as serialised file) .................................................... ................................ 37Java to XML Example (as DOM in memory) .......................................................... ...................... 38


6/79


v

Java to XML Example (as serialised memory output) .................................................................. 39XML (as file) to Java Example ........................................................... ........................................... 39XML (as inputStream ) to Java Example ..................................................................................... 41Java to Database URL Example ........................................................ ........................................... 41Java to Open RDBMS Connection Example ........................................................... ...................... 42Database URL to Java Example ........................................................ ........................................... 42Open RDBMS Connection to Java Example ........................................................... ...................... 43Java to Java (with Error Handler) Example ........................................................... ...................... 43

Java to Java (with embedded objects) Example............................................ ................................ 44Java to Generic Example ......................................................... ..................................................... 46Generic to Java Example ........................................................ ..................................................... 46

Flat File to Flat File example (using the built in flat file adapters) ............................................. 47LONG-LIVED DEPLOYMENT EXAMPLES.............................................................. ................................ 47

Simple Example ............................................................... ........................................................... ... 47Java to Java Example ............................................................... .................................................... 55Java to XML (as serialised file) Example ..................................................... ................................ 56Java to XML (as DOM in memory) Example .......................................................... ...................... 57Java to XML (as serialised memory output) Example .................................................................. 58XML (as file) to Java Example ........................................................... ........................................... 58XML (as W3C Document ) to Java Example ................................................................................ 59XML (as inputStream ) to Java Example ...................................................................................... 60JMS Queue to JMS Queue (With XML) ........................................................ ................................ 60Java to Database URL Example ........................................................ ........................................... 63Java to Open RDBMS Connection Example ........................................................... ...................... 64Database URL to Java Example ........................................................ ........................................... 64Open RDBMS Connection to Java Example ........................................................... ...................... 65

Java to Java (with Error Handler) Example ........................................................... ...................... 66Java to Java (with embedded objects) Example............................................................................ 67Java to Generic Example ......................................................... ..................................................... 69Generic to Java Example ......................................................... ..................................................... 69

CLASSPATH ..................................................... ............................................................. ...................... 70


7/79


vi

Table of Figures

Figure 1Deployment Tool Dialog ............................................................ ............................................. 2Figure 2New Task Project Name ............................................................. ............................................. 3Figure 3New Task Source Type ........................................................................................................... 4Figure 4New Task Source Configuration ............................................................................................. 4Figure 5New Task Target Type ..................................................... ....................................................... 5Figure 6New Task Target Configuration ............................................................ .................................. 5Figure 7New Task Properties ........................................................ ....................................................... 6Figure 8New Task Error Handlers ........................................................... ............................................. 6Figure 9Error Handlers Project Name .................................................................................................. 7Figure 10Error Handlers Target Type ................................ ............................................................. ..... 7

Figure 11Error Handlers Target Configuration ............................................................................... ..... 8Figure 12Error Handlers Project Display ........................................................................................ ..... 8Figure 13New Task Name .............................................................. ...................................................... 9Figure 14Deployment Tool Main Dialog ............................. ........................................................... ..... 9Figure 15New Main Method Task: Main Class.............................. .................................................... 10Figure 16New Main Method Task: Runtime Arguments ................................................................ ... 10Figure 17New Main Method Task: Task Name ...................................... ........................................... 11Figure 18New Main Method Task Completed ................................................................................... 11Figure 19Scheduled Tasks .............................................................. .................................................... 12Figure 20Scheduled Tasks List ...................................................... ..................................................... 13Figure 21Scheduling for New Task ......................................................... ........................................... 13Figure 22Scheduled Task Name .............................................................. ........................................... 14Figure 23Completed New Scheduled Task ............... ........................................................... .............. 14Figure 24Task History ........................................................................................ ................................ 15Figure 25Output ................................................................................................................................. 16


8/79


1

IntroductionTransformation Manager (TM) is used to develop transformation programs that transform data from one formatto another. The developed transformation programs may then be deployed in three main ways. This document

describes how to deploy those programs using various methods:

Using TM Deployment Tool, a stand-alone environment for scheduling and running transforms.

Using Command Line Utilities

Using Java code

TM Deployment Tool provides the easiest deployment method. However, depending upon the deploymentrequirements, the user may choose to run TM transforms from the Command Line or to write a JAVA programthat calls TM Transforms. The latter provides the greatest flexibility and supports many kinds of EAI, EII,Business Intelligence integration architectures. In reality, the Java-calling program required for thesearchitectures are relatively simple and a number of examples are described in the Pure Java Deploymentsection. ETL will provide other examples to fit with EAI, EII, Business Intelligence integration architectures onrequest.


9/79


2

TM Deployment Tool

Introduct ion

The TM Deployment Tool provides a stand-alone environment for scheduling and running transforms. ADeployment Task is a specific transform, with source and target defined, any parameters needed specified, and aschedule for running that transform.

Using TM Deployment Tool

Figure 1Deployment Tool Dialog

The main dialog of the TM Deployment Tool lists the current tasks that have been created by the user. Tasksthat have been created previously will be read in automatically when the Deployment Tool is launched.

There are three main sections to the TM Deployment Tool dialog:

Available Tasks

Scheduled Tasks

Output

Transform Tasks to be used by the TM Deployment Tool must first be created.

Available Tasks may then be Scheduled to run according to the users preference.

Information on the running projects are displayed in the Output section.

Previously created Available and Scheduled Tasks are loaded when the Deployment Tool is launched.


10/79


3

Available Tasks

Within the Available Tasks section of the TM Deployment Tool, the user may:

Create a New Transform Task

Create a New Main Method Task

Edit a Task, or

Delete a Task

Create New Transform Task Wizard

To create a new transform task, click on New Transform Task within the Available Tasks section of themain TM Deployment Tool dialog.

The New Transform Task Wizard will take you through the tasks involved in creating a new task.

Select Project

Select the project that you want to deploy. You may either point at the generated Jar File, or manually specifythe project name and major version.

The project name is case sensitive and must be entered correctly.

The generated Jar file must be in the current classpath:

Figure 2New Task Project Name

Press Next to go to the Source Type page.


11/79


4

Source Type

Select your source model typewhether it is an XML file, RDBMS, Java or generic:

Figure 3New Task Source Type

Press Next to go to the Source Configuration page.

Source Configuration

Specify the source configuration. The configuration fields depend on the type selected on the previous page.

Figure 4New Task Source Configuration

Press Next to go to the Target Type page.


12/79


5

Target Type

Select the target type.

Figure 5New Task Target Type

Press Next to go to the Target Configuration page.

Target Configuration

Specify the target configuration. The configuration fields depend on the type selected on the previous page.

Figure 6New Task Target Configuration

Press Next to go to the Properties page.


13/79


6

Properties

You may specify any properties needed for the task at run-time here. These are usually required if a

getString() appears in the transformation project. The properties are added as named, value pairs.

Figure 7New Task Properties

Press Next to go to the Error Handlers page.

Error Handlers

It is possible to configure an error handling project into the Deployment Task.

Figure 8New Task Error Handlers


14/79


7

Press Add and specify the error handling project's name and major version:

Figure 9Error Handlers Project Name

Specify the target model type:

Figure 10Error Handlers Target Type


15/79


8

Specify the target configuration:

Figure 11Error Handlers Target Configuration

The details of the error handling project will be displayed on the dialog:

Figure 12Error Handlers Project Display

Press Next to go to the Task Name page.


16/79


9

Task Name

Enter a name for the Task.

Figure 13New Task Name

Press Finish to save the task details.

New Main Method Task Wizard

The TM Deployment Tool can launch Java code directly by means of the New Main Method Task Wizard.

Figure 14Deployment Tool Main Dialog

Click on New Main Method Task... to proceed to the Main Class page.


17/79


10

Main Class Page

Specify the fully qualified path of the class that contains the 'main(String[]args)' method to be executed.

Figure 15New Main Method Task: Main Class

Press Next to proceed to the Runtime Arguments page.

Runtime Argument Page

Specify any run-time arguments here. Press Add... or Add Encrypted... if you wish the argument to beprotected (e.g. for a password).

Figure 16New Main Method Task: Runtime Arguments

Press Next to go to the Task Name page.


18/79


11

Task Name Page

Supply a name for the Task Name.

Figure 17New Main Method Task: Task Name

Press Finish to create the task.

Completing the New Main Method Task Wizard

Figure 18New Main Method Task Completed

The newly created task may now be edited, deleted or scheduled.

Edit Task

The stages involved in editing a task are the same as when creating a task. ClickEdit Task

Delete Task


19/79


12

Select the task to be deleted from the TM Deployment Tool main dialog. Choose Delete from the File menu, oruse the Delete button on the toolbar. The task and its history will be permanently deleted.

Scheduled Tasks

Within the Scheduled Tasks section of the TM Deployment Tool, users may:

Create a new Scheduled TaskEdit a Scheduled Task

Delete a Scheduled Task, or

Display the History of a Task

Create New Schedu led Task

The New Scheduled Task Wizard takes the users through the tasks involved in creating a new scheduled task.

The selected tasks will be run in sequence according to the users scheduling rules.

Figure 19Scheduled Tasks

Click on New Scheduled Taskto bring up the New Scheduled Task Wizard.

Select Tasks

Specify at least one task to be scheduled. Press Add to select.


20/79


13

Figure 20Scheduled Tasks List

Press Next to proceed to the Scheduling page.

Scheduling

Specify the scheduling rules.

Figure 21Scheduling for New Task

The Scheduling Page allows users to specify that a test runs as frequently as you wish.

You may choose at which hour of which day of which month the test will first be run and how frequentlythereafter.

In the screenshot above, the test is scheduled to run at 1:00 pm on the 27th

of the month and at the same timeevery day thereafter.

To run a test once a week, choose Run Periodically Every 7 Day(s).

Press Next to proceed to the Task Name page.

Task Name

Specify a name for the task.


21/79


14

Figure 22Scheduled Task Name

Press Finish to complete the task creation.

Completed New Scheduled Task

Figure 23Completed New Scheduled Task

The newly created scheduled task may now be edited or deleted, and its history may also be viewed.

The new scheduled task must be enabled (by ticking the Enabled box) before it will run.

Display the Task History

A summary of the history of a particular scheduled task may be viewed by clicking on History


22/79


15

Figure 24Task History


23/79


16

Output

This section displays the output stream from all running projects, and includes information about transforms andperformance.

Figure 25Output


24/79


17

Command Line Deployment

Introduct ion

This section describes how to deploy transforms from within a script or batch file.

Instruct ions

The following steps describe how to run transformation programs from a command prompt.

Step 1

Ensure the following:

There is a suitable Java JRE on your system. For more information on Java versioncompatibility see the section Java Versions in the Installation Guide.

All required jars are on the classpath. For more information see the section Classpath in thisdocument

Step 2

Runtime configuration is specified in a properties file. If you have previously run the project in the TMTest Tool there is a file that can be copied:

[TMHOME]/config/Exec.properties.

Edit this file if any configuration details need to be changed, for example you may want to specifydifferent database connection details.

Step 3

Run the transforms with this command line (the generated Exec.properties file has been copied and re-named myExec.properties and is in the current directory):

java Dfile=myExec.propertiescom.prismt.transform.runtime.exec.RunMaps

The following command line executes the original Exec.properties file and can be used when that filehas been edited directly:

java com.prismt.transform.runtime.exec.RunMaps

The above command is equivalent to using the following command:

java -Dfile=[TMHOME]/config/Exec.propertiescom.prismt.transform.runtime.exec.RunMaps

NoteThe original Exec.properties file is overwritten each time a transformation is run using the TestTool and will therefore need to be edited repeatedly. Editing and running a copy of it avoids constantediting. You can write this file from the Test Tool without actually executing the transform by usingthe Setup menu option.

Licensing Informat ion

Before deploying code from the command line you must be correctly licensed.

When calling java com.prismt.transform.runtime.exec.RunMaps you need to pass in the licencelocation as a VM parameter (i.e. as a -D argument), using etl.license.file as the key i.e.

java -Detl.license.file=etc/etltm.lic


25/79


18

JNDI Passwo rds

If a project contains lookups which require passwords for JNDI database connections, they are addedas application arguments to the RunMaps main method. The format is password.[JNDIname]=[password]

For example:

com.prismt.transform.runtime.exec.RunMapspassword.firstConnection=pass1

password.secondConnection=pass2

Pure Java Deployment

Introduct ion

TM is meta-data driven, but generates pure Java code. This code may be used on any Java JM (1.3 andabove) to perform data transformations.

From release 2.4 onwards it has been possible to run TM run in the 'short-lived' or 'long-lived' mode.The main difference is that in the 'short-lived' mode TM objects are freshly created for particulartransforms, and then garbage collected, while in 'long-lived' mode TM objects are allowed to persist inmemory. When running in 'short-lived' mode the principal TM class is called TMTransformer; in'long-lived' mode it is called TMTransformExecutor.

Historically, TM has been packaged for use in a stand-alone environment, and hence in 'short-lived'mode. In this mode of operation a run-time interface TMTransformer object exists only for the lifetimeof the transforms. This use of TM is well documented in existing documentation and has itsattractions. Where machine efficiency is not of paramount importance this type of short-livedTMTransformer can be a very neat way of packaging transforms. However, the main drawback is that

it takes time to create the TMTransformer object - even when the transforms are packaged as Java.Where the transforms are packaged as XML (i.e. are supplied as XML meta-descriptions) then thistime increases still further.

For this reason, ETL also provide a long-lived component known as TMTransformExecutor. Inaddition, TMTransformExecutor gives a more fine-grained control over transform execution.

Note that both TMTransformer and TMTransformExecutor extend the common interfaceTMTransformerCommon, and thus contain many of the most important methods.

This guide begins by explaining the method by which Java Code is generated. It then describes allaspects of deploying Java transforms generated by TM at run-time, including such topics as:

An Overview of TMTransformer and TMTransformExecutor Operation

Sessions Divisions and Partitions used, for example, to control transactionsQuery Propagation used to obtain source data in response to a query in terms of the targetmodel

The Guide concludes with a set of examples using 'short-lived' and long-lived transforms.

Licensing Informat ion

There is a method on the net.etltm.TMTransformerFactory

public static void setTMLicenseLocation(String theLocation)

When executing a transform, this call must be made on the TMTransformerFactory before the factoryis used to create a TMTransformer. The location can be path to the licence file, or a URI to thelocation of the license file on a remote machine.


26/79


19

If deploying in a non-file based environment, the call, setTMLicenseLocation must be called (asabove), but passing in a special string (which will be supplied by ETL in these cases), which then stopsthe run-time system from checking for a licence.

Generation o f Java Code

When building projects, TM takes the model meta-data and SML transforms and generates Java code.The generated Java code is compiled and the packaged in a generated jar file. Various settings affectthe building of the jar file, for information see the 'SML Generation Settings' page in the online help.

TMTransformer / TMTransformExecutor: Overview of

Operation

Both TMTransform and TMTransformExecutor are created from the factory classTMTransformerFactory.

Both are provided with a default constructor (zero argument) and a constructor taking an arbitrary JavaObject. If an arbitrary Java object is provided, this can be used by Java functions written in TM to call

back to the calling program. To do this merely requires the functions to use the

TMContext:getUserReferenceObject within their body.As constructed, neither object is initialised, and must be properly initialised before it can be used totransform data. Once initialised, it is in a 'ready' state - available to take input data and transform it intooutput data.

Initialising TMTransformer / TMTransformExecutor

All TMTransformCommon objects must be provided with three pieces of data, namely:

the transform project it will be running,

the source adapter(s), and

the target adapter(s)

Providing p roject detai ls

Where the transform project has been generated into Java, the correct interface to use is:

TMTransformExecutor:addProject(String projectName, String projectVersion).

This will cause the loading of the relevant Java classes from the class path during thecompleteInitialisation call.

Where the generated project is provided as XML the interface is:

TMTransformExecutor:addXMLProject(String projectName, String projectVersion,Boolean isErrorHandler,String theXMLFileSpec, java.io.InputStreamtheXMLFileStream, org.w3c.dom.Document theXMLDocument).

You should only provide the XML data in one of the forms. See also setXMLSMLReaderVersion(intxmlLoaderVersion) & setXMLSMLReaderTransform(String xmlLoaderProject) which are used to setup the version of transformer used to read the meta-data.

More than one project may be set into a given run-time interface, TMTransformer orTMTransformExecutor. Note that the first project provided is known to the system as the 'primary

project'. This is treated slightly differently from other projects in both TMTransformer andTMTransform Executor.

Setting Adaptors

Once the project(s) have been set, it is necessary to set source and target adapters for each project.

There is little difference between using built-in adapter types or generic adapter types.

Where the adapters are built-in (XML, RDBMS, Java Bean), they should beconstructed as TMBuiltInAdaptorFactory


27/79


20

Where the adapters are generic (i.e. user implementations of typesTMGenericReaderAdapter or TMGenericWriterAdapter) then the user creates theadapter object according to their own needs

Once adapters are constructed, they are set on the TMTransformerCommon adapter using:

setSource(prjName,Adapter) and setTarget(prjName,Adapter) calls. There should

be no need to cast the adapter to any particular type when using these calls as theyare already suitably overloaded.

You will also need to set the connection parameters for the adapters this is done usingsetInstanceConnectionData for all types of adapter. This takes a Map of keys and values. Thus, typicalcodefor example to set up the built-in read JDBC adaptercould look like:

TMAdapter srcTMAdapter =

TMBuiltInAdaptorFactory.makeNewReadTMJDBCAdapter(aTMTransformer);

Map srcParams = new HashMap();

srcParams.put (TMJDBCAdapter.NAME_USER, "Scott");

srcParams.put (TMJDBCAdapter.NAME_PASSWORD, "Tiger");

srcParams.put (TMJDBCAdapter.NAME_URL, "jdbc:oracle:thin:@LOCALHOST:1521:garmon");

srcParams.put (TMJDBCAdapter.NAME_DRIVER, "oracle.jdbc.driver.OracleDriver");

srcTMAdapter.setInstanceConnectionData(srcParams);

aTMTransformer.setSource (projectName,srcTMAdapter);

Normally, adapters have the same long lifetime as TMTransformer or TMTransformExecutor itself.However, the 'session' mechanism is available should the adapter wish to divide its lifetime into aseries of independent periods. What happens in each period is entirely user-defined so an adaptermay physically connect to different sources and sinks of data in each of these periods.

Sett ing up TMTemp lateAdapters

The built-in flat file adapters implement TMTemplateAdapter. These adapters require additionalinformation to be loaded at runtime. This is done as shown in this example:

if (anAdapter instanceof TMTemplateAdapter)

{

TMTemplateAdapter aTemplateAdapter = (TMTemplateAdapter) anAdapter;

net.etltm.tmr.TMTemplateAdapterHelper.loadTemplate(aTemplateAdapter,

false, projectName, projectMajor, 0);

}

Note that in the example above, the second argument passed into the 'loadTemplate' is 'false'. Thisargument will change depending on whether the adapter in question is used on the source or target side.

Running the Transforms and Getting Results

Once setSource and setTarget have been called, the design patterns for TMTransformer andTMTransformExecutor differ.

Short-Lived Mode

For a TMTransformer the correct pattern is simply openProjectAndResources followed byrunAllTransforms. Note that where the target is Java or XML target, you may need to ask the writeadapter for the results of the transforms (i.e. the mapped data). Where the source is Java or XML targetthen you may need to place data into the read write adapter before runAllTransforms.

Long-Lived Mode

Completing Initialisation

Once all sources and targets have been provided, TMTransformExecutor's completeInitialisationmethod should be called. This loads XML or Java descriptions of the meta-data. If the adapters arenon-generic then the completeInitialisation method opens the primary sources and targets.

Where the adapters are generic, the method setMetadata will be called inside each user written adapter.This provides the user with access to useful meta-data for the internal operation of the adapter. (Note

that information provided depends on the setting up of setRequiresMetaData in the loader interface.)


28/79


21

TMTransformExecutor does not open generic adapters at this point it is the users responsibility toopen the generic adapters used with TMTransformExecutor.

Note completeInitialisation method will throw exceptions if it is called prematurely or any of thesemethods fail. Calling transform methods is not possible until completeInitialisation has completed.

The Main Loop

Once initialisation is complete, the transformer is ready to serve and transform data. The main controlloop that does this is explained in more detail later in this document.

Closing a TMTransformExecutor

The TMTransformExecutor:closeDown method should be called when no further transformations arerequired. This will call the finish methods inside any generic adapters and will close down any of the

built-in ETL adapters.

Note that both TMTransformer and TMTransformExecutor are single threaded processes and shouldnot be called simultaneously from multiple threads. However, there is nothing to stop generic adaptersfrom using separate threads internallyfor example to send data across a network, or simply to queuedata for efficiency.

In addition TMTransformExecutor may know of 'stalled' items that are necessarily waiting for furtheritems in the input pipelines. Thus, if closeDown is called at the 'wrong time', generic adapters maywish to throw a TMException, while TMTransformExecutor may throw a TMException if the pool of'stalled' items is not empty.

The TM Test Too l

The TM Test Tool is simply an interface onto the short-lived TMTransformer. In addition to testingand interactive transform development (in conjunction with the TM Design Tool), it can be used as a

primitive deployment tool and can be run from the command line. It communicates with the run-timesession itself by executing a Java class called 'com.prismt.transform.runtime.exec.RunMaps'. It picksup the required parameters to run from a file called 'Exec.properties' in the current directory. It is

possible to use this mechanism to run transforms without writing any Java code at all.

JNDI PasswordsIf a project contains lookups which require passwords for JNDI database connections, they arespecified as shown in the following example:

try{

net.etltm.helper.DefaultTMConnectionLocatorConfiguration.configure(net.etltm.helper.DefaultTMConnectionLocator.getInstance());

String[] sArgs = {"password.World=" + "fred","password.AnotherConnection=" + "frank"};

//World is name of the connection used in the lookup,//fred is the actual password//multiple connection / password pairs can be set as shown in this

example

net.etltm.helper.TMPasswordHolder.setPasswords(sArgs);}catch (Exception e){// exception handling code goes here

}


29/79


22

TMTransformExecutor - Main Loop

Basic pattern

Because transforms are called on an instance-by-instance basis, it is easy to implement the basic loopdesign pattern:

////either built-in adapter codeTMDHIterator anIt= getObjectsOfInterest(aTMSourceQuery);

////or generic user code -

TMDHIterator anIt= user code returning an iterator of TMReadElement's

while (true)

{

TMReadElement aInputDh = anIt.getNext();

if (null==aInputDh)

{

break;

}

TMWriteElement aOutputDh

= (TMWriteElement )myTMTransformExecutor.transform(aInputDh);

// further user processing such as:

myUserWrittenTarget.consumeHndle(aOutputDh); //consumeHndle is a user writtenfunction

}

This allows for fine control of the transform process. Note that simple transforms will be statelessoutside the call to "transform". However, transforms that are more complex will require that partTMTransformExecutor retains "stalled" handles and also references to fkey/primary key relationships.

Variations

Simplest Pattern

The easiest sort of generic source to target the loop pattern is:

while (true){

TMReadElement aInputDh

= (TMReadElement)anIt.getNext();

if (null==aInputDh)

{

break;

}

myTMTransformExecutor.transform(aInputDh);

}

// then some call to myWriteAdapter.'getData.'

This pattern is probably more common should the target be a non-generic adapter. Here the returnvalue from the each call to 'transform' is simply ignored. We can do this because the write side handle

is already 'included' in the relevant write adapter. For RDBMS systems, this means that data willeventually be flushed to disc, while for other types of adapter some kind of getData method is availableto getJava or SAX events after the loop has finished.

More Comp lex Patterns

However, it is also possible to use the return value from the transform within the group for increasedflexibility. The principal reason you might wish to use these options with the built-in Java or XMLadapters is to allow the transformation process to scale beyond limitations of memory. For databases,on the other hand, this approach is more likely to be adopted because the user wishes a more intimateinvolvement in transaction control and JDBC optimisations. (Note that by default the ETL databasehandler is fully scalable anyway).

Thus, with the built-in Java adapter as target, the loop could read:while (true)

{

TMDH aInputDh = anIt.getNextDH();


30/79


23

if (null == aInputDh)

{

break;

}

TMDH aDH = executor.transform(aInputDh);

// Now calling aDH.getJavaObject() will get us the objects in the user definedclasses.

}

Thus, with the built-in XML adapter as target, the loop could read:while (true)

{


if (null==aInputDh)

{

break;

}

TMDH aDH = (TMDH)myTMTransformExecutor.transform(aInputDh);

}

Thus, with the built-in RDBMS adapter as target, the loop could read:while (true)

{

//maybe rdbms starting transactions/batching etc


if (null==aInputDh)

{

break;

}

TMDH aDH = (TMDH )myTMTransformExecutor.transform(aInputDh);

// rdbms stopping/committing transactions/batching etc

}

Note that there may be many calls to getObjectsOfInterest followed by particular types of processing.

Control l ing w hat is returned by ObjectsOfInterest

The only parameter to getObjectsOfInterest is a TMSourceQuery. This is, potentially, a very complexstructure, which allows the user to perform a very complex set of selections on the source. (Inconjunction with TMQueryPropagator this selection can even be defined in target terms - seeTMQueryPropagator). Simple uses include:

Getting One Domain

Using the static factory class method in TMQueryObjectsFactory to describe a selection of all objectsfrom one domain.TMSourceQuery aTMSourceQuery =

net.etltm.qp.TMQueryObjectsFactory.createSourceQuery(domain));

anIterator=adapter.getObjectsOfInterest(aTMSourceQuery);

Getting Several Domains

Using the static factory class method in TMQueryObjectsFactory to describe a selection of manyobjects (e.g. 'Account', 'Name', 'Transaction') from several domains.Vector v = new Vector();

v.add("Account");

v.add("Name");

v.add("Transaction");

TMSourceQuery aTMSourceQuery =

net.etltm.qp.TMQueryObjectsFactory.createSourceQueryV(v));


Selecting attributes

Limiting the selection to certain attributes on a domain (e.g. 'payer', 'payee' and 'date') on any particularclass (e.g. 'transaction')TMSourceQuery aTMSourceQuery =

net.etltm.qp.TMQueryObjectsFactory.createSourceQuery(transaction));


31/79


24

aTMSourceQuery.addAttribute("payer");

aTMSourceQuery.addAttribute("payee");

aTMSourceQuery.addAttribute ("date")


Getting all domains

This can be done with:anIterator=adapter.getObjectsOfInterest(null);

But note that some adapters cannot implement this call - the JDBC built-in adapter, for example, doesnot implement this call - you need to know what domains (i.e. tables) you are selecting.

Conv enience Classes

There are number of convenience classes defined for this system mostly in the package'net.etltm.helper';

TMDHListIterator

The class TMDHListIterator found in net.etltm.helper is a simple implementation of a TMDHIteratorbased on a Java list. If your underlying architecture allows you to represent your iterated objects in this

fashion then this class provides a convenient way of building the TMDHIterators required by the abovemethods direct from a Java list.

AbstractTMReadAdapter

Extending the AbstractTMReadAdapter is the easiest way of building a generic read adapter. Thisapproach is suitable if you are building an adapter to handle instances of moderate size. Thus, if youcan reasonably expect the adapter to hold all input objects in memory this is the recommended choice.

AbstractTMWriteAdapter

Extending the AbstractTMWriteAdapter is the easiest way of building a generic write adapter. Thisapproach is suitable if you are building an adapter to handle instances of moderate size. Thus, if youcan reasonably expect the adapter to hold all output objects in memory, before serialising them orsending them to another component, this is the recommended choice.

Notes on the Transform Methods

These basic transform calls are very simpleeach takes a source item and returns a target item.

TMTransformExecutor:transform(TMReadElement)

TMTransformExecutor:transform(TMReadElement, targetDomainList, insist)

Both these calls return a DataHandle, i.e. single items of type TMDH (which can be cast toTMWriteElements or your own types as required).

If you have designed a transform so that it directly "unNests" then it may create a target array ratherthan a single TMDH. In this case, the form to use is:

TMTransformExecutor:transformM(TMReadElement).

This will, potentially, return a list of TMDHs. If you use the transform method and your transformdoes "unNest" then no exception will be thrown, but you will only ever get the first element created.

Note the single parameter call runs any defined transform from TMReadElement's source domain. Ifthere are two or more independent transforms from the same source domain (a design pattern that is notencouraged, but not excluded) then it will simply choose one of them randomly. (Future versions ofthe product may support mode and priority to help make this process more determinable).

The second method resolves this ambiguity and attempts to run the defined transform fromTMReadElement's source domain to the specified list of target domains. The names inTargetDomainList should be in priority order. If "insist" is true, then this is the only operation it willattempt - if there were no such defined transform it will simply throw an error. If "insist" is false and ifit fails to find any successful transforms to the domains in the target Domain it will continue and

perform any transform from TMReadElement's source domain.

If targetDomainList is null then insist is ignored and the behaviour is as the single parame ter call.


32/79


25

Using TM adapters outside TM

All TM adapters, including the built-in adapters, can be used outside TM itself and within your ownsoftware architecture. They are simply re-usable components.

Sessions Div is ions and Part it ions

TMTransformExecutor is a long-lived component. It is possible to open and close adapters owned by aTMTransformExecutor during its lifetime. In addition, the lifetime of each open adapter may bedivided into sets of what we call 'sessions'. These divide its lifetime into a series of open sessionsduring which "blocks" of data are being transformed. There is no limit to the depth of this sub-division.

The precise activity performed by each user adapter as each session (of any particular level) begins andends is entirely user-defined. Typical 'beginSession' actions could include 'connecting' to a new datasource, or beginning some kind of transaction, or doing nothing at all. Typical 'endSession' activitiesmight be 'disconnecting' or ending a transaction. The only requirement is that sessions should be

properly nested i.e. that each 'beginSession' is followed by an 'endSession' to end normally oranother 'beginSession'. Calling a second 'beginSession' at the same level as an already open Session isused to force the creation of a new session and the loss of the previous session at that level. Thus, there

is no need for an abort Session operation. However, these semantics cannot be enforced by TM exceptfor its own concept of 'partitions' and 'divisions' (see below). Note finally that TM's session control isnot restrictivegeneric adapters may implement any of these operations outside the interfaces required

by TM.

Once these user operations have been defined, all TMTransformExecutor's implementation of'beginSession' and 'endSession' does is to call through to 'beginSession' and 'endSession' on each of itscurrent open adapters in turn. The adapters are ordered in order of adapter opening.

While 'endSession' takes no parameters, other than the level of session it is ending, 'beginSession' alsotakes a java.util.Map. This map is keyed into whatever string data is required to "begin a session" atthat level - again this is user-defined. Where a generic adapter has been implemented to co-operate withTMs GUI interfaces, then it is possible, using the call 'getBeginSessionNames' to discover the keysrequired for any particular adaptersee the interface TMGenericInstance for further details.

Finally, note that 'beginSession' returns a Boolean. This should be false if no action is taken, true if anyaction is taken. This allows the Transformer itself finally to return a result for its 'beginSession'operation, which will only be 'true' if any action has been taken by any adapter.

"Partitions" and "Divisions"

TM itself may be set to respond to two levels of session. These are termed "partition" and "division".These are not hard coded to any particular level of session. For flexibility, TMTransformExecutorcontains methods to set and get the levels at which 'beginSession' and 'endSession' are to be interpretedinternally as instructions to begin or end 'partitions' or 'divisions'. For example, the callsetRecognisedDivisionLevel(3) tells TMTransformExecutor to interpret beginSession(3) andendSession(3) as the beginning and ending a TMTransformExecutor "division."

Note that 'beginSession' and 'endSession' events are still also called on all open adapters even if theyare interpreted by TMTransformExecutor. In the case of 'beginSession', place any special processingwithin TMTransformExecutor after 'beginSession' has been called on the other adapters. In the case ofendSession, TMTransformExecutor's own processing takes place before that in the adapters.

Transformation Architecture with Partitions and Division

In order to understand what 'divisions' and 'partitions' actually do it is necessary to have someunderstanding of the internal architecture of the transformation engine at least when it is usinggeneric adapters and configured in long-life mode. When internal adapters are in use, other caching

patterns and optimisations take place.

In long-life mode, every call to TMTransformExecutor's transform method produces zero, one or anumber of data handles. Before returning to control to the user TMTransformExecutor will attempt to

"flush" these handles out to the user's generic implementation by calling the finalize method. However,for reasons of efficiency it will only flush out non-identified handles - handles that have identification


33/79


26

are retained internally. The lifetime of this internal retention is the lifetime of a partition. Thus,ending a partition will have the effect of flushing identifyinghandles through the user layer. Thus, byending, or by not ending, partitions, the user may control how often identifying handles are flushed outto the user's implementation of the generic adapter. Note that all handles are always flushed out whenan adapter closes. Thus, 'partitions' are optional.

The concept of the 'division' however, is quite different. One of the advanced features of TM is itstreatment of batched data. TM understands that batched data may not necessarily arrive in the rightorder, or may refer to other pieces of data in other batches. To do this, it holds lightweight references

between batches. TM chooses to retain this data based on its analysis of value-based relationships.

The lifetime of this lightweight data is known as a "division". Ending a division thus removes frommemory any lightweight references that may be used between batches. Note that divisions are onlymeaningful if TMTransformExecutor::setSupportsCrossSessionHandles has been called. This callsimply turns on TM's ability to remember references. Otherwise, such references are neverremembered.

Stalled Handles

When TM is holding forward references, some of its data handles may be placed into a state known as"stalled." These stalled handles cannot be written out through the user layer because they containunresolved mandatory references. In relational database terms, these are best thought of as lacking datafor mandatory foreign key columns. Such mandatory references may, of course, arrive in latertransforms - if they do, then the stalled handle status is cleared and the handle itself automaticallywritten out as appropriate. However, stalled handles may persist until the adapter closes, at which

point their continued existence may prove a problem. Thus, three methods exist to help handle the poolof stalled handles.

The method getStalledHandleDescriptions returns descriptions of those handles that are stalled -including the domains, and the reason for their stalled status. The methodTMTransformExecutor:destroyStalledHandles may be used to remove these from memory at any time -the data in the handles is simply discarded. The method clearStalledHandleFlags may be used toremove the stalled status of these handles - the next call to end a partition will then attempt to flushthem through to the user layer (this may result in faults at the user layer level as it attempts to deal with

unresolved mandatory references).

If an adapter is closed and stalled handles are present, all normal handles are first flushed through to theuser layer - then an exception will be thrown to describe what is stalled.

Built-in Adapters and Sessions

Note that the built-in adapters for databases Java and XML currently take no special action on thebeginning or ending of sessions at any level. This will, however, change.

Clearing a Project

The method removeXMLProject(String projectName, String projectVersion) may be used to remove aproject from the TMTransformExecutor. Once cleared another set of transforms may be loaded. There

is no corresponding equivalent for Java generated code.

Transform s from $Docum ent, $PreDocument and

$PostDocument

Outside XML, these transforms principally exist in order to allow the SML code to perform systemfunctions directly. These are only automatically run in the short-lived TMTransformer - but can beexplicitly run in a TMTransformExecutor (.e.g. using the methods ' runPostDocumentTransform','runPreDocumentTransform' and ' runDocumentTransform'). In the short-lived TMTransformer, thecalls are run as follows:

$PreDocument transforms are always called before built-in adapters are physicallyconnected. For the generic adapters this means just before the call back to

'setMetaVerificationOn'.


34/79


27

$Document transforms are called just after the call to 'setMetadata'.

$PostDocument transforms are called after just after the call to 'finish'.

Note that generic adapters have no access to any type of $Document object - this object is internallycreated and destroyed entirely within the TMTransformCommon.

Running the Samples

TM is supplied complete with a set of samples in the directory

"samples\\deployment\\examples". (This is a relative path from the TM install directory - thus

if you have chosen to install TM in "X:\\Program Files\\TM" then the complete path is

"X:\\Program Files\\TM\\samples\\deployment\\examples").

To run the samples you will need to follow the following steps:

1) Open TM Design Tool and build each of the projects in turn. They have already been set to

generate the jar code to "samples\\deployment\\lib". When generation has finished the directory

"samples\\deployment\\lib" should contain the files:

1. ETL_BasicXMLErcHandler_1_0.jar

2. ETL_GenericToJava_1_0.jar

3. ETL_JavaToGeneric_1_0.jar

4. ETL_JavatoJavaEmbededXML_2_0.jar

5. ETL_JavaToJavaWithError_2_0.jar

6. ETL_JavaToJava_2_0.jar

7. ETL_JavaToJDBC_2_0.jar

8. ETL_JavaToXML_2_0.jar

9. ETL_JDBCToJavaFilter_1_0.jar

10.ETL_JDBCToJava_2_0.jar

11.ETL_LLJavatoJavaEmbededXML_1_0.jar

12.ETL_XMLToJava_3_0.jar

13.ETL_XMLtoXML_1_0.jar

2) You will also need to build the supplied Java code. The directory

"samples\\deployment\\examples" contains the actual Java samples themselves - the main filehere is "ExamplesInJava". This contains a number of very simple utility procedures for displaying theresults of the transforms, and also the static strings defining the location of the various sources andtargets used in the examples. Note that for reasons of simplicity, these read and write only from thesupplied example MS Access data bases using JDBC/ODBC, not from Oracle or Sybase (as isdescribed in the text). There is, of course, no need to have MS Access actually installed in yoursystem.

The directories tgt and src contain the actual Java instance models used in the examples. All the Javacode must be built for the system to run. This may vary from system to system. Using JBuilder,(recommended) the steps you need to perform are:

Create a new JBuilder project for your examples:

1. Set the source path to something like "X:\\TM\\samples\\deployment\\examples"

2. Add the packages "examples" and "examples.tgt" and "examples.src" to your product.

3. Ensure that the libraries include both ofetsr.jar and the generated jar files listed above.

4. Generate and run the Java code. Running in the JBuilder debugger is highly recommended asit allows you to focus on items of interest.


35/79


28

Short-Lived Deployment Examples

The code included in this section is taken from the standard examples distributed with the product.Note that ALL possible combinations of source and target are legal it is only for conciseness that thisdocument limits the examples to the 16 described here. Thus, it is quite legal to transform betweenXML and XML or between database and database.

Simple Example

File Setup

The example code is built up in a single Java file.

Step 1 File Creation

We start with the outline of a java file containing the main building blocks of any application. Beginby defining the package and class for your deployed code.

package shortlived;

public class ShortLived

{

public ShortLived()

{

}

}

Step 2 - Main

As the entire example will be self-contained in a single file we also add a main method in order to runthe application. The main method below contains a single line to create an instance of the ShortLivedobject.

package shortlived;


{

static public void main(String[] args)

{

ShortLived shortLived = new ShortLived();

}

public ShortLived()

{

}

}

Now these steps are complete, the actual methods to execute the transforms can be considered.

Running the Transform at ion System

As running transformations in Short Lived mode is straightforward, we will only require a singlemethod that will run the transforms.

For the purpose of this example, the source will be taken from an XML file, with the data also beingwritten to a target XML file.

Step 3 Initialise

Add the outline of the method runTransform.


36/79


29

package shortlived;


{


{

ShortLived aShortLived = new ShortLived();

aShortLived.runTransform();

}

public ShortLived()

{

}

public void runTransform()

{

System.out.println("ETL solutions Short Lived XML to XML Deployment Example");

}

}

Step 4 New Transformer

The first step is to create a new Sho rt-lived TMTransformer object. This is the main object used tocarry out the transformations that have been created using the TM Design Tool.

package shortlived;

import net.etltm.*;


{


{


aShortLived.runTransform();}

public ShortLived()

{

}


{


try

{

//Set the licence file location

TMTransformerFactory.setTMLicenseLocation("d:\\mydeployment\\licence\\etltm.lic");

//Short-lived

TMTransformer aTMTransformer =

TMTransformerFactory.getNewTMTransformer(this,"shortLivedProj",1);

}

catch (TMException aTMException)

{

aTMException.printStackTrace();

return;

}

}

}

1. Add the import line at the top of the file. This allows the use of the TMTransformerobject.


37/79


30

2. Add a call to set the licence file. This must be made on theTMTransformerFactory before the factory is used to create a TMTransformer.

3. A Try-Catch block is required around the creation of the TMTransformer.

4. The static factory method used to create the TMTransformer is described in detailwithin the API documentation found within the installation directory \Program

Files\TM\docs\api-docs\index.html.

The 3 parameters passed in are:

1. userRefObject This can be any java object. It may be retrieved later in any of thejava functions defined in the SML project.

2. Project Name The name of the project for the transformations.

3. The Version The major version of the project for the transformations.

Step 5 Source Adapter

The next step is to set up the source adapter. The source adapter is used to access the source instancedata for transformation.

package shortlived;

import net.etltm.*;


{


{



}

public ShortLived(){

}


{


try

{

//Short-lived



//Create SOURCE AdapterTMXMLAdapter aXMLSourceAdapter =

TMBuiltInAdaptorFactory.makeNewReadTMXMLAdapter(aTMTransformer);

aTMTransformer.setSource("shortLivedProj",aXMLSourceAdapter);

}


{


return;

}

}

}

1. The source XML adapter is created in a single line directly from TMs own adapterfactory by simply calling makeNewReadTMXMLAdapter from the


38/79


31

TMBuiltInAdaptorFactory class. Further detail can be found in the APIdocumentation.

2. The second line entered sets the source adapter for the TMTransformer object createdin Step 4. Hence, the TMTransformer will be able to use the TMXMLAdapter whenaccessing the source data.

Step 6

Target Adapter

The target adapter is created in a similar fashion to the source adapter.

1. The target XML adapter is created in a single line directly from TMs own adapterfactory by simply calling makeNewReadTMXMLAdapter from theTMBuiltInAdaptorFactory class. Further detail can be found in the APIdocumentation.

2. The second line entered sets the target adapter for the TMTransformer object createdin Step 4. Hence, the TMTransformer will be able to use the TMXMLAdapter whenwriting the target data.

package shortlived;

import net.etltm.*;


{


{



}

public ShortLived()

{}


{


try

{

//Short-lived



//Create SOURCE Adapter

TMXMLAdapter aXMLSourceAdapter =



//Create TARGET Adapter

TMXMLAdapter aXMLTargetAdapter =

TMBuiltInAdaptorFactory.makeNewWriteTMXMLAdapter(aTMTransformer);

aTMTransformer.setTarget("shortLivedProj",aXMLTargetAdapter);

}


{


return;

}}

}


39/79


32

Step 7 Setting the Source XML File

The xml file to be used as the source file can now be specified for the source TMXMLAdapter.

package shortlived;

import net.etltm.*;import java.util.*;


{


{



}

public ShortLived()

{

}


{


try

{

//Short-lived











//Set the XML file the source adapter will read in

Map param = new HashMap ();

param.put(TMXMLAdapter.NAME_URL, "d:\\xml\\source.xml");

aXMLSourceAdapter.setInstanceConnectionData(param);

}


{aTMException.printStackTrace();

return;

}

}

}

1. Add a new import line at the top of the file in order to use hash maps in the form ofjava.util.

2. Create a new HashMap which will be used to pass parameters into the sourceTMXMLAdapter.

3. Add the location and file name of the source xml file to the parameter hashMap,using the pre-defined static key TMXMLAdapter.NAME_URL for identification.


40/79


33

4. Use the setInstanceConnectionData method on the source adapter, in order to add the parameter.

Step 8 Setting the Target XML File

The setting of the target XML file is done in a similar fashion to the source file described above.

package shortlived;

import net.etltm.*;

import java.util.*;


{


{



}

public ShortLived()

{}


{


try

{

//Short-lived




TMXMLAdapter aXMLSourceAdapter =TMBuiltInAdaptorFactory.makeNewReadTMXMLAdapter(aTMTransformer);










//Set the XML file the target adapter will write to

param = new HashMap ();

param.put(TMXMLAdapter.NAME_URL, "d:\\xml\\target.xml");

aXMLTargetAdapter.setInstanceConnectionData(param);

}


{


return;

}

}

}

1. Create a new HashMap which will be used to pass parameters into the target

TMXMLAdapter.


41/79


34

2. Add the location and file name of the target xml file to the parameter hashMap, usingthe pre-defined static key TMXMLAdapter.NAME_URL for identification.

3. Use the setInstanceConnectionData method on the target adapter, in order to addthe parameter.

Step 9 Initialise the Data in TMTransformer

Now the Source and Target Adapters have been created and the source and target xml files have beenspecified to the respective adapters, the TMTransformer object can open the required information.

package shortlived;

import net.etltm.*;

import java.util.*;


{


{



}

public ShortLived()

{

}


{


try

{

//Short-livedTMTransformer aTMTransformer =










//Set the XML file the source adapter will read inMap param = new HashMap ();







//Open the project information along with the source

//and target adapter information

aTMTransformer.openProjectAndResources();

}


{


42/79


35


return;

}

}

}

This is simply done in a single line that opens the transformation project information and the sourceand target file information.

Step 10 Run the Transforms

The final line to be added is the line that executes the transformations taking the data from the sourceand writing it to the target following the business logic defined using the TM Design Tool.

package shortlived;

import net.etltm.*;

import java.util.*;

public class ShortLived{


{



}

public ShortLived()

{

}


{


try

{

//Short-lived



















//Open the project information along with the source

//and target adapter information


43/79


36


// run the transform

aTMTransformer.runAllTransforms();

}


{


return;

}

}

}

Execut ion

Step 11 Running the Program

In order to run the java file created using the above steps, ensure you have the following in yourclasspath.

1. ofetsr.jar (Which can be found in the installed Program Files \TM\lib directory)

2. ETL_shortLivedProj_1_0.jar (The jar file created from your TM Design Toolmappings, located by default inProgram Files\TM\com\prismt\transform\maps\shortLivedProj\V1\m0)

Changes for Database Connect ions

If the source, target or both were databases, the following changes would be required.

In the following example, we will describe the changes to the target side connection. The changesrequired to the source side would be similar.

Adapter Changes

Where currently the adapter is created as :TMXMLAdapter aXMLTargetAdapter =


For a database connection simply changing the method to refer to a JDBC adapter as shown below willallow this :TMJDBCAdapter aTMJDBCAdapter =

TMBuiltInAdaptorFactory.makeNewWriteTMJDBCAdapter(myTMTransformer);

Connection Changes

Currently connection to the target side information is done simply by specifying the xml file fortransformation as shown below:





For a database connection simply specify the USER, PASSWORD, URL and DRIVER as required,using the pre-defined static keys available with the TMJDBCAdapter.

Map params = new HashMap();

params.put(TMJDBCAdapter.NAME_USER,"root");

params.put(TMJDBCAdapter.NAME_PASSWORD,"root");

params.put(TMJDBCAdapter.NAME_URL,"jdbc:odbc:mySQLTest");

params.put(TMJDBCAdapter.NAME_DRIVER,"sun.jdbc.odbc.JdbcOdbcDriver");


44/79


37

aTMJDBCAdapter.setInstanceConnectionData(params);

Java to Java Example

Notes

This simply transforms a populated array of Java objects (of type "Player") found in the Game objectinto an array of "Person" objects. To run, the classpath must include:

TM Jars and dependencies (see the Classpath section in this document)

the examples.src path (i.e. the directory in which the path examples.src.*class may befound).

the examples.tgt path (i.e. the directory in which the path examples.tgt.*class may befound).

Code

Try

{TMTransformer aTMTransformer =

TMTransformerFactory.getNewTMTransformer(this, projectName, 2);

TMJavaAdapter aSrcTMJavaAdapter =TMBuiltInAdaptorFactory.makeNewReadTMJavaAdapter(aTMTransformer);

aTMTransformer.setSource(projectName,aSrcTMJavaAdapter);

TMJavaAdapter aTgtTMJavaAdapter =TMBuiltInAdaptorFactory.makeNewWriteTMJavaAdapter(aTMTransformer);

aTMTransformer.setTarget(projectName,aTgtTMJavaAdapter);


// add some data to the read adapter - this can be done before

// this point if required

// here we could use aReadAdapter.addObject(aGame) but then the

// only objects known to the read adapter by name are the Game

// objects.

// if you want to transform from Person then we need to tell about

// the PERSON objects

// we could add them one a time using addObject or better still add

// the whole array using addObjects

aSrcTMJavaAdapter.addObjects(aGame.getPlayers());

aTMTransformer.setTxtLogging(System.out, TMC.eLogNONE);

// run the transfrom and get the result object


// the file xmltarget.xml will have been created

SimpleUtilities.showJavaObjects(aTgtTMJavaAdapter, "Player");

}catch (TMException aTMException)

{

System.out.println(aTMException.toString());

return;

}

Java to XML Example ( as serialised file)

This example of Java to XML (as a serialised file) simply transforms a populated array of Java objects(of type "Player") found in the Game object into an XML file. To run, the classpath must include:


the examples.src path (i.e. the directory in which the path examples.src.*classmay be found.)


45/79


38

In this example, the file "xmltarget.xml" will be created in the current directory.

Code

TMXMLAdapter aTMXMLAdapter = null;

try

{TMTransformer aTMTransformer = TMTransformerFactory.getNewTMTransformer(this,

projectName, 2);

TMJavaAdapter aTMJavaAdapter =TMBuiltInAdaptorFactory.makeNewReadTMJavaAdapter(aTMTransformer);

aTMTransformer.setSource(projectName,aTMJavaAdapter);

aTMXMLAdapter=TMBuiltInAdaptorFactory.makeNewWriteTMXMLAdapter(aTMTransformer);

aTMTransformer.setTarget(projectName,aTMXMLAdapter);

Map params = new HashMap();

params.put(TMXMLAdapter.NAME_URL, AllExamplesInJava.EXAMPLE_TGT_PATH +"xmltarget.xml");

aTMXMLAdapter.setInstanceConnectionData(params);


// add some data to the read adapter

aTMJavaAdapter.addObjects(aGame.getPlayers());aTMTransformer.setTxtLogging(System.out, TMC.eLogNONE);


// the file xmltarget.xml will have been created

}


{


return;

}

Java to XML Example (as DOM in memory)

This example simply transforms a populated array of Java objects (of type "Player") found in the Gameobject into an org.w3c.dom.Document object (or DOM). To run, the classpath must include:


the examples.src path (i.e. the directory in which the source classes may be found.)

The transform returns an org.w3c.dom.Document object.

Code

TMTMXMLAdapter aTMXMLAdapter = null;

Document aDocument = null;

try

{

TMTransformer aTMTransformer = TMTransformerFactory.getNewTMTransformer(this,projectName, 2);



// Here we DON'T specify a file to write to as we are going to ask

// for the DOM output




// add some data to the read adapter

aTMJavaAdapter.addObjects(aGame.getPlayers());

aTMTransformer.setTxtLogging(System.out, TMC.eLogNONE);aTMTransformer.runAllTransforms();

aDocument = aTMXMLAdapter.getXMLDocument();


46/79


39

}


{


return;

}

Java to XML Example (as serialised memory output)

This example transforms a populated an array of Java objects (of type "Player") found in the Gameobject into a serialised version of the XML text. To run, the classpath must include:


the examples.src path (i.e. to the directory in which the path examples.src.*class may befound.)

The output returns as an OutputStream.

Code

TMXMLAdapter aTMXMLAdapter = null;

try

{




// but we DON'T specify a file to write to as we are going to ask

// for the serializeStream output



aTMTransformer.openProjectAndResources();// add some data to the read adapter

aTMJavaAdapter.addObjects(aGame.getPlayers());

aTMTransformer.setTxtLogging(System.out, TMC.eLogNONE);


aTMXMLAdapter.serializeStream(System.out);

}


XML (as file) to Java Example

This XML (as serialised file) to Java example simply transforms an XML file into an array of "Person"objects. To run, the classpath must include:

TM Jars and dependencies (see the Classpath section in this document)the examples.tgt path (i.e. to the directory in which the path examples tgt.*classmay be found.)

Code

TMJavaAdapter aTMJavaAdapter = null;

try

{


TMJDBCAdapteraTMJDBCAdapter=TMBuiltInAdaptorFactory.makeNewReadTMJDBCAdapter(aT

Documents

TM Deployment Guide