Lookup and Lookup Caches

8/7/2019 Lookup and Lookup Caches

http://slidepdf.com/reader/full/lookup-and-lookup-caches 1/17

LOOK UP TRANSFORMATIONS &

LOOK UP CACHES

Basics and Advanced Concepts



USE OF LOOKUP TRANSFORMATION

Get a related value. For example, your source includes employee ID, but you

want to include the employee name in your target table to make your summary

data easier to read.

Perform a calculation. Many normalized tables include values used in a

calculation, such as gross sales per invoice or sales tax, but not the calculated

value (such as net sales).

Update slowly changing dimension tables. You can use a Lookup transformation

to determine whether rows already exist in the target.



CONNECTED OR UNCONNECTED LOOK UP TRANSFORMATION

You can configure the Lookup transformation to perform the following types of lookups:

Connected or unconnected. Connected and unconnected transformationsreceive input and send output in different ways.

Relational or flat file lookup.

Cached or uncached. Sometimes you can improve session performance bycaching the lookup table. If you cache the lookup, you can choose to use adynamic or static cache. By default, the lookup cache remains static and doesnot change during the session. With a dynamic cache, the PowerCenter Server inserts or updates rows in the cache during the session. When you

cache the target table as the lookup, you can look up values in the target andinsert them if they do not exist, or update them if they do.



SETTING THE UPDATE STRATEGY

Relational Lookups

When you create a Lookup transformation using a relational table as a lookup source, you canconnect to the lookup source using ODBC and import the table definition as the structure for theLookup transformation.

You can use the following options with relational lookups only:

You can override the default SQL statement if you want to add a WHERE clause or query

multiple tables. You can use a dynamic lookup cache with relational lookups.

Flat File Lookups

When you use a flat file for a lookup source, you can use any flat file definition in the repository,or you can import it. When you import a flat file lookup source, the Designer invokes the Flat FileWizard.

You can use the following options with flat file lookups only:

You can use indirect files as lookup sources by specifying a file list as the lookup file name.

You can use sorted input for the lookup.

You can sort null data high or low. With relational lookups, this is based on the database support.



Lookup Components

Define the following components when you configure a Lookup transformationin a mapping:

Lookup source

Ports

Properties

Condition

Metadata extensions



Guidelines to Overriding the Lookup Query

Use the following guidelines when you override the lookup SQL query: You can only override the lookup SQL query for relational lookups.

Configure the Lookup transformation for caching. If you do not enable caching, the PowerCenter Server does not recognize the override.

Generate the default query, and then configure the override. This helps ensure that all thelookup/output ports are included in the query. If you add or subtract ports from the SELECTstatement, the session fails.

Use a Filter transformation before a Lookup transformation using a dynamic cache when you add

a WHERE clause to the lookup SQL override. This ensures the PowerCenter Server only insertsrows in the dynamic cache and target table that match the WHERE clause. For more information,see Using the WHERE Clause with a Dynamic Cache.

If you want to share the cache, use the same lookup SQL override for each Lookuptransformation.

If you override the ORDER BY statement, the session fails if the ORDER BY statement does notcontain the condition ports in the same order they appear in the Lookup condition or if you do notsuppress the generated ORDER BY statement with the comment notation.

If the table name or any column name in the lookup query contains a reserved word, you mustenclose all reserved words in quotes.



Unconnected Lookup Transformations

An unconnected Lookup transformation is separate from the pipeline in the mapping.You write an expression using the :LKP reference qualifier to call the lookup within

another transformation. Some common uses for unconnected lookups include:

Testing the results of a lookup in an expression

Filtering rows based on the lookup results

Marking rows for update based on the result of a lookup, such as updating slowly changing

dimension tables Calling the same lookup multiple times in one mapping

Complete the following steps when you configure an unconnected Lookup

transformation:

Add input ports.

Add the lookup condition.

Designate a return value. Call the lookup from another transformation.





V

erify that you output the same values to the target that the PowerCenter Server writes to the lookup cache.When you choose to output new values on update, only connect lookup/output ports to the target table insteadof input/output ports. When you choose to output old values on update, add an Expression transformation after the Lookup transformation and before the Router transformation. Add output ports in the Expressiontransformation for each port in the target table and create expressions to ensure you do not output null inputvalues to the target.

When you use a lookup SQL override, make sure you map the correct columns to the appropriate targets for lookup.

When you add a WHERE clause to the lookup SQL override, use a Filter transformation before the Lookuptransformation. This ensures the PowerCenter Server only inserts rows in the dynamic cache and target table

that match the WHERE clause. For details, see Using the WHERE Clause with a Dynamic Cache. When you configure a reusable Lookup transformation to use a dynamic cache, you cannot edit the condition or

disable the Dynamic Lookup Cache property in a mapping.

Use Update Strategy transformations after the Lookup transformation to flag the rows for insert or update for thetarget.

Use an Update Strategy transformation before the Lookup transformation to define some or all rows as update if you want to use the Update Else Insert property in the Lookup transformation.

Set the row type to Data Driven in the session properties.

Select Insert and Update as Update for the target table options in the session properties.



Lookup Caches

Persistent cache. You can save the lookup cache files and reuse them the next time thePowerCenter Server processes a Lookup transformation configured to use the cache.

Recache from source. If the persistent cache is not synchronized with the lookup table, you

can configure the Lookup transformation to rebuild the lookup cache.

Static cache. You can configure a static, or read-only, cache for any lookup source. By

default, the PowerCenter Server creates a static cache. It caches the lookup file or table and

looks up values in the cache for each row that comes into the transformation. When the

lookup condition is true, the PowerCenter Server returns a value from the lookup cache. The

PowerCenter Server does not update the cache while it processes the Lookup

transformation.

Dynamic cache. If you want to cache the target table and insert new rows or update existing

rows in the cache and the target, you can create a Lookup transformation to use a dynamic

cache. The PowerCenter Server dynamically inserts or updates data in the lookup cache and

passes data to the target table. You cannot use a dynamic cache with a flat file lookup. Shared cache. You can share the lookup cache between multiple transformations. You can

share an unnamed cache between transformations in the same mapping. You can share a

named cache between transformations in the same or different mappings.



Using a Persistent Lookup Cache

� Using a Non-Persistent Cache By default, the PowerCenter Server uses a non-persistent cache when you enable

caching in a Lookup transformation. The PowerCenter Server deletes the cache files at

the end of a session. The next time you run the session, the PowerCenter Server

builds the memory cache from the database.� Using a Persistent Cache

If you want to save and reuse the cache files, you can configure the transformation touse a persistent cache. Use a persistent cache when you know the lookup table does

not change between session runs.

When you use a persistent lookup cache, you can specify a name for the cache files.

When you specify a named cache, you can share the lookup cache across sessions.

For more information about the Cache File Name Prefix property, see Lookup

Properties. For more information about sharing lookup caches



Working with a Dynamic Lookup Cache

When the PowerCenter Server reads a row from the source, it updates thelookup cache by performing one of the following actions:

Inserts the row into the cache. The row is not in the cache and you specified to

insert rows into the cache. You can configure the transformation to insert rows into

the cache based on input ports or generated sequence IDs. The PowerCenter

Server flags the row as insert.

Updates the row in the cache. The row exists in the cache and you specified to

update rows in the cache. The PowerCenter Server flags the row as update. The

PowerCenter Server updates the row in the cache based on the input ports.

Makes no change to the cache. The row exists in the cache and you specified to

insert new rows only. Or, the row is not in the cache and you specified to update

existing rows only. Or, the row is in the cache, but based on the lookup condition,

nothing changes. The PowerCenter Server flags the row as unchanged.



Lookup Transformation Tips

The following list describes some situations when you can use a dynamiclookup cache:

Updating a master customer table with new and updated customer information.

You want to load new and updated customer information into a master customer

table. Use a Lookup transformation that performs a lookup on the target table to

determine if a customer exists or not. Use a dynamic lookup cache that inserts

and updates rows in the cache as it passes rows to the target.

Loading data into a slowly changing dimension table and a fact table. You want to

load data into a slowly changing dimension table and a fact table. Create two

pipelines and use a Lookup transformation that performs a lookup on the

dimension table. Use a dynamic lookup cache to load data to the dimension table.

Use a static lookup cache to load data to the fact table, making sure you specify

the name of the dynamic cache from the first pipeline.



A Lookup transformation using a dynamic cache has the following properties:

NewLookupRow. The Designer adds this port to a Lookup transformation configured to use adynamic cache. Indicates with a numeric value whether the PowerCenter Server inserts or updates the row in the cache, or makes no change to the cache. To keep the lookup cacheand the target table synchronized, you pass rows to the target when the NewLookupRowvalue is equal to 1 or 2.

Associated Port. Associate lookup ports with either an input/output port or a sequence ID.The PowerCenter Server uses the data in the associated ports to insert or update rows in thelookup cache. If you associate a sequence ID, the PowerCenter Server generates a primarykey for inserted rows in the lookup cache.

Ignore Null Inputs for Updates. The Designer activates this port property for lookup/outputports when you configure the Lookup transformation to use a dynamic cache. Select thisproperty when you do not want the PowerCenter Server to update the column in the cachewhen the data in this column contains a null value.

Ignore in Comparison. The Designer activates this port property for lookup/output ports notused in the lookup condition when you configure the Lookup transformation to use a dynamiccache. The PowerCenter Server compares the values in all lookup ports with the values intheir associated input ports by default. Select this property if you want the PowerCenter

Server to ignore the port when it compares values before updating a row.



Using Update Strategy Transformations with a Dynamic Cache

When you use a dynamic lookup cache, use Update Strategy transformations to definethe row type for the following rows:

Rows entering the Lookup transformation. By default, the row type of all rows enteringa Lookup transformation is insert. However, you can use an Update Strategytransformation before a Lookup transformation to define all rows as update, or some asupdate and some as insert.

Rows leaving the Lookup transformation. The NewLookupRow value indicates how thePowerCenter Server changed the lookup cache, but it does not change the row type.Use a Filter or Router transformation after the Lookup transformation to direct rowsleaving the Lookup transformation based on the NewLookupRow value. Use UpdateStrategy transformations after the Filter or Router transformation to flag rows for insertor update before the target definition in the mapping.

Note: If you want to drop the unchanged rows, do not connect rows from the Filter or Router transformation



� Rules and Guidelines for Dynamic Caches

Use the following guidelines when you use a dynamic lookup cache:

The Lookup transformation must be a connected transformation.

You can use a persistent or a non-persistent cache.

If the dynamic cache is not persistent, the PowerCenter Server always rebuilds thecache from the database, even if you do not enable Recache from Lookup Source.

You cannot share the cache between a dynamic Lookup transformation and static

Lookup transformation in the same target load order group. You can only create an equality lookup condition. You cannot look up a range of data.

Associate each lookup port (that is not in the lookup condition) with an input port or asequence ID.

Use a Router transformation to pass rows to the cached target when theNewLookupRow value equals one or two. You can use the Router transformation todrop rows when the NewLookupRow value equals zero, or you can output those rows

to a different target.



THANK YOU

Documents

Lookup and Lookup Caches