Usefull Stuff Datastage

8/10/2019 Usefull Stuff Datastage

1/23

Q1.Hi,

I have numeric values embedded with string values for example:

tokyo(2!",sfo(#$%" etc I want only numeric values from the strings.

&or tokyo(2!" output should be 2! &or sfo(#$%" output should be #$% do we have

any function or a method in 'atatage #.! parallel )ob developer to achieve this*

SOL: Hi +a)ish, If all your data have the same format with what you give below, like

all the numbers are being placed inside ( and ", then you can use &I-' function.

'ata:

tokyo(2!"

sfo(#$%"

&I-'(ol/ame, 0(0, 2" 1 the result would be:

2!"

#$%"

to remove the ", we can use the convert function, so the nal code would

be:

3/4+5(0"0,00,&I-'(ol/ame, 0(0, 2""

3utput:

Hi +a)ish,

If all your data have the same format with what you give below, like all the

numbers are being placed inside ( and ", then you can use &I-' function.

'ata: tokyo(2!" sfo(#$%" &I-'(ol/ame, 0(0, 2" 1 the result would be:

2! #$%

6lease note that this is only applicable if all your data has the same

format with what you provided as sample data. 7ll numbers are enclosed in ("

and after the alphabet characters


2/23

2SOL)8se nested onvert(" functions9 the inner one returns the original string with

numeric characters removed, the outer one removes all those from the original

string.

onvert(onvert(%!2;#$, , In-ink.5hetring", , In-ink.5hetring"

2Q)I had a le which is split into # parts due to si?e constraints. o I want to read all

the # parts into a single into a single tables by using a single datastage )ob. an

any one please assist me*

1SOL" Hi 5o read all les into single le dene following path in se@le.

le!,le2,leA.)txt

Hi, olution for this problem is

5o read all # les into a single )ob 5ake one &lat le and mention &ile name like 5his

below

x: &ile is split into

&ile!,&ile2,&ile,&ile;,&ile,&ile#

/ow give it like this in le /ame: &ile*.txt It reads all # les in a single )ob.

2SOL)Bou put your # dataset lesCse@uences les in one datastage )ob and then

you passe all # output link into a funnel and from funnel you link out only one linkout which contains all your # les

3SOL)3r use a single se@uential le with &ile name in 6attern mode (all parts have

to follow a pattern in their names"

3Q)Hi

I have a re@uirement where I have to fetch few les from a folder of similar pattern

and having same metadata and then need to apply some transformations on it and

then I have to produce same no of les fetched from the source folder and place

them in target folder.

x: dir! has 2!% les and i need to fetch some


3/23

3SOL). &irst read all the les that are to be processed.

2. 7ssign 8ni@ue I' or number to each le.(se@uential".

. 'esign a )ob. ;. 8sing loop activity read each le and process.


4/23

"Bou put your # dataset lesCse@uences les in one datastage )ob and then you

passe all # output link into a funnel and from funnel you link out only one link out

which contains all your # les

7QUES)Hi

I have a re@uirement where I have to fetch few les from a folder of similar pattern

and having same metadata and then need to apply some transformations on it and

then I have to produce same no of les fetched from the source folder and place

them in target folder. x: dir! has 2!% les and i need to fetch some


5/23

A Initiali?e 4ariables A

vvent'etail 0 0 v6artitiontart !$%%%% v6artitiontop !$%%%%

SOL)A111A A et all )ob parameters from specied ini le into 6aram-ist A

A111A 6aram-ist Ket6arameter7rray(p6arm&ile" 9 A get parameters from the .ini leIf 6aram-ist ++3+ 5hen all '-og&atal(rror occured getting parameter

array from le 1 : p6arm&ile, 3p6erfube" nd pssbaseBear

Ket6aram(6aram-ist, pssbaseBear"

pssbaseDonth Ket6aram(6aram-ist, pssbaseDonth"

p'wh'b Ket6aram(6aram-ist, p'wh'b"

p'wh8serid Ket6aram(6aram-ist, p'wh8serid"

p'wh6asswd Ket6aram(6aram-ist, p'wh6asswd"

8Q)6lease let us know more information,

ventually you have a column as integer or decimal and u are sending a character

which is not correct or if you are sending a integer and you have decimal in your

target then the format is not correct, you have had 0.0 a point. &or the second

suggestion is concerning if you are using >.< you will not have this issue on #.!. Lust

let me know more information about your )ob and all warning you get so then I can

solve it more easy for you.

I am getting a warning in my )ob while converting string to decimal : /umeric string

expected, got 1! . 8se 'ecimal default value. an anyone help me how to avoidthis. I need sign value in decimal eld.

1SOL) Hi, in source iCp format is !21)une12%%$,but i need in target like this format

!211%=12%%$,How can we change I tried like this... i have used transformer stage in

tringtodate function,but i didn0t get corrct oCp...defalut i am getting system oCp.....

Ohat is actual process 2" Hi, I think I3/4 or 33/4 internal functions can be

usedHI , Oe can use AtringtodateA function in stage variable then we can reverse

the stagevariable to desired format . testdatetring5o'ate('-ink2.'ate,Pdd1

Pmmmm1Pyyyy" naldatetestdateF$,2G:010:testdateF=,2G:010:testdateF!,;G

9Q)I have a some data where the tate column may have a tate spelled out or anabbreviation 3r the state column may contain a country. o I do a lookup in a

country lookup to see if the tate is a country and place it in the country column. I

also do a tate lookup to see if the tate column has an abbreviation like 67, /L and

then I pul the state.. Here is the problem. 5he state lookup table has an integer to

lookup based on a number it also has the tate 7bbreviation and tate spelled out.

5he 7bbreviation column is 2 har. I use this to see if any rows in the tate olumn

of my data contain a state abbreviation. Ohat is happening is Ohen /epal comes


6/23

in as the state and looks up against the abbreviations it Datches on / and so I am

getting /ebraska as the state when I don0t expect it. 'espite the fact that /epal

does not e@ual /. I have tried changing the metadata in 'atastage so the

7bbreviation is read into a < character eld and it does the same thing. It still thinks

/epal is e@ual to / 7nyone have any idea what is going on here*

9SLO)/epal will not match / unless you take only the rst two characters 2"

Beah, that0s what I would have thought but 'atastage is not doing that. 'atastage is

only looking at the rst 2 chars I guess because that0s the length of the lookup key.

ut I changed the lookup key length to < and it still messes up.I got around this by

putting anything in the state column len(tate" Q < into a diMerent eld to use for

my look up. It works ne. I can0t gure out why 'atastage is acting as expected but

I see strange things like this all the time." It is the problem with eld width check

ur eld width it is taking 2 only thats why this problem coming

10Q)Hi, 7ll I have a !% les, which is created by datastage. basically this is re)ect

le from !% diM )obs running in ! se@uencer. now i wand to check for the data in thele and if data found then i want to email this le. how can i achieve this rek.

10SOL)Bou can do it in unix or in 'atastage. In can loop through the list of all le

names. 7nd do the following. if test 1s Ri then mail 1s ub)ect emailSremoved Q

Ri

2"Hi +ocky, 7nyways in order to send an email initially please congure the email

server in your 'atastage server. 7fter which you can congure it to send email from

'atastage or from -inux server. would write a routine to check if the le has

records, if it does, then create another routine to send the email with the

attachment. Bou might be able to use the email stage but if not you can use thebasic command 'endDail(" which you can customi?e and send the attachment

that way....

11Q)Hi, is there any limitation in datastage when passing environment variable as

parameters in xecute command activity stage in a se@uencer, in my )ob i am using

execute command activity stage to fetch some values from a le and the command

i am using is cat dirCsubdirClename, when i hard code the entire path its working

ne , but when i pass environment variable to dene my directory and sub directory

, it throws an error , is there any other way of acheving this

11SOL)5here is a specic place holder for parameters. If that does not work, youcan even provide in command option itself. 'o a cross check on the value that0s

been passed on to the parameter.

!" Bou can verify from the 'irector log. Ohat is the error you are getting....cross

check the value of parameter. It should be same as that when you hard code

it (which in your case is working ne".


7/23

2" Hi , I have cross checked the values of parameters ,there is no issue with the

values. I get the following error 7ctivityEexecute1command:xecutable 6ath

cannot contain parameters I get this error when I compiling my )ob , as I said

earlier its working ne when I hard coded the values mentioned in

parameters.I am using ' #.%.! and Oin 2%% 3

" Bou can use 8ser 4ariable 7ctivity 1 dene a user variable and set its value

e@ual to parameter. 8se this user variable in execute command activity

12Q)Hi 7ll,

I always use 5ransformer stage for adding new eld with initial value (e@: adding

reated'ate column with urrent5imestamp(" in each of record". ut after doing the

stress test, I0ve found that this could be problem for the performance. Is there other

way to do the same without using 5ransformer* I0d tried to use Dodify tage but

unfortunately it can0t be done if not derive from Inputolumn.

SOL)Bou can use also column generator. Bou add new column eg:mydate and you

give it your current date.&or modify stage you have to dene in your output your

new column manually and it will work, you do not need to map your input to your

output, )ust give the same name to out put as your specication name, it should

work

If I use column generator, do it will automatically combine with the original record**

I will check it out, thanks for the enlightment 9p 7s as I know, copy stage can only

add new column with the existing value, not with new valueBes 7rash, I0d tried it

with columngen and it works as I want 9" ut still it only accept staticChard code

value in generator value. Is it possible if I want to get value from Lob parameter or

routine*If you target is ', you don0t even have to deal the reate'ate in

'atastage, rather you can handle in 'ata ase itself. Ohich will have even more

appropriate timestamp.

13Q)8/-7I&I'CC I have dates like this: 2%!!1%212! that need to be converted

to timestamp. Ohat is the best way to covert this date*

SOL) Bou can concatenete do this: 2%!!1%212! : %%:%%:%%.%%%

3conv(S'75,'1BD'F;,2,2G":010:3conv(S5ID,D5"

tring5o5imetamp('ate5otring(2%!!1%212!,0Pyyyy1Pmm1Pdd0":00:0%%:%%:%%0,Pyyyy1Pmm1Pdd Phh:Pnn:Pss"

14Q)Good Day everyone

I have the following data that re@uires me to validate that every D4 comes either

after an / or an other D4. what would be the best way to validate that * Is there


8/23

anyway to dene a temp4alue what will hold the value in the previous cell in order

to compare the value in the previous cell with the one in the current cell* / ,D4

,D4 ,/ ,D4 ,4/ ,4/ ,4/ ,/, D4 ,4/ 4/ 4/ / D4 D4 D4 D4 D44/ 4/ / D4 D4

/ D4 D4 / D4 / D4 D44/ 4/ 4/ / ,

SOL)Bes. In transformer, you can use stage variable to hold the previous value andcheck for the current records value 2)6rev4alueurr4alue urr4alueInput.&ield If

urr4alue0D40 7nd (6rev4alue0D40 or 6rev4alue0/0" 5hen logic!0 lse 0-ogic20

5hanks 'eepak, but I0m still have a little bite of confusion: by doing

6rev4alueurr4alue urr4alueInput.&ield It looks like both 6rev4alue and

urr4alue will always have the same value which is the Input.&ield right* if that0s the

case, I0d be comparing the same value over and over which will result in true . an

please explain a little bit*

Hope the following clears your doubt. 6rev4alueurr4alue If urr4alue0D40 7nd

(6rev4alue0D40 or 6rev4alue0/0" 5hen logic!0 lse 0-ogic20urr4alueInput.&ield ither case, while assigning the 6rev4alue, the urr4alue will

be holding the value of previous records value. -ater urr4alue will be assigned with

the urrent records value. 5he urr4alue will hold its value until the next

assignment, which is only at next record. -et me know, if you need any further

explanation

5hanks so much 'eepak for your explanations. I tested it , it works. I0m new to using

this tool, I have a lot to learn, and I appreciate everyone0s help on this forum

15Q)H!

I need to run a multiple instance Lob > times simultaneously(in parallel".I know oftwo methods to accomplish this !.7 script using ds)ob command and a loop. 2.7

Lse@ containing as many )ob activities as the number of parallel run of the )ob i

want. Lust want to know if there is any other way to accomplish this ,preferably not

using a script.

SOL" Hi, I think you can go for a se@uencer by which you have a start loop and end

loop and place the )ob activity and give the details about the )ob .I hope it works

ne(Bou can use loop in your stage It0s more easy" an looping activity be

independent* Is that what you mean to say* -ooping will not be simultaneous, it willbe one at a time. Bou would be better oM using se@uencer to manage. Dake sure

you follow the rules for multi1instance )obs and parameteri?e le names, etc

7s I know, you can create your single )ob and active multiple instance, and then use

it in your loop and run it, i had the similar )ob before and it worked ne, Bou can use

diMerent parameter and run them independently as you hopelet me know if you

have more @uestion and if you would use it :1"


9/23

16Q)I don0t understand why my where clause is not working: /7DE3/ like 0Oells

P0 the )ob goes to completion but my records are getting re)ected. Hi friend I 5ried

the same thing in my pc . It0s working ne .....ould you please provide some more

details.... 7" 5hank you for responding. I have found out that the where clause in the

lter stage is case sensitive. 7nd my le is in all 76. ie, /7DE3/ like 0OellsP0

doesn0t work, but /7DE3/ like 0O--P0 works. 'o you know if there is any wayto not have it case sensitive* 8se 866+(" or -3O+(" function on /ameE3ne and

put the value accordingly in the -IT Hi,use upper(" or lower(" to check it out once.I

have tried it and its working ne. Bes, you can use uppercase in yr transformer and

then do your lter 7s long as Oells is not O-- you should be ok

A"S)y applying a conversion to 8pper case to /7DE3/ then applying the like

0O--P0 will satisfy even a column with mixed case names. hances are all names

are converted to 866+ case before they are written to the /7DE3/ column. If

they are not all 866+ case and you use the 866+(/7DE3/" like 0O--P0 any

index on /7DE3/ could be nullied. heck out the table and see if all /7DE3/

columns are upper. Hello good morning Bour table table name is 76.......In'atabase table values should be case sensitive. (eg: emp table,dept table.....in

oracle"

17Q)I am getting a warning in my )ob while converting string to decimal : /umeric

string expected, got 1! . 8se 'ecimal default value. an anyone help me how to

avoid this. I need sign value in decimal eld.

SOL)Bou got this because you are sending a character to a numerical type such int

or decimal, 5he other reason could be, because you are ending int to decimal then

u have to send the correct format, mean . is missing ( this happened only in >.< and

not in #.!" -et me know more about your )ob and your warning log to help 2)you/ope , I am getting warning when I am trying to do it from H7+ to decimal but

this warning is not coming in case of varchar to decimal.3) Lust check the length of

char and decimal, as char data can be padded with default value which is not

accepted by decimal type . 5his can be one reason not sure4) saurabh, it is the

same as i said before, and that could have some many reason, but i )ust gave u

some example, and here is the example of ur case :1 " if u are doing a transypage

for a date or decimal etc.... it s always advised to do it from varchar, because when

you use char, for the null character 'atastage interpret it on -inux as a haracter it

s why u are getting this warning5) 5ry to trim that char eld while moving to a

decimal eld="I already did trimming and all before moving it to decimal eld.7nyways I was developing it on >.< version , while I moved to #.! it is resolved

18Q" I have data coming in where the -ast/ame column may also contain a uUx

like Lr or r or III If the last name has a suUx it would be separated with a comma

like ... mith, Lr I need to extract the Lr or r or whatever it is after the comma and


10/23

put it in another column called suUx I also have another set of data where the users

did not use a comma and those last names look like mith Lr 5he last name column

may contain a second middle name then the last name then Lr or r... -ike -astname

0'oe mith Lr0 where the name is Lohn 'oes mith Lr In 5his case I need to extract

the Lr or r or whatever it is at the end of the name and put it in another column

called suUx I am using 'atastage #.! on Oindows in a parallel )ob 7ppreciate thehelp 5hanks

SOL) &irst check for comma and separate the uUx using &ield function. If you don0t

nd any comma take the last eld based on space as separated and check for pre

dened values. (Lr, r, II, III* etc". 6re dened values need to be built by you for

accurate results. Bou can always use the @uality stage with uropean /ames to

segregate it 'eepak, thanks for the help but can you show me the syntax for the

&ield function or were to nd it. I have not used the eld function and do not see it

on the 6arallel guide.ount(QInput.&ieldV,Q'elimiter tringV" 1 5his gives you the

total count of the 'elimiter.

g: ount(Input.&ield,,"

&ield(QInput.&ieldV,Q'elimiter tringV,3ccurrence" Kives the eld based on the

occurance

g: &ield(Input.&ield,,,!" gives you the 2nd eld of the input based on comma.

'eepak, I have nally tried this and got one le correct. 5he le with name like Lohn

'oe, Lr I managed to separate the Lr from the name with &ield([email protected],,,1!"

that gets me the last name and &ield([email protected],,,2" get me the suUx I have a

second source that does not contain a comma so its like Lohn 'oe Lr 7nd some

names have multiple &irst and last names -ike Lohn 7lexander mith 'oe Ohere

mith 'oe are in the last name eld other times there is like names like 'e -a

Kar?a I have not been able to gure out the ount function to get me where I need

to be and also I need to make sure I am pulling out Lr or II or III and not the last part

of the last name I0m thinking I need a stage variable to get the ount on the

occurrences of the spaces and then check to see if the last string in the name is Lr,

r, II or III does that sound right.

Kood practice is to use a Information 7naly?er to nd out the pattern of the last

names. 7nd pull out all the set of possible uUx that you need to pull put and have

it as a lookup or if the count is small enough, hold it in stage variables to do thetransformations5hat0s all good but I can0t seem to remove the Lr, r I,II or III because

all the delimiters are spaces

#) -ets do it again. :&irst consolidated the list of uUxes. -oad it in to a lookup

table or in stage variables. eparate out the last eld based on space. ince

all the words in last name can be separated out based on space even if they

have separated out of comma. 'o a lookup with the consolidated list. If you


11/23


12/23

$" greatn i see you are still following this post,

i am not saying you are wrong , i )ust suggested what was written as performance

from ID, yes, you are right there is the diMerent way to create a )ob, but i )ust give

some idea for performance , if it s not interested forget it

20)Q I need to implement the below logic in 'atastage, If only ! record is read with

a particular K+61I',then its 5B6 is changed to +7. If more than ! record A is read

with the same K+61I', then they aren0t change. How do I do this, any help really

appreciate.

SOL) "o$ %&'( 'ear on yo&r re*&!re%en$+

Is your re@uirement to change the 5B6 based on the K+61I'* If so, use sort stageto add the lusterTeyhagne column based on the K+61I'. ort it again by

'escending based on the K+61I' and lusterTeyhagneolumn. If you get two

continues !s then its +7.

1) Hi 'eepak, 5hanks for the reply, now my out put looks like below Dy output

now looks below K+61I' Teychange


13/23

Hi 7s you see when you have % after sorting 5hat mean you have more than !

records o then you can make your changeX

Kood that you already got it xed. ut anyway, the solution with your previous

approach would be, to sort the records based on Teyhange in opposite order. o

that your %0s comes rst. /ow, &or those records whose Teyhange !, you need touse tage variable to check the previous values to see if its !. If so then you need

ot assign 0+70.

21) Q Hi, Dy re@uirement is I need to get the output in a eld as

lenameEcurrenttimestamp.dat.g? in the derivation of transformer I have given

0lenameE0:currenttimestamp(":0dat0:0g?0 am getting the output as lenameE2%!!1

%1%$ !;:2:;; but my re@uirement should be this way lenameE2%!!%%$1%22$;!

which am unable to get.. even i have tried to concatenate with the parameter

currenttimestamp but this is not possible an any one help me in this regard 776

as its urgentXXXX thanx in advance21SOL)8se 5imestamp5otring function to convert its format

If you are using datastage server, in transfomer server you can do it easily by

iconvCoconv

but if you are using the datastage px do as deepak say in above Bou transtyp it to

string and then you use convert(010,00,yourstring" it should be 3T

22Q" Hi , I am working on datastage v >.


14/23

DD1'' and if it is not the case you will get such warning the other thing is if you are

getting the data as i said in above BBBB1DD1'' but the date is wrogn ex: 2%!%1%21

! 0 which does not exist in real time" you get that warning

24Q)Hi 7ll, Dy @uestion is how to remove a duplicates in record. for example the

input column having string like %%7%%%%%%'%!7%!7%!%!%%7%% i want theoutput as %%7%%%%%%'%!7%! Here you have to take two numeric and one

alfabat as one value N:%%7 How can we design the code to sort out the above

re@uirement*

24SOL)5his is on similar lines what 'eepak mentioned. 5his worked for me. 5ry it:

'ene stage variables and their derivations in following way, a 1 '-ink2.inputF!,G

- 1 If '-ink2.inputF;,G aT(en a E.e a:'-ink2.inputF;,G' 1 If

Index(b,'-ink2.inputF>,G,!" V %$(en - e.e b:'-ink2.inputF>,Gd /

Index(c,'-ink2.inputF!%,G,!" V %$(en ' e.e c:'-ink2.inputF!%,Gand so on..

-et us know if any error you get

25Q)H! A

Oe have created a se@uence )ob which has )obs in it. Ohat I want to do is that if

any data is re)ected and written into the re)ect le in the rst )ob then the second

and third )ob should not be executed

25SOL)Ohat you are using to re)ect the record * if you are using a 5ransformer

then in the contraint you can abort the )ob after the rst row. /ow when the )ob is

aborted in the se@uencer you can use the status for any purpose like the second

and third )ob will not be executed. Oe have re)ect le at lookup stage in the )ob.

Ohat we have to do is 1. In case there are any re)ect records, then send these

records in a particular format via mail(&or which we have written a shell script which

is run after execution of rst )ob " 2++e@uirement is that 1 in case this mail is sentvia shell script then the next )ob in se@uence should not be executed. 5ransformer

has 7bort after n rows option

8se )ob activity to execute the Dain Lob. 8se xecute command 7ctivity to check for

the number of lines in the re)ect le. If the /umber of line is greater than ?ero, send

a return code as ! ()ust example" else %. 8se /ested ondition with


15/23

ustom(conditions" to check the return code, based on that, either you can execute

the mail /otication or the execute the other )obs

ompletely agree with 'eepak, 8se )ob activity to execute the Dain Lob. 8se

xecute command 7ctivity to check for the number of lines in the re)ect le. If the

/umber of line is greater than ?ero, send a return code as ! ()ust example" else %.8se /ested ondition with ustom(conditions" to check the return code, based on

that, either you can execute the mail /otication or the execute the other )obs.

In the se@uence )ob, after your rst )ob use a execute command stage and using the

wc 1l unix command you can check for the number of lines in your re)ected le.

7nd as said by 'eepak, you can then decide whether or not to run the remaining

)obs in se@uence depending on the line count that you get as output of execute

command stage

26Q)6osted by +a)esekhar on Dar # at =:2= 6D Hi, I am using 'atatage 4ersion

#.!, wanted to develop a parallel )ob. Here is the logic that I wanted to implement:'b2 nt tage (ource" 1 5his has date as a column in it. I have a reference table

'b2 nt stage which I will use to do lookup and get few columns. I want to pass the

date of '2 ource stage to '2 +eference tage and get the desired results. '2

reference table will have a @uery which will use the date of '2 ource stage.

elect .ol!, +.ol2 &+3D .5ab!, +.5ab2 OH+ .5ab!.ol!+.5ab2.ol! and

+.tab!.'ate Q .5ab2. 'ate I can do all this in a single @uery suing a '2 nt stage

but I am dealing with huge data, I am not sure of the performance if I do so. an

any one please tell me is there any better way to implement this*

26A"S)How big your +eference 'ata and source data is* Its is always better to use

the one single @uery in the '28' stage and pull out the output and do complextransformation after that. &or the performance improvement, Bou need to make both

the ource and reference table has be properly indexed or portioned on the right

keys which you are )oining.

27Q)Ohat is syntax le* Bou me compiler* Bou0ll need to map the compiler path to

the nvironmental variables. Bou need to compile the routine using ZZ compiler.

reate a 3b)ect le and link the ob)ect le from the 'atastage routine. (If you

choose ob)ect method" Bou can check the Danager guide on how to link the ob)ect

le.

27SOL)'id u get this error during compilation or during runtime* /ot able to open

! le ** does not mean that your le is not there XXX

Hi all , 5he issue got solved . 7ctually instead of providing directory path, I gave

directory path plus le name . ut now , It gives error if I pass only one key ()oin

key" column in output out of two les .since the les are )oined(inner )oin" on these


16/23

columns it shud allow to select any one key column from any one le . Hi divya, It

was run time error . Kot resolved, the path provided Oas for le, but re@uired was

directory path ,7nd you don0t need the other columns at all* Ohat is the exact error

message*

28Q)Hi 7ll, Is there anyone who could kindly tell me the main diMerence between5ansformer tage and asic 5ransformer tage* In addition, I0m using ' >.


17/23

30Q)Hi 7ll, I am using a se@uence )ob which contains a number of parallel )obs. I

want the se@uence )ob to get aborted, if the no records are loaded in the target

table specied in one of the parallel )obs inside. Is there any way in 'atatage

#.! to forcefully abort the )ob based on the resultset of a Y- @uery* Oe can check

the number of records loaded in the target table through 7fter Y- and then how

can we implement it to abort the )ob, if the result of the @uery is ?ero* Oe are using'atatage #.! in Oindows 2%% server environment.

30SOL)8se the result in as a parameter in the @uery stage and based on the result

of @uery divert the [ow to a terminator stage Bou can include a email notication

stage to monitor it by support.

!" !.Bou could add a 0xecute command stage0 in your se@,including a script or

batch le. 2.Dake your @uery result on the target table as your exit code for

the script or batch le. (e.g. the exit code % means no record has returned"

.-ink your 0xecute command stage0 to a 05erminator stage0 with the trigger

as +eturn4alue1(onditional" %

2" Bou can use the +outineE7ctivity stage with: "+outine/ame

8tility7bort5o-og (+outineVsdkV8tilityV8tility7rbort5o-og" and 7rgument

7rbot message that you want to add in log

" &or you specic case, this could be a solution:

;" !1reate a )ob that count the number of record loaded and write it into a textle /ame&ile.txt

" 21!1call previous )ob (!1" through a Lob activity

#" 2121then add an xecuted command to read the text le with command cat

/ame&ile.txt

$" 211then add a 8serE4ariableE7ctivity stage to get the contain of the value inthe le

!%" 21;1then use a /estedEondition with an output link wahen the

previous value (8serE4ariableE7ctivity" is e@ual to %

!!" 21


18/23

!2" and 7rgument message to add to log

31Q)Dy input source date is in varchar(!%", and like this: Dy input source

date is in varchar(!%", and like this: =C!C2%%!

31A"S)U.e .$r!n$oda$e &n'$!on+

stringtodate(0=C!C2%%!0,0Pm1Pd1Pyyyy"ince I need to populate it into '2

table, the format will be in BBBB1DD1'',

32Q) &.e .$r!n$oda$e &n'$!on $o 'onver$ var'(ar $o da$e+ y .o&r'e

da$a !. !e, 2010/12/01

y 'ode !. a. oo., ! DSL!n418+e(!reD$ :;; $(en ;; e.e

S$r!n$oda$eyyyy/>%%/>dd=)

error %e..ae,

A?T@o%-!nedOera$or@on$roer


19/23

Y"

I have a data like this: D7/H7557/ 7TI/K +I'K, / /O B3+T, /B /O

B3+T /B /B 6-7I/3+3. /L I want to separate city and state like I/ '75757K

L3. an i do this*

A"S)I need to know if you )ust looking for )ust parsing or looking for cleansing as

well. Bou can do the earlier using the 5ransformer (Day not even re@uire, if you read

the le with delimiter". -ater would need the Yualitytage stages. 7nd the rst

records doesn0t have the tate name as /B and second record doesn0t have the

proper tate name as /L and your


20/23

! is subtracted because you do not want to include the blank in city. tate is derived

as ereplace(inputeld, left(inputeld, index(inputeld, 0 0, count(inputeld, 0 0""", 00"

replace will substitute a null string in place of the city data. 5his time the 1! is left

oM because you want to remove the last blank as well.

34Q)Oe have atransation table,it contains Lan month transcation data and &eb

monthtranscation data(urrent Donth". would like to run the )ob in Lan month,it

should be display Lan month transcationdata -ike that if i will run the &eb month,it

should be display Lan month Z feb 2 months transcation data similarly if i will run

the Darch month,it should be display Lan Z&ebZDarch monthstranscation data

etc.

Ohat is logic for above @ueries and let me know how to create a )ob* &or that )ob

how many targets are re@uired* an you please help in the above @ueries in

datastage*

34SOL)Bou can simply select the data with the where class of the current year.

-ooks like you have the live transaction table. o all the every month you run, you

are expecting to have all the data. If you think, you might go back and do the back

data run, you can have the where class based on the Donth. 7nd passing the month

as parameter should be decided by your design team. 7nd about the 5arget, its

totally depends on your business logic. Bou )ob shouldn0t look complex as, the

condition can be achieved in ' @uery level. 2) imple orcl1Vtransformer1

Vds(or"any target in transformer if then else if monthfromdate(currentdate(""

0)an0 then )andata else if 0feb0 then )andata Z febdata and so on upto

december in >.< or #.% it is possible

2) Inserting a lter (Ohere clause" while extracting from source would be little

more eUcient than, ltering (5ransformer or lter"it out after extraction.

3) In , and all parallel versions, it is always better to do as much in the Y- as

possible......the database is designed for the ultimate performance so it0s not

)ust a little betterCfaster, it is the most eUcient way to develop using

'atatage. It0s best to do as much data conversion in the select statement

as possible, including using the where clause to select only the data that0s

needed......only select columns re@uired at the target.......by doing so, you

make each of the re@uired steps in ' easier to code and manage.....and you

reduce the number of items on the pallet. 8se Y- to it0s fullest in the selectY- and you will nd there is much less demand on ', increasing eUciency

and throughput, and the amount of ' coding that is re@uired.

4) /ot all the case. 'atastage are strong in 5ransformations. Oith dedicated

68s and memory, and after all the sole purpose of the 5- is to do that. Oe

need to dene the line based on the resource 8sage in either of the servers.


21/23

3verloading highly utili?ed ' servers with comparatively idle 5- servers

may not be an advisable option


22/23

'atatage oMers solutions for 5- and -5 that normal Y- is not able to perform.

5his debate will rage on for years between 5- developers and Y- coders until the

end of time......and is outside the scope of this debate. If you wish to implement

everything in 'atatage, go ahead....but there are trade oMs. ame goes if you wish

to create an entire system with only Y-......

5- has a place, as does Y-. It is the combined results of both that give 'atatage

it0s power, among other features. It is the wise use of both that gives the 'atatage

developer an advantage. 5here are always many solutions to a given problem, and

while saying that it 0depends0 is certainly true based upon the environments,

business re@uirements, and available e@uipmentCresources. 5hat being said, )ust

because a solution worked in a given situation does not make it proper, does not

mean that it complies with best practices, does not make it maintainable, nor does

it make it wise. It simply means that it worked once in that particular situation. I0ve

opened canned foods with a knife......it wasn0t very smart when can openers do the

)ob safely and much more eUciently.....-ogical explanation would be* when a data

was extracted from a high visible 3-56 ', for miscellaneous purpose (compared tothe actual purpose of ' itself" like nice to have reporting. Hence it was given only

a percentage of 68 allocation and short batch window. xceeding the limit would

block the extract option from the '. Ohen the data need to be extracted after

couple of heavy aggregation and followed by a couple of )oins 68 and memory and

memory usage and the processing time shot up way high beyond the limitXX

Ohatever the hints or index or partitions we use. It was resolved simply by pulling

the data with simple select and aggregation and )oin was made to perform in 5-

server in the memory. Dathematically, it satised the re@uirementXX

35Q)Ohen running a )ob, i get the following errors, LoinE


23/23

If you have columns with the same name on both input link and don0t use these

columns in )oin key, the process generate warning.

orry, after check, except for the two )oin keys, there is no other columns with the

same name on input link and output link,

36QS) In the transformer derivative for a eld i have this %% :

-eft(+ight(7mount5otring,!>",!;" : +ight(7mount5otring,2" an anyone explain

me the functioning of this

SOL) !. 6ad two ?eros to the right. %% 2. Ket substring of rd to !;th position

(skip rst character" F-eft(+ight(7mount5otring,!>",!;"G . 7ppend rst 2

character to the last. F+ight(7mount5otring,2"G

-ets say 7mount5otring value is %!2;#$7'&KHIL(2% characters"

+ight(7mount5otring,!>" right(%!2;#$7'&KHIL,!>" ;#$7'&KHIL (5ake !> characters from ur right hand side"

-eft(+ight(7mount5otring,!>",!;"

left(;#$7'&KHIL,!;";#$7'&K (5ake !; characters starting from

ur left hand side"

+ight(7mount5otring,2" right(%!2;#$7'&KHIL,2" IL (5ake 2 characters

starting from ur right hand side" &inal functional: %% :

-eft(+ight(7mount5otring,!>",!;" : +ight(7mount5otring,2"

%%;#$7'&KIL

Documents

Usefull Stuff Datastage