33
TCS GE Confidential TCS GE Confidential AutoSys is an automated job control system for scheduling, monitoring and reporting. An AutoSy s job is any single command, executable, script, or NT batch file. AutoSys job definition contains a variety of qualifying attributes, including the conditions specifying when and where a job should be run. There are the two methods to create job definitions: Using the AutoSys Graphical User Interface (GUI). Using the AutoSys Job Information Language (JIL) through a command-line interface. Introduction to AutoSys 

Autosys Details

Embed Size (px)

Citation preview

Page 1: Autosys Details

8/3/2019 Autosys Details

http://slidepdf.com/reader/full/autosys-details 1/33

TCS GE ConfidentialTCS GE Confidential

AutoSys is an automated job control system for scheduling, monitoring andreporting.

An AutoSys job is any single command, executable, script, or NT batch file.

AutoSys job definition contains a variety of qualifying attributes, including the

conditions specifying when and where a job should be run.

There are the two methods to create job definitions:Using the AutoSys Graphical User Interface (GUI).Using the AutoSys Job Information Language (JIL) through a

command-line interface.

Introduction to AutoSys 

Page 2: Autosys Details

8/3/2019 Autosys Details

http://slidepdf.com/reader/full/autosys-details 2/33

TCS GE ConfidentialTCS GE Confidential

NameStarting Conditions

AlarmsRestart Conditions

Current statusStart timeEnd timeExit code

Command to executeFile to source

Machine to run onStandard output filesStandard input file

File NameMinimum File size

Watch Interval

Box

TIMEDEPENDS ON

JOB

COMMAND

FILEWATCHER

Run Commandon Machine

Watch for File

CONTAINER

Definition

Current State

Autosys Job File Details

Page 3: Autosys Details

8/3/2019 Autosys Details

http://slidepdf.com/reader/full/autosys-details 3/33

TCS GE ConfidentialTCS GE Confidential

Box jobs are containers that hold other jobs.A box job can be used to organize and control process flow. The box itself performsno actions, although it can trigger other jobs to run. An important feature of this typeof job is that boxes can be put inside of other boxes.Default Box Job Behavior:Some important rules to remember about boxes are

Jobs run only once per box execution.Jobs in a box will start only if the box itself is running.As long as any job in a box is running, the box remains in RUNNING state; the boxcannot complete until all jobs have run.By default, a box will return a status of SUCCESS only when all the jobs in the boxhave run and the status of all the jobs is "success.By default, a box will return a status of FAILURE only when all jobs in the box haverun and the status of one or more of the jobs is "failure."Unless otherwise specified, a box will run indefinitely until it reaches a status ofSUCCESS or FAILURE.Changing the state of a box to INACTIVE changes the state of all the jobs in the boxto INACTIVE.

Page 4: Autosys Details

8/3/2019 Autosys Details

http://slidepdf.com/reader/full/autosys-details 4/33

TCS GE ConfidentialTCS GE Confidential

Defining Autosys Jobs 

• AutoSys Graphical User Interface:

The AutoSys GUI allows you to interactively set the attributes thatdescribe when, where, and how a job should run.

• Job Information Language:

AutoSys JIL is a specification language, with its own syntax, that isused

to describe when, where, and how a job should run.

Job Information Language JIL) is a scripting language whichprovides a

way to specify how AutoSys jobs should behave. JIL scripts containone

or more JIL sub-commands and one or more attribute statements;these

elements constitute a job definition.

Page 5: Autosys Details

8/3/2019 Autosys Details

http://slidepdf.com/reader/full/autosys-details 5/33

TCS GE ConfidentialTCS GE Confidential

Various states in autosys:

• INACTIVE: The job has not yet been processed. Either the job has never been run, or its statuswas intentionally altered to “turn off” its previous completion status

•  ACTIVATED: The top-level box that this job is in is now in the RUNNING state, but the job itselfhas not started yet.

•  STARTING: The event processor has initiated the start job procedure with the Remote Agent.

•  RUNNING: The job is running. If the job is a box job, this value simply means that the jobs withinthe box may be started (other conditions permitting). If it is a command or file watcher job, thevalue means that the process is actually running on the remote machine.

•  

SUCCESS: The job exited with an exit code equal to or less than the “maximum exit code for success.” By default, only the exit code “0” is interpreted as “success.” If the job is a box job, thisvalue means that all the jobs within the box have finished with the status SUCCESS (the default),or the “Exit Condition for Box Success” evaluated to true 

•  FAILURE: The job exited with an exit code greater than the “maximum exit code for success.” Bydefault, any number greater than zero is interpreted as “failure.” AutoSys issues an alarm if a jobfails

•  

TERMINATED: The job terminated while in the RUNNING state. A job can be terminated if a usersends a KILLJOB event or if it was defined to terminate if the box it is in failed. If the job itself fails,it has a FAILURE status, not a TERMINATED status. A job may also be terminated if it hasexceeded the maximum run time (term_run_time attribute, if one was specified for the job), or if itwas killed from the command line through a UNIX kill command. AutoSys issues an alarm if a jobis terminated.

Page 6: Autosys Details

8/3/2019 Autosys Details

http://slidepdf.com/reader/full/autosys-details 6/33

TCS GE ConfidentialTCS GE Confidential

• RESTART: The job was unable to start due to hardware or application problems, andhas been scheduled to restart.

•  QUE_WAIT: The job can logically run (that is, all the starting conditions have been

met), but there are not enough machine resources available.•  

ON_HOLD: This job is on hold and will not be run until it receives theJOB_OFF_HOLD event.

•  ON_ICE: This job is removed from all conditions and logic, but is still defined toAutoSys. Operationally, this condition is like deactivating the job. It will remain on ice

until it receives the JOB_OFF_ICE event. 

Page 7: Autosys Details

8/3/2019 Autosys Details

http://slidepdf.com/reader/full/autosys-details 7/33TCS GE Confidential

TCS GE Confidential

Status Abbreviations: • The following table lists the abbreviations used in the ST (status)

column of the autorep report, and gives the status for each

abbreviation.

AC - ACTIVATEDFA - FAILUREIN - INACTIVEOH - ON_HOLD

OI - ON_ICEQU - QUE_WAITRE - RESTARTRU - RUNNINGST - STARTINGSU - SUCCESS

TE - TERMINATED

Page 8: Autosys Details

8/3/2019 Autosys Details

http://slidepdf.com/reader/full/autosys-details 8/33TCS GE Confidential

TCS GE Confidential

Note: 

The difference between "on hold" and "on ice" is that when an "on hold" job istaken off hold, if its starting conditions are already satisfied, it will be

scheduled to run, and it will run. On the other hand, if an "on ice" job is taken"off ice," it will not start, even if its starting conditions are already satisfied.This job will not run until its starting conditions reoccur.The other major distinction is that jobs downstream from the job that is "onice" will run as though the job succeeded. Whereas, all dependent jobs do not

run when a job is on "on hold"—nothing downstream from this job will run. 

Some basic Autosys Commands:

This will display a list of jobs with complete details with box/jobname, last/latest

run date & time, status, exit code, etc.autorep –j <job_name>

example:

autorep –j gpag0201f_orders_load_daily_inf

Page 9: Autosys Details

8/3/2019 Autosys Details

http://slidepdf.com/reader/full/autosys-details 9/33

Page 10: Autosys Details

8/3/2019 Autosys Details

http://slidepdf.com/reader/full/autosys-details 10/33TCS GE Confidential

TCS GE Confidential

SAMPLE JIL FILE

• Sample JIL file saved as *.jil extension

Page 11: Autosys Details

8/3/2019 Autosys Details

http://slidepdf.com/reader/full/autosys-details 11/33TCS GE Confidential

TCS GE Confidential

STEPS TO RAISE A CASE FOR AUTOSYS RELATED ISSUE

The following steps is abt how to raise a request on kintana for your jobs(New/change/delete/One-Time)!!

1) Go to http://insidegis.ge.com 2) Login with your SSO Id / password3) On the left hand side under 'Change Management', there is a link 'Modify a

Service/Asset'. Clicking on this link refreshes and brings up a new page.4) The first field is 'Service' which is a drop down field. Choose 'Server' as applicable to

you.5) Enter you contact information/Desired Start Time/Desired Completion Time.

6) Enter the 'Change Type' . Most of the cases it will be 'Standard'.7) In 'Affected Scope', you need to enter the server system name(gcoasys01).8) In Change Reasons, enter the type of request.

a) Production Scheduling Request - New (For a new Job)b) Production Scheduling Request - Change (For changing a job parameter)c) Production Scheduling Request - Special One Time Process (For forcestarting/putting a job ON/OFF ICE/HOLD)

d) Production Scheduling Request - Remove (For deleting a job)9) You can also enter any free text in the 'Change Description' area.10) Click on Submit and note down the ticket number.

Page 12: Autosys Details

8/3/2019 Autosys Details

http://slidepdf.com/reader/full/autosys-details 12/33

TCS GE ConfidentialTCS GE Confidential

Monitoring Autosys Jobs

Option I• GUI Link For Autosys

• http://gcoasys02.corporate.ge.com/autosys/login 

• Job Monitoring & Reports:

• GUI to Monitor the Jobs

• Reporting Through GUI

• Dashboard for Functional Users: In Progress

Option II

• Autosys Command Prompt

• Go to Remote Login Server > Start > Unicenter Autosys Cmd Prompt

> Enter the command ―autorep –j autosys jobname‖ 

Page 13: Autosys Details

8/3/2019 Autosys Details

http://slidepdf.com/reader/full/autosys-details 13/33

TCS GE ConfidentialTCS GE Confidential

FAQ’s 

• What if I don't want to run the ETL cycle for the given day?

• How do I delay a box?

• How do I delay the job within a box?

• How do I cancel a box?

• How do I cancel a job within a box?

• How do I run a failed/errored out job or box on the same day?• How do re run any job/box on the same day?

Page 14: Autosys Details

8/3/2019 Autosys Details

http://slidepdf.com/reader/full/autosys-details 14/33

TCS GE ConfidentialTCS GE Confidential

How to contact Autosys team?

There are several ways to reach the Autosys team:1. Here in this community you could enter a case and direct it to any expert that lists Autosys in

their expertise but only for general questions. We would prefer that you use the proceduredocumented in Request for Service sub-category under the Process category in thiscommunity. See document there titled "How to Use RFC for Production Scheduling requests"to follow the directions to have a request ticket entered in our system. The documentaddresses Autosys job/schedule requests as either New, Change or Remove that go to theschedulers or "One Time" requests that go to the production monitors on duty.

2. If you have a problem, a question about Autosys or need to make a request, you follow adifferent path. Still using http://insidegis.ge.com/ you can select "Ask A Question" (to ask aquestion or make a request), select "Modify a Service/Asset" under "Change Management" (torequest a remote agent install or license key) or select "Incident Management" to enter anissue. There's a new document in this community's Process section called "How to ask aquestion, report a problem or submit a non-schedule change request" that has all the details.See the attachment to this FAQ for some further details on submitting a request or reporting aproblem. GIS Help Desk is also available via *222-6666. The InsideGIS portal lists the various

ways to reach the help desk.

3. Via e-mail there's a distribution list @CORP GIS Autosys Support that can be used to reachthe entire team. However, once a request has been sent and you want to contact the Autosysschedulers to forward documents, etc., you can use the distribution list @CORP GIS AutosysSchedulers to reach them. For installs and license keys you can use the DL @CORP GISAutosys Installers. 4) For Sametime access just take any of the DL's listed above, remove the"@" and that is a group name you can add to your list

Page 15: Autosys Details

8/3/2019 Autosys Details

http://slidepdf.com/reader/full/autosys-details 15/33

TCS GE ConfidentialTCS GE Confidential

 – What is Autosys?

 – Limitations of BO DI Scheduler

 – Distinctive Features

 – Offerings by GIS

 – GIS Customers

Page 16: Autosys Details

8/3/2019 Autosys Details

http://slidepdf.com/reader/full/autosys-details 16/33

TCS GE ConfidentialTCS GE Confidential

 – What is Autosys?

CA-Autosys is a batch monitoring and scheduling tool for the

Open Systems environment.

It uses several supplemental products to provide a complete

cross platform scheduling and monitoring solution for GIS.

Page 17: Autosys Details

8/3/2019 Autosys Details

http://slidepdf.com/reader/full/autosys-details 17/33

TCS GE ConfidentialTCS GE Confidential

Limitations of BO DI Scheduler:

 – Jobs can only be scheduled based on Day/Time

 – Jobs can not be scheduled on the Success or Failure of Particular Job

 – Lots of manual intervention is needed to complete the Data Warehouse

Load

 – Continuous Monitoring efforts required

 – Longer Job Cycle Time due to Manual Intervention

Page 18: Autosys Details

8/3/2019 Autosys Details

http://slidepdf.com/reader/full/autosys-details 18/33

TCS GE ConfidentialTCS GE Confidential

• Distinctive Features of Autosys:

 – Increased  Performance: Improved performance using multi-threaded processing to enable more jobs to run in parallel.

 – No Manual Intervention: Due to scheduling capability based on

Time/day, file arrival and job dependency, the entire loadingprocess can be automated.

 – Unparalleled Reliability and Scalability: Scales tens ofthousands of jobs easily over hundreds of multiple platform

machine per night, while maintaining its hallmark reliability.

Page 19: Autosys Details

8/3/2019 Autosys Details

http://slidepdf.com/reader/full/autosys-details 19/33

TCS GE ConfidentialTCS GE Confidential

• Distinctive Features of Autosys:

 – Enterprise-wide cross Platform Job Management: Simplifies crossplatform job management through direct integration communication toUnicenter Job management Solutions 

Page 20: Autosys Details

8/3/2019 Autosys Details

http://slidepdf.com/reader/full/autosys-details 20/33

TCS GE ConfidentialTCS GE Confidential

• Offerings of GIS for Autosys• High availability:   Provides High Availability of Autosys Scheduling

Server with fell over Servers (Cincinnati and Alpharetta)

 – 24/7 Support Team: Provides 24 * 7 support team for monitoring ofAutosys Jobs

 – Web Based User interface: Provides single point of control andadministration of jobs and reduces IT learning and operational costs

 – No Additional Cost to GE Security : GIS offers the Autosys serviceswith no additional cost of Hardware/Software/Services

 – Job Scheduling Flexibility: Job scheduling and Monitoring flexibility isoffered by GIS to the Application team. The application team may or may

not choose the option for Scheduling and Monitoring the Jobs.

Page 21: Autosys Details

8/3/2019 Autosys Details

http://slidepdf.com/reader/full/autosys-details 21/33

TCS GE ConfidentialTCS GE Confidential

• Pre-requisite for Autosys Scheduling:• Oracle SQL*Net V2.3 and Oracle TCP/IP Adapter are required

• Installer must have access to the server• The TNS entry to be made for two Autosys Oracle databases

• Autosys Agent to be Installed on Server

• Getting Started:

• http://data.supportcentral.ge.com/upload/16285/doc_172780.doc 

Page 22: Autosys Details

8/3/2019 Autosys Details

http://slidepdf.com/reader/full/autosys-details 22/33

TCS GE ConfidentialTCS GE Confidential

• Types Of Autosys Jobs:

• Command Job: This is what is commonly referred to as “a job”. The

command can be a shell script, an executable program, a file transfer,etc. When the job runs, there is the execution of a specific command onthe client machine.

• Box Job: The box job or box is a container of other jobs. It can be usedto organize and control process flow. The box itself performs no actions,

although it can trigger other jobs to run.

• File Watcher Job: A file watcher job starts a process that monitors forthe existence and size of a specific file. Once that file reaches thespecified minimum size, and is no longer growing in size for a specifiedtime, the file watcher job completes successfully, indicating that the filehas arrived.

Page 23: Autosys Details

8/3/2019 Autosys Details

http://slidepdf.com/reader/full/autosys-details 23/33

TCS GE ConfidentialTCS GE Confidential

• Process to Schedule Jobs through Autosys:1. Identify the Jobs to be Schedule

2. Decide upon the Dependency3. Logical grouping of the Jobs (Box)

4. Creating JIL Files

5. Raise a Request to GIS team to Schedule them

OR

Get Access to the GUI tool to schedule them 

Page 24: Autosys Details

8/3/2019 Autosys Details

http://slidepdf.com/reader/full/autosys-details 24/33

TCS GE ConfidentialTCS GE Confidential

Sensing Production Jil Files

Page 25: Autosys Details

8/3/2019 Autosys Details

http://slidepdf.com/reader/full/autosys-details 25/33

TCS GE ConfidentialTCS GE Confidential

Autosys Job Monitoring from DI server

Log in to Autosys Command Prompt

Start->All Programs->Computer Associates ->Unicenter->UnicenterAutosys ->Unicenter Autosys Command Prompt

Page 26: Autosys Details

8/3/2019 Autosys Details

http://slidepdf.com/reader/full/autosys-details 26/33

TCS GE ConfidentialTCS GE Confidential

• GUI Link For Autosys

• http://gcoasys02.corporate.ge.com/autosys/login 

• Job Monitoring & Reports:• GUI to Monitor the Jobs

• Reporting Through GUI

• Dashboard for Functional Users: In Progress

Page 27: Autosys Details

8/3/2019 Autosys Details

http://slidepdf.com/reader/full/autosys-details 27/33

TCS GE ConfidentialTCS GE Confidential

Gui Monitoring of Jobs

Various Views ( Prod/QA/Dev) with jobs under them

Page 28: Autosys Details

8/3/2019 Autosys Details

http://slidepdf.com/reader/full/autosys-details 28/33

TCS GE ConfidentialTCS GE Confidential

Various Boxes giving Status of Jobs in them ( Green –Success,Yellow –Ice, Red- Failure )

Boxes are used to bring together similar jobs

Page 29: Autosys Details

8/3/2019 Autosys Details

http://slidepdf.com/reader/full/autosys-details 29/33

TCS GE ConfidentialTCS GE Confidential

Jobs in Boxes can be seen by double clicking on Box which gives job details

Page 30: Autosys Details

8/3/2019 Autosys Details

http://slidepdf.com/reader/full/autosys-details 30/33

TCS GE ConfidentialTCS GE Confidential

Link to Create Ticket for Gui Account

http://gis.kintana.corporate.ge.com/itg/web/knta/crt/RequestDetail.jsp?REQUEST_ID=5070271 

Page 31: Autosys Details

8/3/2019 Autosys Details

http://slidepdf.com/reader/full/autosys-details 31/33

TCS GE ConfidentialTCS GE Confidential

To run Informatica job from Autosys we have two shell scripts1) Normal job not having flat file load 

 /inforprdsec/data/Scripts/exe_workflows.sh SEC_SC_PSI w_Sec_SC_PSI_Intransit_STG  /inforprdsec/data/Scripts/exe_workflows.sh : -- The shell script that is called.Take two params

Param 1 -- SEC_SC_PSI  – Folder Name

Param 2 -- w_Sec_SC_PSI_Intransit_STG --Workflow Name1) Job execution having flat file. 

 /inforprdsec/data/Scripts/exe_workflow_flat_file.sh ILX_TMP_EMEA_OVERDUE_DEMAND.ipfSEC_EMEA_Miscellaneous w_SEC_EMEA_Miscellaneous_Tmp_Emea_Overdue_Demand

 /inforprdsec/data/Scripts/exe_workflow_flat_file.sh ---------- Shell script usedTakes 3 param

Param 1 --- ILX_TMP_EMEA_OVERDUE_DEMAND.ipf --Parameter File of the Job

Param 2 --SEC_EMEA_Miscellaneous --Folder

Param 3 -- w_SEC_EMEA_Miscellaneous_Tmp_Emea_Overdue_Demand --Work flow name

What happens in the shell script on High level isBased on the parameter file name this select is runselect FIELD02_VALUE_TXT ||'~'|| FTP_DIRECTORY_NAM ||'~'||

FTP_ARCHIVE_DIRECTORY_NAM || '~' || EMAIL_MISSING_FILES_NAM ||'~'||FTP_FILE_EXTENSION_NAM ||'~'|| FIELD03_VALUE_TXT fromtb_etl_ctl_parameters

where PARAMETERFILE_NAME = '$PARAMETER_FILE_NAME';

Page 32: Autosys Details

8/3/2019 Autosys Details

http://slidepdf.com/reader/full/autosys-details 32/33

TCS GE ConfidentialTCS GE Confidential

Then this section

FILE_PAT_TMP=`awk -F~ '{print $1}' $PARAM_TMP` --- FIELD02_VALUE_TXT ---Pattern Of the File 

SOURCE_DIR=`awk -F~ '{print $2}' $PARAM_TMP` --- FTP_DIRECTORY_NAM ---Source Direcotry 

TARGET_DIR=`awk -F~ '{print $3}' $PARAM_TMP` ---FTP_ARCHIVE_DIRECTORY_NAM – Target Directory

MAIL_LIST=`awk -F~ '{print $4}' $PARAM_TMP` ---EMAIL_MISSING_FILES_NAM --- Mailing List

FILE_EXTN=`awk -F~ '{print $5}' $PARAM_TMP` ---FTP_FILE_EXTENSION_NAM -- File extension 

FTP_FLAG=`awk -F~ '{print $6}' $PARAM_TMP` --- FIELD03_VALUE_TXT – FTPflag decides if FTP from DI is req or not

If FTP from DI is req then following is run

${SEC_SCRIPTS}/ftp_get_flat_files.sh ${PARAMETER_FILE_NAME}"${FILE_PAT}"

Shell Script Name Parameter File Name File Pattern

This shell script basically opens FTP session Pulls file and does Archiveof the file in DI server.

After the FTP part is complete below is done

# Loop through the files in the directory 

for CURR_FILE in `ls ${FILE_PAT} 2>/dev/null` --- Loops thru the filelist to be loaded 

do

FCNT=`expr $FCNT + 1` --- Takes care Count

Page 33: Autosys Details

8/3/2019 Autosys Details

http://slidepdf.com/reader/full/autosys-details 33/33

TCS GE Confidential

------------- Below part updates the Parameter Table withFile name and file count 

${SQLPLUS} -s $DBUSER/$DBPASS@$DBNAME <<EOF >${ORA_OUT} ---- All these $dbuser things come

from Env_Var_Config.sh WHENEVER SQLERROR EXIT 1;

WHENEVER OSERROR EXIT 1;

update tb_etl_ctl_parameters set FIELD04_VALUE_TXT ='$CURR_FILE' , FIELD05_VALUE_TXT = '$FCNT' wherePARAMETERFILE_NAME = '$PARAMETER_FILE_NAME';

commit;EXIT;

EOF 

And then runs the Informatica Workflow 

# Call the workflow to execute the job

echo "Invoking the workflow

${INFORMATICA_FOLDER}.${WORKFLOW_NAME}" >> ${LOG_FILE}$SEC_SCRIPTS/exe_workflows.sh $INFORMATICA_FOLDER$WORKFLOW_NAME >> ${LOG_FILE}