Upload
rajesh716
View
265
Download
4
Embed Size (px)
Citation preview
8/3/2019 Autosys Details
http://slidepdf.com/reader/full/autosys-details 1/33
TCS GE ConfidentialTCS GE Confidential
AutoSys is an automated job control system for scheduling, monitoring andreporting.
An AutoSys job is any single command, executable, script, or NT batch file.
AutoSys job definition contains a variety of qualifying attributes, including the
conditions specifying when and where a job should be run.
There are the two methods to create job definitions:Using the AutoSys Graphical User Interface (GUI).Using the AutoSys Job Information Language (JIL) through a
command-line interface.
Introduction to AutoSys
8/3/2019 Autosys Details
http://slidepdf.com/reader/full/autosys-details 2/33
TCS GE ConfidentialTCS GE Confidential
NameStarting Conditions
AlarmsRestart Conditions
Current statusStart timeEnd timeExit code
Command to executeFile to source
Machine to run onStandard output filesStandard input file
File NameMinimum File size
Watch Interval
Box
TIMEDEPENDS ON
JOB
COMMAND
FILEWATCHER
Run Commandon Machine
Watch for File
CONTAINER
Definition
Current State
Autosys Job File Details
8/3/2019 Autosys Details
http://slidepdf.com/reader/full/autosys-details 3/33
TCS GE ConfidentialTCS GE Confidential
Box jobs are containers that hold other jobs.A box job can be used to organize and control process flow. The box itself performsno actions, although it can trigger other jobs to run. An important feature of this typeof job is that boxes can be put inside of other boxes.Default Box Job Behavior:Some important rules to remember about boxes are
Jobs run only once per box execution.Jobs in a box will start only if the box itself is running.As long as any job in a box is running, the box remains in RUNNING state; the boxcannot complete until all jobs have run.By default, a box will return a status of SUCCESS only when all the jobs in the boxhave run and the status of all the jobs is "success.By default, a box will return a status of FAILURE only when all jobs in the box haverun and the status of one or more of the jobs is "failure."Unless otherwise specified, a box will run indefinitely until it reaches a status ofSUCCESS or FAILURE.Changing the state of a box to INACTIVE changes the state of all the jobs in the boxto INACTIVE.
8/3/2019 Autosys Details
http://slidepdf.com/reader/full/autosys-details 4/33
TCS GE ConfidentialTCS GE Confidential
Defining Autosys Jobs
• AutoSys Graphical User Interface:
The AutoSys GUI allows you to interactively set the attributes thatdescribe when, where, and how a job should run.
• Job Information Language:
AutoSys JIL is a specification language, with its own syntax, that isused
to describe when, where, and how a job should run.
Job Information Language JIL) is a scripting language whichprovides a
way to specify how AutoSys jobs should behave. JIL scripts containone
or more JIL sub-commands and one or more attribute statements;these
elements constitute a job definition.
8/3/2019 Autosys Details
http://slidepdf.com/reader/full/autosys-details 5/33
TCS GE ConfidentialTCS GE Confidential
Various states in autosys:
• INACTIVE: The job has not yet been processed. Either the job has never been run, or its statuswas intentionally altered to “turn off” its previous completion status
• ACTIVATED: The top-level box that this job is in is now in the RUNNING state, but the job itselfhas not started yet.
• STARTING: The event processor has initiated the start job procedure with the Remote Agent.
• RUNNING: The job is running. If the job is a box job, this value simply means that the jobs withinthe box may be started (other conditions permitting). If it is a command or file watcher job, thevalue means that the process is actually running on the remote machine.
•
SUCCESS: The job exited with an exit code equal to or less than the “maximum exit code for success.” By default, only the exit code “0” is interpreted as “success.” If the job is a box job, thisvalue means that all the jobs within the box have finished with the status SUCCESS (the default),or the “Exit Condition for Box Success” evaluated to true
• FAILURE: The job exited with an exit code greater than the “maximum exit code for success.” Bydefault, any number greater than zero is interpreted as “failure.” AutoSys issues an alarm if a jobfails
•
TERMINATED: The job terminated while in the RUNNING state. A job can be terminated if a usersends a KILLJOB event or if it was defined to terminate if the box it is in failed. If the job itself fails,it has a FAILURE status, not a TERMINATED status. A job may also be terminated if it hasexceeded the maximum run time (term_run_time attribute, if one was specified for the job), or if itwas killed from the command line through a UNIX kill command. AutoSys issues an alarm if a jobis terminated.
8/3/2019 Autosys Details
http://slidepdf.com/reader/full/autosys-details 6/33
TCS GE ConfidentialTCS GE Confidential
• RESTART: The job was unable to start due to hardware or application problems, andhas been scheduled to restart.
• QUE_WAIT: The job can logically run (that is, all the starting conditions have been
met), but there are not enough machine resources available.•
ON_HOLD: This job is on hold and will not be run until it receives theJOB_OFF_HOLD event.
• ON_ICE: This job is removed from all conditions and logic, but is still defined toAutoSys. Operationally, this condition is like deactivating the job. It will remain on ice
until it receives the JOB_OFF_ICE event.
8/3/2019 Autosys Details
http://slidepdf.com/reader/full/autosys-details 7/33TCS GE Confidential
TCS GE Confidential
Status Abbreviations: • The following table lists the abbreviations used in the ST (status)
column of the autorep report, and gives the status for each
abbreviation.
AC - ACTIVATEDFA - FAILUREIN - INACTIVEOH - ON_HOLD
OI - ON_ICEQU - QUE_WAITRE - RESTARTRU - RUNNINGST - STARTINGSU - SUCCESS
TE - TERMINATED
8/3/2019 Autosys Details
http://slidepdf.com/reader/full/autosys-details 8/33TCS GE Confidential
TCS GE Confidential
Note:
The difference between "on hold" and "on ice" is that when an "on hold" job istaken off hold, if its starting conditions are already satisfied, it will be
scheduled to run, and it will run. On the other hand, if an "on ice" job is taken"off ice," it will not start, even if its starting conditions are already satisfied.This job will not run until its starting conditions reoccur.The other major distinction is that jobs downstream from the job that is "onice" will run as though the job succeeded. Whereas, all dependent jobs do not
run when a job is on "on hold"—nothing downstream from this job will run.
Some basic Autosys Commands:
This will display a list of jobs with complete details with box/jobname, last/latest
run date & time, status, exit code, etc.autorep –j <job_name>
example:
autorep –j gpag0201f_orders_load_daily_inf
8/3/2019 Autosys Details
http://slidepdf.com/reader/full/autosys-details 9/33
8/3/2019 Autosys Details
http://slidepdf.com/reader/full/autosys-details 10/33TCS GE Confidential
TCS GE Confidential
SAMPLE JIL FILE
• Sample JIL file saved as *.jil extension
8/3/2019 Autosys Details
http://slidepdf.com/reader/full/autosys-details 11/33TCS GE Confidential
TCS GE Confidential
STEPS TO RAISE A CASE FOR AUTOSYS RELATED ISSUE
The following steps is abt how to raise a request on kintana for your jobs(New/change/delete/One-Time)!!
1) Go to http://insidegis.ge.com 2) Login with your SSO Id / password3) On the left hand side under 'Change Management', there is a link 'Modify a
Service/Asset'. Clicking on this link refreshes and brings up a new page.4) The first field is 'Service' which is a drop down field. Choose 'Server' as applicable to
you.5) Enter you contact information/Desired Start Time/Desired Completion Time.
6) Enter the 'Change Type' . Most of the cases it will be 'Standard'.7) In 'Affected Scope', you need to enter the server system name(gcoasys01).8) In Change Reasons, enter the type of request.
a) Production Scheduling Request - New (For a new Job)b) Production Scheduling Request - Change (For changing a job parameter)c) Production Scheduling Request - Special One Time Process (For forcestarting/putting a job ON/OFF ICE/HOLD)
d) Production Scheduling Request - Remove (For deleting a job)9) You can also enter any free text in the 'Change Description' area.10) Click on Submit and note down the ticket number.
8/3/2019 Autosys Details
http://slidepdf.com/reader/full/autosys-details 12/33
TCS GE ConfidentialTCS GE Confidential
Monitoring Autosys Jobs
Option I• GUI Link For Autosys
• http://gcoasys02.corporate.ge.com/autosys/login
• Job Monitoring & Reports:
• GUI to Monitor the Jobs
• Reporting Through GUI
• Dashboard for Functional Users: In Progress
Option II
• Autosys Command Prompt
• Go to Remote Login Server > Start > Unicenter Autosys Cmd Prompt
> Enter the command ―autorep –j autosys jobname‖
8/3/2019 Autosys Details
http://slidepdf.com/reader/full/autosys-details 13/33
TCS GE ConfidentialTCS GE Confidential
FAQ’s
• What if I don't want to run the ETL cycle for the given day?
• How do I delay a box?
• How do I delay the job within a box?
• How do I cancel a box?
• How do I cancel a job within a box?
• How do I run a failed/errored out job or box on the same day?• How do re run any job/box on the same day?
8/3/2019 Autosys Details
http://slidepdf.com/reader/full/autosys-details 14/33
TCS GE ConfidentialTCS GE Confidential
How to contact Autosys team?
There are several ways to reach the Autosys team:1. Here in this community you could enter a case and direct it to any expert that lists Autosys in
their expertise but only for general questions. We would prefer that you use the proceduredocumented in Request for Service sub-category under the Process category in thiscommunity. See document there titled "How to Use RFC for Production Scheduling requests"to follow the directions to have a request ticket entered in our system. The documentaddresses Autosys job/schedule requests as either New, Change or Remove that go to theschedulers or "One Time" requests that go to the production monitors on duty.
2. If you have a problem, a question about Autosys or need to make a request, you follow adifferent path. Still using http://insidegis.ge.com/ you can select "Ask A Question" (to ask aquestion or make a request), select "Modify a Service/Asset" under "Change Management" (torequest a remote agent install or license key) or select "Incident Management" to enter anissue. There's a new document in this community's Process section called "How to ask aquestion, report a problem or submit a non-schedule change request" that has all the details.See the attachment to this FAQ for some further details on submitting a request or reporting aproblem. GIS Help Desk is also available via *222-6666. The InsideGIS portal lists the various
ways to reach the help desk.
3. Via e-mail there's a distribution list @CORP GIS Autosys Support that can be used to reachthe entire team. However, once a request has been sent and you want to contact the Autosysschedulers to forward documents, etc., you can use the distribution list @CORP GIS AutosysSchedulers to reach them. For installs and license keys you can use the DL @CORP GISAutosys Installers. 4) For Sametime access just take any of the DL's listed above, remove the"@" and that is a group name you can add to your list
8/3/2019 Autosys Details
http://slidepdf.com/reader/full/autosys-details 15/33
TCS GE ConfidentialTCS GE Confidential
– What is Autosys?
– Limitations of BO DI Scheduler
– Distinctive Features
– Offerings by GIS
– GIS Customers
8/3/2019 Autosys Details
http://slidepdf.com/reader/full/autosys-details 16/33
TCS GE ConfidentialTCS GE Confidential
– What is Autosys?
CA-Autosys is a batch monitoring and scheduling tool for the
Open Systems environment.
It uses several supplemental products to provide a complete
cross platform scheduling and monitoring solution for GIS.
8/3/2019 Autosys Details
http://slidepdf.com/reader/full/autosys-details 17/33
TCS GE ConfidentialTCS GE Confidential
Limitations of BO DI Scheduler:
– Jobs can only be scheduled based on Day/Time
– Jobs can not be scheduled on the Success or Failure of Particular Job
– Lots of manual intervention is needed to complete the Data Warehouse
Load
– Continuous Monitoring efforts required
– Longer Job Cycle Time due to Manual Intervention
8/3/2019 Autosys Details
http://slidepdf.com/reader/full/autosys-details 18/33
TCS GE ConfidentialTCS GE Confidential
• Distinctive Features of Autosys:
– Increased Performance: Improved performance using multi-threaded processing to enable more jobs to run in parallel.
– No Manual Intervention: Due to scheduling capability based on
Time/day, file arrival and job dependency, the entire loadingprocess can be automated.
– Unparalleled Reliability and Scalability: Scales tens ofthousands of jobs easily over hundreds of multiple platform
machine per night, while maintaining its hallmark reliability.
8/3/2019 Autosys Details
http://slidepdf.com/reader/full/autosys-details 19/33
TCS GE ConfidentialTCS GE Confidential
• Distinctive Features of Autosys:
– Enterprise-wide cross Platform Job Management: Simplifies crossplatform job management through direct integration communication toUnicenter Job management Solutions
8/3/2019 Autosys Details
http://slidepdf.com/reader/full/autosys-details 20/33
TCS GE ConfidentialTCS GE Confidential
• Offerings of GIS for Autosys• High availability: Provides High Availability of Autosys Scheduling
Server with fell over Servers (Cincinnati and Alpharetta)
– 24/7 Support Team: Provides 24 * 7 support team for monitoring ofAutosys Jobs
– Web Based User interface: Provides single point of control andadministration of jobs and reduces IT learning and operational costs
– No Additional Cost to GE Security : GIS offers the Autosys serviceswith no additional cost of Hardware/Software/Services
– Job Scheduling Flexibility: Job scheduling and Monitoring flexibility isoffered by GIS to the Application team. The application team may or may
not choose the option for Scheduling and Monitoring the Jobs.
8/3/2019 Autosys Details
http://slidepdf.com/reader/full/autosys-details 21/33
TCS GE ConfidentialTCS GE Confidential
• Pre-requisite for Autosys Scheduling:• Oracle SQL*Net V2.3 and Oracle TCP/IP Adapter are required
• Installer must have access to the server• The TNS entry to be made for two Autosys Oracle databases
• Autosys Agent to be Installed on Server
• Getting Started:
• http://data.supportcentral.ge.com/upload/16285/doc_172780.doc
8/3/2019 Autosys Details
http://slidepdf.com/reader/full/autosys-details 22/33
TCS GE ConfidentialTCS GE Confidential
• Types Of Autosys Jobs:
• Command Job: This is what is commonly referred to as “a job”. The
command can be a shell script, an executable program, a file transfer,etc. When the job runs, there is the execution of a specific command onthe client machine.
• Box Job: The box job or box is a container of other jobs. It can be usedto organize and control process flow. The box itself performs no actions,
although it can trigger other jobs to run.
• File Watcher Job: A file watcher job starts a process that monitors forthe existence and size of a specific file. Once that file reaches thespecified minimum size, and is no longer growing in size for a specifiedtime, the file watcher job completes successfully, indicating that the filehas arrived.
8/3/2019 Autosys Details
http://slidepdf.com/reader/full/autosys-details 23/33
TCS GE ConfidentialTCS GE Confidential
• Process to Schedule Jobs through Autosys:1. Identify the Jobs to be Schedule
2. Decide upon the Dependency3. Logical grouping of the Jobs (Box)
4. Creating JIL Files
5. Raise a Request to GIS team to Schedule them
OR
Get Access to the GUI tool to schedule them
8/3/2019 Autosys Details
http://slidepdf.com/reader/full/autosys-details 24/33
TCS GE ConfidentialTCS GE Confidential
Sensing Production Jil Files
8/3/2019 Autosys Details
http://slidepdf.com/reader/full/autosys-details 25/33
TCS GE ConfidentialTCS GE Confidential
Autosys Job Monitoring from DI server
Log in to Autosys Command Prompt
Start->All Programs->Computer Associates ->Unicenter->UnicenterAutosys ->Unicenter Autosys Command Prompt
8/3/2019 Autosys Details
http://slidepdf.com/reader/full/autosys-details 26/33
TCS GE ConfidentialTCS GE Confidential
• GUI Link For Autosys
• http://gcoasys02.corporate.ge.com/autosys/login
• Job Monitoring & Reports:• GUI to Monitor the Jobs
• Reporting Through GUI
• Dashboard for Functional Users: In Progress
8/3/2019 Autosys Details
http://slidepdf.com/reader/full/autosys-details 27/33
TCS GE ConfidentialTCS GE Confidential
Gui Monitoring of Jobs
Various Views ( Prod/QA/Dev) with jobs under them
8/3/2019 Autosys Details
http://slidepdf.com/reader/full/autosys-details 28/33
TCS GE ConfidentialTCS GE Confidential
Various Boxes giving Status of Jobs in them ( Green –Success,Yellow –Ice, Red- Failure )
Boxes are used to bring together similar jobs
8/3/2019 Autosys Details
http://slidepdf.com/reader/full/autosys-details 29/33
TCS GE ConfidentialTCS GE Confidential
Jobs in Boxes can be seen by double clicking on Box which gives job details
8/3/2019 Autosys Details
http://slidepdf.com/reader/full/autosys-details 30/33
TCS GE ConfidentialTCS GE Confidential
Link to Create Ticket for Gui Account
http://gis.kintana.corporate.ge.com/itg/web/knta/crt/RequestDetail.jsp?REQUEST_ID=5070271
8/3/2019 Autosys Details
http://slidepdf.com/reader/full/autosys-details 31/33
TCS GE ConfidentialTCS GE Confidential
To run Informatica job from Autosys we have two shell scripts1) Normal job not having flat file load
/inforprdsec/data/Scripts/exe_workflows.sh SEC_SC_PSI w_Sec_SC_PSI_Intransit_STG /inforprdsec/data/Scripts/exe_workflows.sh : -- The shell script that is called.Take two params
Param 1 -- SEC_SC_PSI – Folder Name
Param 2 -- w_Sec_SC_PSI_Intransit_STG --Workflow Name1) Job execution having flat file.
/inforprdsec/data/Scripts/exe_workflow_flat_file.sh ILX_TMP_EMEA_OVERDUE_DEMAND.ipfSEC_EMEA_Miscellaneous w_SEC_EMEA_Miscellaneous_Tmp_Emea_Overdue_Demand
/inforprdsec/data/Scripts/exe_workflow_flat_file.sh ---------- Shell script usedTakes 3 param
Param 1 --- ILX_TMP_EMEA_OVERDUE_DEMAND.ipf --Parameter File of the Job
Param 2 --SEC_EMEA_Miscellaneous --Folder
Param 3 -- w_SEC_EMEA_Miscellaneous_Tmp_Emea_Overdue_Demand --Work flow name
What happens in the shell script on High level isBased on the parameter file name this select is runselect FIELD02_VALUE_TXT ||'~'|| FTP_DIRECTORY_NAM ||'~'||
FTP_ARCHIVE_DIRECTORY_NAM || '~' || EMAIL_MISSING_FILES_NAM ||'~'||FTP_FILE_EXTENSION_NAM ||'~'|| FIELD03_VALUE_TXT fromtb_etl_ctl_parameters
where PARAMETERFILE_NAME = '$PARAMETER_FILE_NAME';
8/3/2019 Autosys Details
http://slidepdf.com/reader/full/autosys-details 32/33
TCS GE ConfidentialTCS GE Confidential
Then this section
FILE_PAT_TMP=`awk -F~ '{print $1}' $PARAM_TMP` --- FIELD02_VALUE_TXT ---Pattern Of the File
SOURCE_DIR=`awk -F~ '{print $2}' $PARAM_TMP` --- FTP_DIRECTORY_NAM ---Source Direcotry
TARGET_DIR=`awk -F~ '{print $3}' $PARAM_TMP` ---FTP_ARCHIVE_DIRECTORY_NAM – Target Directory
MAIL_LIST=`awk -F~ '{print $4}' $PARAM_TMP` ---EMAIL_MISSING_FILES_NAM --- Mailing List
FILE_EXTN=`awk -F~ '{print $5}' $PARAM_TMP` ---FTP_FILE_EXTENSION_NAM -- File extension
FTP_FLAG=`awk -F~ '{print $6}' $PARAM_TMP` --- FIELD03_VALUE_TXT – FTPflag decides if FTP from DI is req or not
If FTP from DI is req then following is run
${SEC_SCRIPTS}/ftp_get_flat_files.sh ${PARAMETER_FILE_NAME}"${FILE_PAT}"
Shell Script Name Parameter File Name File Pattern
This shell script basically opens FTP session Pulls file and does Archiveof the file in DI server.
After the FTP part is complete below is done
# Loop through the files in the directory
for CURR_FILE in `ls ${FILE_PAT} 2>/dev/null` --- Loops thru the filelist to be loaded
do
FCNT=`expr $FCNT + 1` --- Takes care Count
8/3/2019 Autosys Details
http://slidepdf.com/reader/full/autosys-details 33/33
TCS GE Confidential
------------- Below part updates the Parameter Table withFile name and file count
${SQLPLUS} -s $DBUSER/$DBPASS@$DBNAME <<EOF >${ORA_OUT} ---- All these $dbuser things come
from Env_Var_Config.sh WHENEVER SQLERROR EXIT 1;
WHENEVER OSERROR EXIT 1;
update tb_etl_ctl_parameters set FIELD04_VALUE_TXT ='$CURR_FILE' , FIELD05_VALUE_TXT = '$FCNT' wherePARAMETERFILE_NAME = '$PARAMETER_FILE_NAME';
commit;EXIT;
EOF
And then runs the Informatica Workflow
# Call the workflow to execute the job
echo "Invoking the workflow
${INFORMATICA_FOLDER}.${WORKFLOW_NAME}" >> ${LOG_FILE}$SEC_SCRIPTS/exe_workflows.sh $INFORMATICA_FOLDER$WORKFLOW_NAME >> ${LOG_FILE}