AutoSys

Introduction AutoSys is an automated job control system for scheduling, monitoring, and reporting. These jobs can reside on any AutoSys-configured machine that is attached to a network. An AutoSys job is any single command, executable, script, or UNIX batch file. Each AutoSys job definition contains a variety of qualifying attributes, including the conditions specifying when and where a job should be run.

In addition, AutoSys provides utilities to help you define, run, and maintain AutoSys instances and jobs. The included utilities are platform-specific; however, all platforms include the AutoSys graphical user interface (GUI) and Job Information Language (JIL). Both the GUI and JIL enable you to define, manage, monitor, and report on jobs.

What is a Job?

A definition you create that instructs the system what command, executable, or batch file to run

Note: In AutoSys JM 4.0, the Job name may be no more than 30 characters in length. However, in AutoSys JM 4.5 the Job name may be more than 80 characters in length. No spaces are permitted, but you may use dashes, underscores, and periods.

What is an Event?

Any status changes posted to the database are called Events.

The following sections discuss the main AutoSys system components

Event ServerThe Event Server, or AutoSys database, is the data repository for all system information and events. Event Server refers to the database where all the AutoSys information, events, and job definitions are stored. Occasionally, the database is called a data server, which actually describes a server instance. It is associated data space (or raw disk storage), that can include multiple databases or table spaces.

Event ProcessorThe Event Processor is the heart of AutoSys; it interprets and processes all the events it reads from the AutoSys database. Sometimes called the event demon, the Event Processor is the program that actually runs AutoSys. It schedules and starts jobs. The Event Processor continually scans the database for events to be processed. When it finds one, it checks whether the event satisfies the starting conditions for any job in the database. Based on this information, the Event Processor first determines what actions to take, then instructs the appropriate Remote Agent process to perform the actions. These actions might be the starting or stopping of jobs, checking for resources, monitoring existing jobs, or initiating corrective procedures.

Remote AgentThe Remote Agent is a temporary process started by the Event Processor to perform a specific task on a remote, or client, machine. The Remote Agent starts the command specified for a given job, sends running and completion information about a task to the Event Server, then exits. If the Remote Agent is unable to transfer the information, it waits and tries again until it can successfully communicate with the database.

Example Scenario

The example scenario in Figure-1 below and the numbered explanations that follow it illustrate the interactions between the Event Server, the Event Processor, and Remote Agents. In the example, the following UNIX command line is to be run on WorkStation_2, at the start date and time specified in the AutoSys job definition:

rm /tmp/mystuff/*

Understanding this example will help you answer many questions that might arise during your experiences with AutoSys.

Figure

Note In the example above, the three primary components are shown running on different machines. For UNIX, the Event Processor and the Event Server typically run on

the same machine.

Explanation

The following steps explain the interactions in the example scenario:

From the AutoSys Event Server, the Event Processor reads a new event, which is a start job event with a start time condition that has been met. Then the Event Processor reads the appropriate job definition

from the database and, based on that definition, determines what action to take. In the example, it runs the rm /tmp/mystuff/* command on WorkStation_2.

The Event Processor communicates with the Remote Agent on WorkStation_2. As soon as the Remote Agent receives the instructions from the Event Processor, the connection between the two processes is dropped. After the connection is dropped, the job will run to completion, even if the Event Processor stops running.

The Remote Agent performs resource checks, such as ensuring that the minimum specified number of processes are available, then “forks” a child process that will actually run the specified command.

The command completes and exits, and the Remote Agent captures the command’s exit code.

The Remote Agent communicates the event (exit code, status, etc.) directly to the Event Server. If the AutoSys database is unavailable for any reason, the Remote Agent will go into a wait and resend cycle until it can deliver the message. Only two AutoSys processes need to be running: the Event Processor and the Event Server. When these two components are running, AutoSys is fully operational. The

Remote Agent is started on a client machine once per job. As soon as the job ends and the Remote Agent sends a completion event to the database, the Remote Agent exits.

Events Sent by Users

By using the Send Event dialog, you can send execute events that affect the running of a job. These are the execute events that you can send, if you have the appropriate permissions:

• STARTJOB

• FORCE_STARTJOB

• KILLJOB

• DELETEJOB

• CHANGE_STATUS

• JOB_ON_HOLD

• JOB_OFF_HOLD

• JOB_ON_ICE

• JOB_OFF_ICE

Note: Make sure you understand the consequences of EVENTS prior to execution of the events.

In addition to sending execute events on jobs, you can schedule jobs to start at certain times or under certain conditions. When a job is scheduled to start automatically,

permissions are checked on the Remote Agent machine on which the job is to run. The AutoSys Event Processor scans the Event Server for any jobs with starting

conditions that have been met. When the starting conditions for a job are met, the Event Processor sends a STARTJOB event to the designated Remote Agent machine.

What are Alarms?

Alarms signal situations requiring your attention and are informational only. Alarms are sent through the system as an event. Jobs can be scheduled to run based on a number of conditions, but some thought is required to address incidents that require manual intervention. For example, a set of jobs could be dependent on the arrival of a file, and the file is long overdue. It is important that someone investigate the situation, make a decision, and resolve the problem. Below table shows when an alarm may get generated.

Alarm Manager : Lets you view, acknowledge, and close alarms.

AutoSys User Types

AutoSys uses the notion of these three types of users for any job:

• Owner. The user who created the job.

• Group. Any user who is in the same primary group as the owner.

• World. Every user.

AutoSys uses the ID (uid) and group ID (gid) of a job’s owner to control the following:

• Who can edit, override, or delete a job definition.

• Who can execute the UNIX command specified in a job.

The owner of a job can allow other users to edit and execute the job by setting the permissions in the job definition (discussed in the next section).

AutoSys Permission Types

By default, only the owner has edit and execute permissions on a job, and all edit and execute permissions are valid only on the machine on which the job was defined. However, the owner can grant different types of permissions when defining a job. Similar to UNIX, AutoSys associates different types of permissions with each job. Every job has the following permission types:

• Edit. Users can edit, override, or delete a job definition.

• Execute. Users can send an execute event that affects the running of a job by using the sendevent command or the Send Event dialog.

• Machine. Users logged onto a machine other than the one on which a job was created can edit or execute the job.

Note In order for a job to run on a machine other than the one on which the job was defined, the owner of that job must have an account on that machine.

Job Types and Structure

Figure 2 illustrates the structure of a job. There are three types of jobs:

1) Command 2) File Watcher & 3) Box

These job types have a majority of job attributes in common, and AutoSys treats them all similarly. The primary differences between them are the actions that are taken when the job is run.

Figure 2

As their names imply, Command Jobs execute commands, Box Jobs are containers that hold other jobs (including other boxes), and File Watcher Jobs watch for the arrival of a specified file.

Command Jobs

The Command Job is commonly thought of (and referred to) as “a job.” The command can be a shell script, an executable program, a file transfer, and so forth. When this type of job is run, the result is the execution of a specified command on a client machine. When all the starting conditions are met,

AutoSys runs this command and captures its exit code upon completion. The exit event (either SUCCESS or FAILURE) and the exit code value are stored in the database. In addition to the primary functionality.

FileWatcher Jobs

A File Watcher Job is similar to a Command Job. However, instead of starting a user-specified command on a client machine, it starts a process that monitors for the existence and size of a specific operating system file. When that file reaches a certain minimum size, and is no longer growing in size, the File Watcher Job completes successfully, indicating that the file has arrived. Using File Watcher Jobs provides a means of integrating events which are external to AutoSys into the processing conditions of AutoSys jobs. For example, a file needs to be downloaded from a mainframe, and it is expected to arrive after 2:00 a.m. After it arrives, a batch job is to be run to process it, possibly even starting a whole sequence of jobs. You could set up a File Watcher Job to start at 2:00 a.m. wait for the arrival of the specified file, then exit. You could also set up the batch job so that the completion of the File Watcher Job is its only starting condition.

Box Jobs

In the AutoSys environment, the Box Job (or box) is a container of other jobs. A Box Job can be used to organize and control process flow. The box itself performs no actions, although it can trigger other jobs to run. An important feature of this type of job is that boxes can be put inside of other boxes. When this is done, jobs related by like starting conditions (not by similar application types) can be grouped and operated on in a logical way.

Note Box Jobs are very powerful tools for organizing, managing, and administering large numbers of jobs that have similar starting conditions and/or have complex logic

flows. Knowing how and when to use boxes is often the result of some experimentation.

Starting Conditions for Box Jobs

If no other starting conditions are specified at the job level, a job within a box will run as soon as the starting conditions for the box are satisfied. If several jobs in a box do not have job-level starting conditions, they will all run in parallel. Each time any job in a box changes state, the other jobs are checked to see if they are eligible to start running. If jobs in a box have a priority attribute setting, they will be processed in order of priority, highest to lowest. Jobs inside of boxes will be run only once per box execution. If you do specify multiple start times for a job during one box processing cycle, only the first start time will be used. This prevents jobs in boxes from inadvertently running multiple times. AutoSys starts a job if the current time matches, or is later than, the start time. In addition to explicit starting conditions, jobs inside of boxes have the following implicit condition: the Box Job itself is running. This means that jobs inside of a box will start only if the Box Job itself is running. However, if a job inside a box starts and the Box Job is stopped, the started job runs to completion.

Job Definitions

Each job definition contains a variety of qualifying attributes, including the conditions specifying when, where, and if a job should be run.

The following table lists the basic file definition attributes:

Job Statuses

At any given point in time, the Event Server will display one of the following statuses for a specified job. The statuses are organized logically in the order in which they will likely be encountered:

Starting Parameters

AutoSys determines whether or not it should start a job based on the evaluation of the starting conditions (or starting parameters) defined for the job. These conditions can be one or more of the following:

• Date and time scheduling parameters are met (it is or has passed the specified date and time)

• Starting Conditions specified in the job definition evaluate to true.

• For jobs in a box, the box must be in the RUNNING State.

• The current status of the job is not ON_HOLD or ON_ICE.

• Every time an event changes any of the above conditions, AutoSys finds all the jobs that might be affected by this change, and determines whether or not to start them.

Note It is very important to keep in mind the above four conditions. In order for a job to start, all defined starting conditions must be true.

• Be aware that for a job in a box to start, the Box Job must be running. Placing a job in a box means that the job inherits all the starting conditions of the Box Job. It also means

that if there are no additional conditions on the job, it will be started as soon as the box is started. Also, a job runs only once per box execution. By default, there is no concept of

sequential job processing in a box. For example, if there are three jobs inside a box, and none of them has any additional conditions, then when the box is started, all three jobs

will be started. To implement a sequence within a box, you must specify additional starting conditions for each job. For example, “Job1” may have no starting conditions, while

“Job2” is dependent on the completion of “Job1”, and “Job3” is dependent on the completion of “Job2”. Be aware that if this scenario were implemented and “Job2” were placed

ON_ICE, then “Job1” and “Job3” would start simultaneously as soon as the box they are in started running. (Jobs that are dependent on a job that is ON_ICE run as if that

starting condition has been satisfied).

What Happens when a Box Runs

As soon as a box starts running, all the jobs in the box (including sub-boxes) change to status ACTIVATED, meaning they are eligible to run. (Because of this, jobs in boxes do

not retain their statuses from previous box cycles.) Then each job is analyzed for additional starting conditions. All jobs with no additional starting conditions are started, without

any implied ordering or prioritizing. Jobs with additional starting conditions remain in the ACTIVATED state until those additional dependencies have been met. The box

remains in the RUNNING state as long as there are activated or running jobs in the box. If a box is terminated before a job in it was able to start, the status of that job will change

directly from ACTIVATED to INACTIVE.

**To Be Continued …..

!!! Note: If in doubt please consult the shift lead before executing anything on AutoSys. ** I will be updating this Doc. With more advanced info on AutoSys shortly. Team members are encouraged to modify and improve this Doc.

Thanks

Jasmeet Singh Arora.