Upload
nguyencong
View
239
Download
7
Embed Size (px)
Citation preview
Jobtrac Conversion – The good, the bad & the ugly.
Presented by Paul Robins
Reason for choosing Control-M
Single Enterprise GUICross datacentre schedule integrationFlexible exception alertingConditional scheduling (e.g. RC based triggers)Server Sysout analysisServer Sysout lifecycle managementViewing sysout via GUI
Volumes…
2,500 Production JCL members converted7,900 Tech (DBA) JCL members converted3,700 zOS job schedules converted600 x-platform (MS-Win) job schedules converted
Peripheral Challenges…Operator TrainingUser Training (~50 users at remote site)Source Control management (SCLM)Lack of testing environmentsInstability in conversion codeIncorrect conversion manualsNo gateway from Non-Prod mainframe to DTLConversion of ‘JCL Build’ automationEvent Management setupRACF setup – User administrationClient rolloutOPSMVS to Control-OChange Moratorium
Tech Challenges – Bugs & EnhancementsISS03523214 – XML output fails validation by deftable utilityISS03534321 – Jobs with DUMMY overlib and INSTREAM JCL=Y are
submitting JCLISS03534344 – Conversion has not produced required MAYBE conditionsISS03542351 – X-Platform command line statements with invalid variable
concatenationISS03548180 – Conversion PREVENT-NCT2 parm overrides product
defaultISS03551469 – JCLLIB and DOCLIB concatentation featureISS03499256 – JOB1 DAXML DD missing, DEFAULTS member
&DEFTYPE parm missingISS03501128 – Multiple abends. Unsupported new features.ISS03505113 – conditions not in JOBNAME-OK format
Tech Challenges – Alternate Member NameCA-Jobtrac provides a feature to provide an alternate
member name to the scheduler event name. This allows JCL member names to be consistent across environments while the event name reflects the different environment prefixes.
e.g.
This is catered for by the conversion by using the INSTREAM JCL feature:
Tech Challenges – JCLLIB ConcatenationCA-Jobtrac offers a JCL and DOC library concatenation
feature to allow non-prod jobs to search up through the development lifecycle for JCL members.
To our surprise this can be replicated in Control-M!Specify the libraries and sequence in IOA PARM
member IOADSNL. This assigns a logical DD which is used as your MEMLIB reference in job schedules. (Wish\exit mod required).
Tech Challenges JCL TruncationIn many cases Control-M auto-edit concatentation is more
verbose than CA-Jobtrac JMF concatenation.Manual intervention may be required to remediate truncation
issues.Use a JCL reformatter to convert JCL to
well-formed JCL prior to conversion (e.g. ASG-JOB/SCAN).
Beware IOA Col 72 – 80 processing changes introduced in 6.3.04
Tech Challenges – Additional JCL changesThe CA-Jobtrac JCL members did not have any JOBLIB
statements. This was inserted by Jobtrac during submission.
The CA-Jobtrac CA11NR DD statements were not converted to CTRNORST by the conversion utility.
ASG-JOB\SCAN was used to make these changes
Tech Challenges – Temporary JCL overridesCA-Jobtrac provides an excellent JCL override feature
whereby a temporary member is allocated for changes to be applied to, which is only associated with the instance of the job on the AJF. This reduces risk that temp changes may be referenced by subsequent job orders.
Due to use of the INSTREAM JCL feature the standard OVERLIB is not available. This could however be built into the JCLLIB concatenation if required.
Tech Challenges: DON’T CORRUPT THE DATELIB!Jobtrac variables are held in a PDS
member which can be converted do a Control-M variable member.
Jobtrac variable members can contain referrals to other variable members.
e.g. Member JMFSYM01 contains the statement: JLOC002:JOBTRAC.PROD.DATELIB(JMFSYM02)
The CONTROL-M conversion utility will convert ALL MEMBERS referred to in this manner.
If you take a copy of the member for conversion, remove all referrals to other members.
Tech Challenges – CPM supportCA-Jobtrac has a facility called Critical Path Manager (CPM)
which is similar to Control-M\BIMConversion of CPM alerting is not included in the BMC
utilities. This is a manual task.
Tech Challenges: NCT2 PreventionJobtrac\CA11 has the capability to perform NCT2 prevention
by deleting duplicate DSNs prior to job execution.IMHO, NCT2 prevention is Bad.If NCT2 prevention has been in place for an extended period
of time, you may find that you have a large number of failures post-conversion unless you turn it on in Control- M.
Tech Challenges: Reserved Variable UsageThere is a possibility that Jobtrac JMF variables, when
converted to autoedit variables, will match Control-M reserved system variables.
Any such variables will need to be remediated manually.
e.g. JMF variable &:RDATE is a user defined variable. Autoedit %%RDATE is a reserved system variable.
Tech Challenges – MAXWAIT valueThe conversion will only provide for a default MAXWAIT
value.This may not be suitable for your entire job library (e.g.
Cyclic jobs will probably be MAXWAIT 0).Select the most prevalent value for your jobs and retrofit any
exceptions.
Tech Challenges – DUMMY jobs default to MFIn Jobtrac there is nothing to identify what platform a Dummy
job should run on.The conversion tool defaults Dummy jobs to mainframe
definitions.This can result in x-platform job streams having mainframe
jobs in the middle.If only converting x-platform jobs, this could result in jobs
being lost.
Tech Challenges – Cross Data Centre job streamsThe conversion tool is unable to identify conditions that need
to be passed between data centres.Because of the prevalence of cross data centre job streams
in our schedules we chose to make ALL conditions Global conditions.
This will need to be reviewed at a later date due to the amount of gateway traffic it creates.
Tech Challenges – Jobtrac commentsIf a Jobtrac job definition contains a comment in the middle
the conversion utility will produce two schedule definitions. One definition was usually found to be invalid and would fail.
Tech Challenges: Shout Severity RetrofitJobtrac does not provide a facility for shout severities.A REXX intercepted all job alerts and downgraded alerts
based on an ‘exclusion list’.The conversion process only provides for 1 default severity.The default shout severity was set to ‘V’ and manually
downgraded according to the exclusion list.
Tech Challenges – Inconsistent Start Day Of WeekDue to a legacy setup issue, our site has a different Start
Day Of Week for Control-M\zOS and Control-M\Server.When converting Windows jobs the conversion tool
references the Mainframe IOA parms for the Start Day of Week.
This was overcome by setting up an alternate IOA Parm and alternate conversion JCL with the Server date setting.
Tech Challenges – Missing GROUP statementsWhen converting Jobtrac schedules that contain Non-
Mainframe job definitions, all definitions will need to be converted to XML and loaded through Enterprise Manager.
The resultant XML file did not have the required GROUP statements. These were added in manually via a mass update of the JOB tag.
Tech Challenges – Condition naming standardsThe Jobtrac conversion utility will probably not deliver
condition names that meet your site standard.These will have to be modified.BMC provided a PTF to produce conditions in the format
JOBNAME-OK which we easily modified to %%GROUP- JOBNAME-OK during the XML phase of the conversion.
When performing mass changes of conditions be sure to consider Maybe conditions and Globals (if exist).
Tech Challenges – !MANUAL conditionsAny IN conditions that are not matched to OUT conditions
during the conversion process are flagged with a !MANUAL prefix.
These conditions need to be remediated.For this reason it is advisable not to break down the
schedule conversion too much.
Tech Challenges – LATESUB shoutsFor any job that has a FROM TIME value, the conversion
will produce a LATESUB shout on the job.The LATESUB value will match the FROM TIME value.This can result in erroneous alerts and should be modified.
Tech Challenges – ctmdefine errorThere is a known bug that the EM incorrectly flags
definitions containing both DATES and WDAYS/WCAL parameter as invalid.
Contact BMC Support for recommended workaround.
Tech Challenges: Agent installationIf your Jobtrac installation is running x-platform jobs you
must identify all Nodes that jobs are submitted to and install a Control-M Agent.
Ensure that the Agent has the same permissions, or runs under the same user, as the Jobtrac jobs.
Tech Challenges – Newday timeIn order to standardise your site you may need to run
Control-M with a different ‘Newday’ time to Jobtrac.If this is the case the conversion process should cater for
any shifts in scheduling days.However, you will need to carefully consider the running of
jobs at cutover time between the end of the last Jobtrac day and the start of the first Control-M day.
Tech Challenges – APIsWe have a number of ISPF and REXX processes using the
Jobtrac API that have been converted to Control-M.Discovery of these processes can be problematic.During the discovery phase of your conversion, ensure that
Control-M can replicate all of the functions that Jobtrac allows via API.
In some cases API functions were converted to Control-O functions.
Tech Challenges – Interact with DesktopWe have a number of legacy Windows jobs which have
been designed to raise ‘pop-up’s to the server console when an error is encountered.
Under our default install of Control-M\Desktop these errors are not visible via MS RDP.
We are currently investigating implementing the Control-M ‘Interact With Desktop’ capability to provide this function.
Tech Challenges – MASTER conversionAs well as providing a ‘User Daily’ style facility, Jobtrac also
provides a ‘MASTER’ schedule definition which can contain scheduling data for job scheduling tables.
The contents of MASTER are converted into STC schedules which submit the CTMJOBPR utility to order the tables.
We chose to deschedule these and add the tables to standard User Daily jobs.
Tech Challenges – Scheduling with no single point of truthWe discovered with our Jobtrac definitions, that the individual job
definition did not reflect the truth about scheduling of the job.e.g.JOBA might have a definition that is scheduled 7 days per week.However this job might be added to the AJF by User Daily 1 on
weekdays, and by User Daily 2 on Saturdays, and not run on Sundays.
This situation required significant remediation.
Tech Challenges – Job Schedule CorruptionISS03649976Due to the previous mentioned usage of INSTREAM JCL we
encountered a bug in EM whereby mainframe schedules can be corrupted during Download to EM.
This resulted in jobs failing with NOMEM errors.This has the potential to impact every job in a schedule
table.See BMC Support for the fix.
Tech Challenges – X-Platform docmemsAs Jobtrac schedules x-platform jobs from the mainframe,
the documentation members also reside on the mainframe.
Members will need to be downloaded to a location where the Control-M\Agent can read them.
The conversion will populate the DOCLIB field with a single default value specified at conversion time.
Tech Challenges – X-platform Maybe conditionsIt is important to remember that
when you are running Maybe condition processing on any given platform, it will not ‘see’ jobs on the AJF of another data centre.
This was overcome by defining dummy jobs on the predecessor platform to action the Maybe dependency and pass a standard condition to the successor data centre.
Recommendations
• Run a Proof of Concept first• Ensure that you have sufficient Jobtrac and Control-M
skills engaged with the project. Perform additional training of project staff prior to conversion if required.
• Engage with BMC. Ensure you have the correct levels of software and manuals. Request any known issues. Ensure that the conversion team are aware of the project.
• New BMC ‘amigo’ initiative.• Get Control-M\Forecast!