Making It Hard for Those IMS Problems to Hide - IMS UG March 2013 DC_JAX_PA

IBM Software Group

Jeff MaddixIBM Silicon Valley [email protected]

Making It Hard for Those IMS

Problems to Hide

Session Number 3262A

1

IBM Corporation, 2012c

Making it Hard for those IMS Problems to Hide

2

Please note

IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM’s sole discretion.

Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision.

The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. The development, release, and timing of any future features or functionality described for our products remains at our sole discretion.

Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user’s job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.



3

Making it Hard for those IMS problems to Hide - Agenda

� Section 1: MVS Setup Recommendations

� Section 2: IMS Setup Recommendations

� Section 3: Preservation of Standard Documentation

� Section 4: Manual Dump Intervention

� Section 5: Situation Specific Diagnostics



4

Section 1.

MVS Setup Recommendations



5

MVS Setup RecommendationsMaximize System Trace Table Size

�Maximize System Trace Table Size.

� Advantages:

� Extremely valuable for a large variety of problem types such at ABENDs, loops, and performance issues.

�Setup

� Set MVS System Trace table size to 999K or 3M in z/OS 1.9 or higher

–Default size is only 1M

–z/OS 1.9 max size is 9G

� The MVS command "TRACE ST,999K" or “TRACE ST,3M” can be specified in the MVS COMMNDxx SYS1.PARMLIB member.

� See the z/OS MVS Systems Commands Manual for further detail



6

�Maximize System Trace Table Size.

� Use the MVS Display command to ensure the system trace table is active:

–DISPLAY TRACE

IEE843I 13.17.07 TRACE DISPLAY 479 IEE843I 13.17.07 TRACE DISPLAY 479 IEE843I 13.17.07 TRACE DISPLAY 479 IEE843I 13.17.07 TRACE DISPLAY 479

SYSTEM STATUS INFORMATION SYSTEM STATUS INFORMATION SYSTEM STATUS INFORMATION SYSTEM STATUS INFORMATION

ST=(ON,3000K,12000K)ST=(ON,3000K,12000K)ST=(ON,3000K,12000K)ST=(ON,3000K,12000K) AS=ON BR=OFF EX=ON MO=OFF MT=(ON,1000K) AS=ON BR=OFF EX=ON MO=OFF MT=(ON,1000K) AS=ON BR=OFF EX=ON MO=OFF MT=(ON,1000K) AS=ON BR=OFF EX=ON MO=OFF MT=(ON,1000K)

�Considerations:

� System Trace Table is page fixed storage.

� Supply a reasonable value to the nnnM or nG parameter after considering the available central storage and the actual storage required for system trace.

� Supplying a large value for nnnM or nG might cause a shortage of pageablestorage in the system.

MVS Setup RecommendationsMaximize System Trace Table Size



7

MVS Setup RecommendationsIncrease Master Trace Table Size � Increase master trace table size.

�Advantages:

� Master trace maintains a table of the most recently issued operator messages.

� Allows view of external events at the time of failure.

� Setup

� Set MVS Master Trace Table size to 1000K

–Default size is only 24K

� "TRACE MT,1000K" can be specified in the SCHEDxx member of SYS1.PARMLIB.

� See z/OS MVS Diagnosis: Tools and Service Aids, z/OS Initialization and Tuning

Guide/ Ref, and z/OS MVS System Commands manuals for complete details



8

MVS Setup RecommendationsIncrease Master Trace Table Size

� Increase master trace table size

�Use the MVS Display command to ensure the master trace table is active:

–DISPLAY TRACE

IEE843I 13.17.07 TRACE DISPLAY 479 IEE843I 13.17.07 TRACE DISPLAY 479 IEE843I 13.17.07 TRACE DISPLAY 479 IEE843I 13.17.07 TRACE DISPLAY 479

SYSTEM STATUS INFORMATION SYSTEM STATUS INFORMATION SYSTEM STATUS INFORMATION SYSTEM STATUS INFORMATION

ST=(ON,3000K,12000K) AS=ON BR=OFF EX=ON MO=OFF ST=(ON,3000K,12000K) AS=ON BR=OFF EX=ON MO=OFF ST=(ON,3000K,12000K) AS=ON BR=OFF EX=ON MO=OFF ST=(ON,3000K,12000K) AS=ON BR=OFF EX=ON MO=OFF MT=(ON,1000K)MT=(ON,1000K)MT=(ON,1000K)MT=(ON,1000K)

�Considerations:

� Ensure Master Trace Table is large enough to span most error time frames

� Uses Subpool 229 Key 0 High Private Pageable Storage of the master scheduler address space



9

MVS Setup RecommendationsUse the Common Storage Tracker

�Use the Common Storage Tracker

� Advantages:

� SVC dumps (or RMF reports) will provide CSA/ECSA ownership information with jobname, time, and requesting module information.

� Great for common storage outage (storage leaks) problems and fragmentation analysis.

�Setup

� Use the DIAGxx SYS1.PARMLIB member to contain the request. Specify

DIAG=xx in the IPL system parameters or use the SET DIAG=xx operator command.

–Example: DIAGxx member: VSM TRACK CSA(ON) SQA(ON)

� See z/OS MVS Diagnosis: Tools and Service Aids, z/OS Initialization and Tuning

Guide/ Reference, and z/OS MVS System Commands manuals for complete details.



10

MVS Setup RecommendationsUse the Common Storage Tracker

�Use the Common Storage Tracker

� Display the Common Storage Tracker status by using the following MVS command:

�DISPLAY DIAG

�Considerations:

� Negligible Performance impact. Table stored in ESQA

IGV007I 11.44.05 DIAG DISPLAY 713

VSM TRACK CSA(ON) SQA(ON)

VSM TRACE GET(OFF) FREE(OFF)

VSM ALLOWUSERKEYCSA(YES)

VSM BESTFITCSA(NO)

VSM USEZOSV1R9RULES(YES)

TRAPS NAME()

CBLOC

VIRTUAL24()

VIRTUAL31()

REUSASID(NO)

AUTOIPL SADMP(NONE) MVS(NONE)



11

MVS Setup RecommendationsTune CHNGDUMP MAXSPACE

� Ensure adequate CHNGDUMP MAXSPACE is specified to hold the internal SVC

dump.

�Advantages:

� Higher likelihood that SVC dumps will be captured in their entirety without worry of "Partial Dump".

�Setup

� Ensure adequate CHNGDUMP MAXSPACE is specified to hold the internal SVC dump.

� Use the COMMNDxx SYS1.PARMLIB member to issue the appropriate

CHNGDUMP command at IPL time.

–Example: CD SET,SDUMP,MAXSPACE=3000M - Default size is 500M

�3000M is standard for large multi-address space SVC Dumps.

–See z/OS MVS Diagnosis: Tools and Service Aids, z/OS MVS System

Commands, and z/OS Initialization and Tuning Guide for additional details.

�Continued on next page



12



dump.

Display the MAXSPACE setting using the following MVS Command:

D DUMP,OPTIONS

SDUMP- ADD OPTIONS

(ALLPSA,NUC,SQA,LSQA,RGN,LPA,TRT,CSA,SWA,SUMDUMP,

Q=YES,GRSQ,XESDATA),BUFFERS=00000000K,

MAXSPACE=00003000M,MSGTIME=99999 MINUTES,

MAXSNDSP=015 SECONDS,AUXMGMT=ON ,DEFERTND=NO

•Continued on next page



13



dump.

�Considerations:

�MAXSPACE is the total virtual storage used by SDUMP

�SDUMP data capture is serialized so only one dump can be written at a time.

However, MVS releases the serialization after data capture, so another dump can be started before the first one gets written out (especially if something causes a delay in writing the first dump out to DASD). So MAXSPACE virtual storage might have to be temporarily "shared" by more than one dump.

� Ensure central storage and/or local page datasets are large enough to contain their normal peak load plus additional SVC dumps!

� See MVS Initialization and Tuning Guide for additional details.



14

MVS Setup RecommendationsAutomatic Dump Dataset Allocation

� Ensure automatic dump dataset allocation is in place.

�Advantages:

� SVC dumps will be allocated to the correct size without worry of "Partial Dump".

�Setup

� Use the COMMNDxx SYS1.PARMLIB member to issue the appropriate DUMPDS commands to set this up.

–Example: DUMPDS NAME=, DUMPDS ADD, and DUMPDS ALLOC=ACTIVE

–See z/OS MVS Diagnosis: Tools and Service Aids and z/OS MVS Systems Commands manuals for complete details.

�Considerations:

� Ensure assigned storage class has enough space to ensure adequate SVC dump storage requirements



15

�Section 2.

IMS Setup Recommendations



16

IMS Setup RecommendationsUse FMTO Option D

� Specify IMS control region EXEC parm value: FMTO=D

�Produces an SDUMP for terminating and non-terminating errors

� Non-terminating errors include:

– IMS dynamic allocation failures.

– Some ESAF (External Subsystem Attach Facility) failures.

�Allows use of SYSMDUMP as a backup dump in case of SDUMP failures.



17

IMS Setup RecommendationsSYSMDUMP DD for IMS Control Regions

� The SYSMDUMP specification will be used as a backup dump by IMS in the event

that SDUMP processing should fail.

� Specify SYSMDUMP DD in JCL of:

�IMS CTL Region.

�IMS DLI/SAS Region.

�IMS DBRC Region.

� The following dump options should be specified in the SYS1.PARMLIB(IEADMR00)

member to ensure that adequate areas of MVS storage are dumped to diagnose the

problem under most circumstances:

� SDATA=(CSA,RGN,SQA,SUM,TRT,NUC,LPA)

�The SYSMDUMP SDATA options can be added via the CHNGDUMP command

�CD SET,SYSMDUMP=(CSA,RGN,SQA,SUM,TRT,NUC,LPA)



18

IMS Setup RecommendationsSYSMDUMP DD, Continued

� SYSMDUMP DD Specification

�DISP=MOD should be specified because it is possible for multiple tasks to

generate dumps during the processing of an error. We probably will need to see

the first dump produced.

GDG Example

//SYSMDUMP DD DISP=(MOD,CATLG),DSN=IMSSP05.IBA1IMS.SYSMDUMP(+1),

// DCB=(LRECL=4160,RECFM=FB,BLKSIZE=29120),UNIT=3390,

// SPACE=(CYL,(750,100),RLSE),DATACLAS=dtcls,VOL=SER=volser

� Testing the SYSMDUMP function can be done on a test system by temporarily setting the SDUMP to NODUMP via the CHNGDUMP command. This forces a SYSMDUMP to be created.

�CD SET,SDUMP,NODUMP



19

IMS Setup RecommendationsSYSUDUMP DD

� The SYSUDUMP specification will be used by IMS dependent regions for failure events.

�Many application dump formatter tools rely on the SYSUDUMP DD statement to be present in the JCL.

� Specify SYSUDUMP DD in JCL of:

�IMS Dependent Regions.

� The following dump options should be specified in the MVS SYS1.PARMLIB(IEADMP00) member to ensure that adequate areas of MVS storage are dumped.

� SDATA=(CB,ERR,SUM) PDATA=(JPA,LPA,PSW,REGS,SA,SPLS)



20

IMS Setup RecommendationsIMS Table Traces Activated

� Turn on the IMS Dispatcher, Scheduler, DLI, and Lock traces to capture general system diagnostics and database access history

�Specify the following options in the IMS PROCLIB member: DFSVSMxx

� DISP=ON, SCHD=ON, DL/I=ON, LOCK=ON

�Or, use IMS command:

� TRA SET ON TABLE nnnn

� Where nnnn is alternately = DISP, SCHD, DLI, and LOCK

�These traces have relatively low CPU overhead

� SCHD being the lowest and DLI being the highest.

� For non-production environments, when recreating failures, run with traces for the functional areas being tested



21

IMS Setup RecommendationsExternal Trace Environment

� IMS external tracing allows for IMS trace table output to be placed on IMS trace datasets rather than on the IMS OLDS when:

� DFSVSMxx PROCLIB member with nnnn=OUT is specified rather than nnnn=ON

�DISP=ON, SCHD=ON, DL/I=OUT, LOCK=OUT

� The IMS TRACE command specifies OPTION LOG

�/TRA SET ON TABLE nnnn OPTION LOG

� Increases IMS system throughput should external tracing be desired versus sending trace data to the IMS OLDS.

� External trace dataset logging adds less than 1% to the system assuming IO subsystem isn’t constrained

� External trace datasets are allocated in the following order:

1) DASD JCL - DFSTRA01 and DFSTRA02 DDNAMES

2) DASD MDA- DFSTRA01 and DFSTRA02 Dynamic Allocation Members

3) TAPE MDA - DFSTRA0T Dynamic Allocation Member

4) IMS OLDS - If none of the above are found



22


� IMS external tracing specifications

• DFSTRA01 and DFSTRA02DSORG SequentialRECFM VBLRECL 4004BLKSIZE 20024 (LRECLx n +4)

-Set up unique GDG with - minimum 24 hour limit for archive

- ideal to last IMS restart or longer

hlq.imsid.TRACE.ARCHIVE(0)

-Use automation to archive filled trace dataset off to trace archive GDG datasets on external trace dataset switch

DFS2864I EXTERNAL TRACE DATA SET xxxxxxxx FULL - SWITCHING TO yyyyyyyy. imsid

- and for IMS ABEND Scenario –

DFS3896I ATTEMPTING TO PURGE OLDS BUFFERS



23


� Display IMS external tracing specification

Command: /DIS TRACE XTRC.

IMS ACTIVE TRACES

IMS EXTERNAL TRACE IS TRACING TO DISK

XTRC DDNAME ALLOC STATUS STATUS

DFSTRA01 DYNA ALLOCATED CLOSED

DFSTRA02 DYNA ALLOCATED ACTIVE

DFSTRA0T DYNA UNALLOCATED CLOSED



24

�Section 3.

Preservation of Standard Documentation



25

Preservation of Standard Documentation

�Consider implementing normal operating procedures to preserve the following documentation near the time of error.

�MVS Console (SYSLOG)

�SYS1.LOGREC

�IMS OLDS/SLDS

�JES JOBLOG of jobs related to failure

�Any Dumps Produced

�For elusive problems, these items might be requested a month or more after the

problem has been reported.



26

Ideal Time Periods for Syslog, Logrec, SLDS, and Joblog Preservation

�MVS Console (SYSLOG), Logrec, SLDS and Joblogs

–The ideal time frame:

�Back to the last IMS restart.

�MVS Console from the prior clean execution (For comparison)

�The moderate time frame:

�24 hours of MVS Console messages.

�The minimum time frame:

�Two IMS system checkpoint intervals.



27

Dump Preservation

� All dumps created near the time of IMS failure should be retained.

� SYS1.DUMP Datasets should be examined.

� Multiple dumps may be created.

� Keep all dumps at time of failure regardless of the subsystem.

� SYSMDUMP for the IMS Control, DLI/SAS, and DBRC regions need to be examined in case of primary SYS1.DUMP failures.

� Save these datasets if a dump was produced.

� If GDG is used for copies, examine in ISPF for tracks used

� SYSUDUMP should be saved for affected IMS dependent regions.



28

�Section 4.

Manual Dump Intervention



29

Additional Manual Dump Intervention

�While IMS produces SDUMPs for most internal errors without human intervention, IMS Wait/Loop or partial loss of function conditions will require intervention to produce an SVC dump

�IMS hangs can be caused by interaction with many address spaces including:

CTL DLI/SAS DBRC IRLM CQS

OM RMSCIODBA

cctlMPP

(DBDC,

DBCTL,

DCCTL)

IFP BMP

IMS Connect

JMP JBP

APPC VTAMTCPIP

RRS

MQ

Series

Non-IBM

Tools

IBM Tools

IXGLOGR

DB2ODBM

WLM

Other?

**

******

*

*

ap thr ap thr ap thr ap thrap thr ap thr

thrap

Subsys *

Subsys *



30

How Much Data to Dump?

� Due to the complex interactions between the many address spaces, it is sometimes difficult to determine exactly where thesource of the problem lies without a dump of the associated address spaces and their associated data areas.

� Omission of any interrelated address space or data area adds to the possibility that the dump may not be sufficient to solve the problem.

�MVS dumps have a maximum job total of 15 per dump



31

IMS Dump Techniques

IMS SVC Dumps can be requested using various techniques

�MVS SYS1.PARMLIB IEADMCxx

� DUMP Command Parmlib

�MVS SYS1.PARMLIB IEASLPxx

�SLIP Command Parmlib

�Manual MVS DUMP command

�Customized JCL can be built and submitted

�Others?....



32

IEADMCxx, MVS SYS1.PARMLIB

� DUMP Command Parmlib Member

� Create SYS1.PARMLIB members called IEADMCxx for each customized dump command.

�Advantages

– Can be used to customize IMS dumps prior to error event.

– Simple operator interface.

�Disadvantages

– All Jobs must be active or no dump is produced on the system on which the

dump command is entered. Remote systems without a jobname match will dump the DUMPSRV address space.

– Avoid issues:

–Use wildcards * (multiple chars) and ? (single char) and similar naming conventions can be used to specify the jobnames.

–Edit the Parmlib member jobnames prior to issuing the command

� See z/OS MVS Initialization and Tuning Guide and z/OS System Commands



33

� Create SYS1.PARMLIB member called IEADMCI1 containing the following DUMP parameters:

IEADMCxx Example for IMS

JOBNAME=(j1,j2,j3,j4,j5,j6,j7),JOBNAME=(j1,j2,j3,j4,j5,j6,j7),JOBNAME=(j1,j2,j3,j4,j5,j6,j7),JOBNAME=(j1,j2,j3,j4,j5,j6,j7),

SDATA=(CSA,PSA,RGN,SQA,SUM,TRT,ALLNUC,LPA,GRSQ)SDATA=(CSA,PSA,RGN,SQA,SUM,TRT,ALLNUC,LPA,GRSQ)SDATA=(CSA,PSA,RGN,SQA,SUM,TRT,ALLNUC,LPA,GRSQ)SDATA=(CSA,PSA,RGN,SQA,SUM,TRT,ALLNUC,LPA,GRSQ)

Where: Where: Where: Where:

j1 = IMS Control Region Jobname. j2 = DBRC Region Jobname.j1 = IMS Control Region Jobname. j2 = DBRC Region Jobname.j1 = IMS Control Region Jobname. j2 = DBRC Region Jobname.j1 = IMS Control Region Jobname. j2 = DBRC Region Jobname.

j3 = IMS DLI Region Jobname. j4 = IRLM Region j3 = IMS DLI Region Jobname. j4 = IRLM Region j3 = IMS DLI Region Jobname. j4 = IRLM Region j3 = IMS DLI Region Jobname. j4 = IRLM Region JobnameJobnameJobnameJobname....

j5 = IMS CCTL Region 1. j6 = IMS CCTL Region 2j5 = IMS CCTL Region 1. j6 = IMS CCTL Region 2j5 = IMS CCTL Region 1. j6 = IMS CCTL Region 2j5 = IMS CCTL Region 1. j6 = IMS CCTL Region 2

j7 = IMS CCTL Region 3.j7 = IMS CCTL Region 3.j7 = IMS CCTL Region 3.j7 = IMS CCTL Region 3.

Note: It is recommended to use wildcards if possible to ensure jobname matches to allow

for cases where one or more regions might already be down.

Example: Using the following naming conventions, the JOBNAME statement could be

reduced: IMSPCTLA, IMSPDLIA, IMSPDRCA IMSPRLMA CICSP1A CICSP2A CICSP3A

JOBNAME=(IMSP*A,CICSP?A)



34

IEADMCxx DUMP Activation

� To request a dump from the IEADMCI1 parmlib member, enter the following MVS commands:

� DUMP TITLE=(DUMP OF IMS and CCTL Regions ),PARMLIB=(I1)

A dump dataset will be created on the MVS image from which the dump command is entered



35

IMS SYSPLEX Dump Considerations

�IMS SYSPLEX implementations need to consider the possibility that a hang or problem on one IMSplex member may be due to a problem originating from another member, particularly in the area of IRLM locking

�Problems such as IMS Wait/Loops or partial loss of function conditions which require intervention to produce an SVC dump, should include SVC dumps from other members of the IMSplex

�Challenge:

� Ensure that a dump is taken for all necessary address spaces on each system.



36

� Create SYS1.PARMLIB member called IEADMCI1 containing the following DUMP parameters:

Sysplex IEADMCxx Example

It is recommended to use wildcards if possible to ensure jobname matches to allow for

cases where one or more regions might already be down.

Note: The XESDATA, COUPLE and REMOTE parms are for use in SYSPLEX environments

JOBNAME=(j1,j2,j3,j4,j5,j6,j7),

SDATA=(CSA,PSA,RGN,SQA,SUM,TRT,ALLNUC,LPA,GRSQ,XESDATA,COUPLE),

DSPNAME=('j4'.*,'XCF*'.*),

REMOTE=(SYSLIST=(*('j1','j2','j3','j4’,’j5’,’j6’,’j7’),

SDATA,DSPNAME))

Where:

j1 = IMS Control Region Jobname j2 = DBRC Region Jobname

j3 = IMS DLI Region Jobname j4 = IRLM Region Jobname

j5 = CCTL Region 1 J6 = CCTL Region 2

j7 = CCTL Region 3



37

Sysplex IEADMCxx DUMP Activation

� To request a dump from the IEADMCI1 parmlib member, enter the following MVS command:

� DUMP TITLE=(IMS/CCTL SYSPLEX Dumps),PARMLIB=(I1)

� A dump dataset will be created on each MVS image in the SYSPLEX matching the REMOTE specifications for the JOBNAMEs.



38

Storage as a Moving Target

� Storage can be a moving target while being dumped, especially if multiple address spaces interact. Here are some considerations:

�If the MVS sdump CHNGDUMP SET, SDUMP=(Q=Y) command , is specified, all address spaces that are not exempt from system-wide nondispatchability are set nondispatchable while global storage (CSA, SQA) is dumped.

–By quiescing most system activity during SVC dump global storage capture, the dump will contain a more consistent image however, your system activity will nearly cease during global storage capture.

– This option is generally not recommended for this reason.



39

Storage as a Moving Target, Continued

� Storage can be a moving target, continued…

•In all cases (Q=Y or Q=N), dumped address spaces are made nondispatchable while global storage and the private region storage is being dumped for those address spaces

�There is no intentional dispatching coordination between requested address spaces in a dump.

�All private storage is dumped through scheduling of SRBs to the requested address spaces.

�This SRB activity is asynchronous.

� Multiple SRBs to multiple address spaces will run independentlyand loosely in parallel



40

�Section 5.

IMS Situation Specific Diagnostics



41

�Situation Specific Diagnostics:Control Region Wait/Hang

� Control Region Wait/Hang

� Take a SVC dump using one of the methods discussed earlier in the presentation.

� Consider dumping related regions

�If a partial wait occurs, consider taking two of the above mentioned SVC Dumps to identify normal activity versus abnormal activity.

�Note: Modify dumps of IMS are no longer the preferred method to obtain wait documentation.

�IMS Control Region

�IMS DLI/SAS Region

�DBRC Region

�IRLM Region

�CQS

�Operations Mgr

�Resource Mgr

�Structure Call Interface

�TCPIP

�Websphere

�ESAF - DB2, MQSeries, others

�Other Regions ???

�Other IMSplex members with

all their related regions

�ODBM

� = Can't do without

�Troublesome IMS Dep Regs

�CCTL Regions

�ODBA

�IXGLOGRC

�RRS

�APPC

�WLM

�VTAM



42

�Situation Specific Diagnostics:Control Region Wait/Hang, Continued

JOBNAME=(isa?ims,isa?dls,isa?irlm,ci1?sa??),


DSPNAME=('ISA?IRLM'.*,'XCF*'.*),

REMOTE=(SYSLIST=(

SP1A('isa?ims','isa?d*','isa?irlm','ci1?sa??'),

SP1C('isa?ims','isa?d*','isa?irlm','ci1?sa??'),

SP1E('isa?ims','isa?d*','isa?irlm','ci1?sa??'),

SP1F('isa?ims','isa?d*','isa?irlm','ci1?sa??'),

SDATA,DSPNAME))

� IEADMCxx Parmlib (IEADMCA1) example for a slow moving performance problem or hang in a DBCTL data sharing environment with IRLM

� Dump Command to activate the above dump:

� DUMP TITLE=(IMS/CCTL SYSPLEX Dumps),PARMLIB=(A1)

� One dump will be created on each of the specified sysplex members



43

�Situation Specific Diagnostics:Application System Timeout

� Application lockouts are detected by IRLM and MSG DXR162I is issued.

DXR162I irlmx CYCLE NUMBER nnnnnnnn PROCESSED FOR TIMEOUT

Explanation: A TIMEOUT candidate has been detected on at least one IRLM. If an

incompatible blocker was found, the IMS TIMEOUT EXIT has been driven to

present WAITER/BLOCKER information

Set the following SLIP to capture a dump for this event

SLIP SET,A=(SVCD),MSGID=DXR162I,


JL=(ISA*,CI1?SA??),ID=M162,DSPNAME=('ISA?IRLM'.*,'XCF*'.*),

REMOTE=(SYSLIST=(SP1A,SP1C,SP1E,SP1F),JOBLIST,DSPNAME,SDATA),

END



44

�Situation Specific Diagnostics:IMS Loop

� If the IMS Control, DLI, DBRC or Dependent Region enters a loop state:

�Turn on the MVS System Trace Table with BR=ON

�TRACE ST,3M,BR=ON - (size greater than 1m in z/OS 1.9 and above)

�If possible, and not already set, use IMS command: /TRA SET ON TABLE nnnn

� Where nnnn alternately = DISP, SCHD, DLI, LOCK, and LATCH.

� Take a SVC dump using one of the methods discussed earlier in the presentation with related IMS regions.

� Note: IMSplex partner dumps are probably not required for loop problems, unless they are also looping.

� Reset the MVS system trace table and IMS trace tables to their original settings.



45

�Situation Specific Diagnostics:DBRC Related Problems

� DBRC Related Problems

�Obtain a listing of the DBRC RECONs prior to repair attempts.

� Use Recover Control Utility (DSPURX00) - LIST.RECON

command.

� If the RECON dataset listing cannot be obtained due to size,

then alternatively obtain a subsystem listing (LIST.RECON is

preferred).

� Use Recover Control Utility (DSPURX00) - LIST.SUBSYS

ALL command



46

�Situation Specific Diagnostics:DBRC Related Problems, Continued� DBRC Related Problems, Continued

�Use D GRS,CONTENTION command on each system sharing the RECON to

determine if the RECON is held at the exclusion of other waiters

D GRS,CONTENTION

S=SYSTEMS DSPURI01 RECON1.IMSBP1

SYSNAME JOBNAME ASID TCBADDR EXC/SHR STATUS

BT21 IBA2FDBR 0124 009AF7F8 EXCLUSIVE OWN

BT23 IBA1FDBR 0090 009AF1B0 EXCLUSIVE WAIT

BT21 IBA4FDBR 05E0 009AF7F8 EXCLUSIVE WAIT

BT26 IBA4DBRC 05DE 009F15B8 EXCLUSIVE WAIT

BT21 IBA1DBRC 00D7 009FF118 EXCLUSIVE WAIT

BT23 IBA2DBRC 0600 009F15C8 EXCLUSIVE WAIT

� If so, dump the owning address space

DUMP COMM=(dump title)

R nn,JOBNAME=(j1),SDATA=(CSA,PSA,RGN,SQA,SUM,TRT,ALLNUC,LPA,GRSQ),END



47

�Situation Specific Diagnostics:DBRC Related Problems, Continued

� Obtain before and after copies of the RECONs

�JCL Example for Recon dump to file

//JJJJJJJJ JOB ( )

//* DUMP RECON TO FILE

//STEP1 EXEC PGM=ADRDSSU,REGION=0M

//SYSPRINT DD SYSOUT=*

//DASD DD DSN=hhhhhhhh.RECON.BACKUP,DISP=(NEW,CATLG),

// UNIT=3390,SPACE=(CYL,(5,5),RLSE)

//SYSIN DD *

DUMP DS(INCLUDE(IMS.RECON*)) -

OUTDD(DASD)

/*



48

�Situation Specific Diagnostics:DBRC Related Problems, Continued

� JCL Example for Recon restore from file

//JJJJJJJJ JOB ( )

//* TO DOWNLOAD TAPE LOADED WITH DFSDSS

//* NEED TO USE RENUNC TO RENAME TO YOUR HLQ

//* TAPE MAP WILL NOT SHOW LRECL OR BLKSIZE....

//S01RSTR EXEC PGM=ADRDSSU


//IN DD DISP=SHR,DSN=hhhhhhhh.RECON.BACKUP

//DASD DD UNIT=3390,DISP=OLD,VOL=SER=SYS285

//SYSIN DD *

RESTORE DATASET(INCLUDE( ** )) -

INDDNAME(IN) -

OUTDDNAME(DASD) -

CANCELERROR -

CATALOG -

REPLACE -

IMPORT -

TGTALLOC(SOURCE) -

RENAMEUNCONDITIONAL(hhhhhhhh) -

WAIT(2,2)

/*



49

�Situation Specific Diagnostics:DBCTL Related Problems

� DBCTL Related Problems

� Issue IMS /DISPLAY ACTIVE and /DISPLAY CCTL commands, multiple times

� /DIS ACTIVE

� /DIS CCTL ALL

� /DIS CCTL cctlname

� Set the AP portion of the CICS trace to level 1-2

� Set the FC (FILE CONTROL) portion of the CICS trace to level 1-2

�Take a SVC dump using one of the methods discussed earlier in the presentation with related IMS regions, especially the CCTL regions



50

�Situation Specific Diagnostics:Database Problems

�For problems in the database arena, obtain

�Damaged database dataset

�Database Image copy of broken database in state prior to damage

�Image Copy of logically related databases

�IMS SLDS from all data sharing IMS subsystems

� Save to last good database image copy of damaged database

�If possible, and not already set, use IMS command and save output:

� /TRA SET ON TABLE nnnn OPTION LOG

– Where nnnn alternately = DLI and LOCK

�SYSOUT from the Pointer Checker broken database

�SYSOUT from batch jobs that accessed the damaged database



51

�Situation Specific Diagnostics:Database Problems, Continued

�For problems in the database arena, obtain (Cont...)

�DBD for broken databases

�PSBs that access the broken DB

�List.RECON and LIST.HISTORY DBD(broken database)

�SMF 60 (VSAM Volume Data Set Updated), 62 (VSAM Component or Cluster Opened), and 64 (VSAM Component or Cluster Status) records from all data sharing systems

� Back to last good image copy of broken database



52

�Situation Specific Diagnostics:Database Problems, Continued

�For problems in the database arena, obtain (continued)

�For VSAM datasets, close the broken KSDS and run IDCAMS commands:

� VERIFY

� EXAMINE ITEST NODTEST

� EXAMINE NOITEST DTEST� LISTCAT ALL

� If the dataset cannot be closed, run IDCAMS commands:

� EXAMINE ITEST NODTEST� EXAMINE NOITEST DTEST

� EXAMINE ITEST NODTEST

� EXAMINE NOITEST DTEST� LISTCAT ALL

� If the data set is open and being updated then an EXAMINE may reflect what appears to be a problem because the

data set is in a state of flux. Thus if you run the EXAMINEs twice and the errors are the same, there is probably a

problem. If the errors are different, then there is no problem.



53

�Situation Specific Diagnostics:ESAF Interface Problems

� Problems involving the ESAF (External Subsystem Attach Facility)

� Turn on the IMS SUBS TRACE to the external trace dataset

ƒIMS command: /TRA SET ON TABLE SUBS OLDS

� Take a SVC dump of the related regions before and after the recreate attempt.

�Ensure that pertinent regions for the affected subsystem (db2mstr, db2dbm1, mqseries) are included with the other related IMS regions

�Note: IMSplex partner dumps are probably not required for ESAF problems, unless they are also experiencing the problem.



54

�Situation Specific Diagnostics:DC Related Problems

� Problems involving Data Communications

�Issue IMS DISPLAY NODE x command.

� /DIS NODE nodename

�Turn on IMS NODE trace. Data will be captured in the IMS TYPE6701 log record. Print records using IMS utility programs DFSERA10/DFSERA30.

� /TRA SET ON NODE nodename

�Turn on the VTAM Buffer Trace and VTAM Internal Trace to complement the IMS NODE trace. Note: GTF must be active with the USR option.

� F NET,TRACE,TYPE=BUF,ID=nodename

� F NET,TRACE,TYPE=VTAM,MODE=EXT,OPT=(API,PIU,MSG)

�Take a SVC dump using one of the methods discussed earlier in the presentation with related IMS regions - especially VTAM.



55

�Situation Specific Diagnostics:APPC Related Problems

� Problems involving APPC

� Turn on the IMS LUMI Trace to the external trace dataset.

� /TRACE SET ON TABLE LUMI OPTION LOG

� /TRACE SET ON LUNAME XXXXXXX INPUT

� /TRACE SET ON LUNAME XXXXXXX OUTPUT

– (XXXXXXX is the partner LU)

�Turn on the VTAM Buffer Trace and VTAM Internal Trace to complement the IMS LUMI trace. Note: GTF must be active with the USR option.

� F NET,TRACE,TYPE=BUF,ID=nodename

� F NET,TRACE,TYPE=VTAM,MODE=EXT,OPT=(API,PIU,MSG)

�Take a SVC dump using one of the methods discussed earlier in the presentation with related IMS regions - especially VTAM and the APPC regions.



56

�Situation Specific Diagnostics:APPC Related Problems, Continued

� Problems involving APPC (cont...)

�Turn on the program trace for TPPCB DL/1 calls.

� /TRACE SET ON PROGRAM pppppppp

– pppppppp is the program name of the application.

� Turn on the MVS APPC component trace.

� TRACE CT,ON,200M,COMP=SYSAPPC

– nn,OPTIONS=(GLOBAL),END

� When the problem has been recreated, stop the CTRACE.

� TRACE CT,OFF,COMP=SYSAPPC

� APPC component trace sends its trace buffers to a SYS1.DUMP dataset.

� The following IPCS commands can be used to format the trace:

� CTRACE COMP(SYSAPPC) SHORT - One line summary entries

� CTRACE COMP(SYSAPPC) FULL - Detailed trace entryformatting



57

�Situation Specific Diagnostics:CQS Additional Manual Dump Intervention

� IMS/CQS environment additional dump considerations include:

�Structure dumps – Primary, Overflow, Lock entries

�CQS Region

�Other CQS Clients with their related CQS regions

�Above regions from the other IMSplex members

�MVS Logger



58

� Create two SYS1.PARMLIB members:

�Situation Specific Diagnostics:CQS - IEADMCxx Example with Structures

SYS1.PARMLIB IEADMCIA SYS1.PARMLIB IEADMCIA SYS1.PARMLIB IEADMCIA SYS1.PARMLIB IEADMCIA –––– IMS PartnersIMS PartnersIMS PartnersIMS Partners

JOBNAME=(j1,j2,j3,j4,j5,j6,j7,j8,j9),JOBNAME=(j1,j2,j3,j4,j5,j6,j7,j8,j9),JOBNAME=(j1,j2,j3,j4,j5,j6,j7,j8,j9),JOBNAME=(j1,j2,j3,j4,j5,j6,j7,j8,j9),

SDATA=(CSA,PSA,RGN,SQA,SUM,TRT,GRSQ,XESDATA,COUPLE), SDATA=(CSA,PSA,RGN,SQA,SUM,TRT,GRSQ,XESDATA,COUPLE), SDATA=(CSA,PSA,RGN,SQA,SUM,TRT,GRSQ,XESDATA,COUPLE), SDATA=(CSA,PSA,RGN,SQA,SUM,TRT,GRSQ,XESDATA,COUPLE),

DSPNAME=(DSPNAME=(DSPNAME=(DSPNAME=(‘‘‘‘j4'.*,'XCF*'.*),REMOTE=j4'.*,'XCF*'.*),REMOTE=j4'.*,'XCF*'.*),REMOTE=j4'.*,'XCF*'.*),REMOTE=

(SYSLIST=(*('j1','j2','j3','j4','j5(SYSLIST=(*('j1','j2','j3','j4','j5(SYSLIST=(*('j1','j2','j3','j4','j5(SYSLIST=(*('j1','j2','j3','j4','j5‘‘‘‘,,,,’’’’j6j6j6j6’’’’,,,,’’’’j7j7j7j7’’’’,,,,’’’’j8j8j8j8’’’’,,,,’’’’j9j9j9j9’’’’),),),),

SDATA,DSPNAME)) SDATA,DSPNAME)) SDATA,DSPNAME)) SDATA,DSPNAME))

Where: Where: Where: Where:

j1 = IMS CTL Region j1 = IMS CTL Region j1 = IMS CTL Region j1 = IMS CTL Region JobnameJobnameJobnameJobname j3 = DBRC Region j3 = DBRC Region j3 = DBRC Region j3 = DBRC Region JobnameJobnameJobnameJobname

j2 = IMS DLI Region j2 = IMS DLI Region j2 = IMS DLI Region j2 = IMS DLI Region JobnameJobnameJobnameJobname j4 = IRLM Region j4 = IRLM Region j4 = IRLM Region j4 = IRLM Region JobnameJobnameJobnameJobname

j5 = IMS CQS Region j6 = APPC Regionj5 = IMS CQS Region j6 = APPC Regionj5 = IMS CQS Region j6 = APPC Regionj5 = IMS CQS Region j6 = APPC Region

j7 = APPC Scheduler j8 = VTAM j7 = APPC Scheduler j8 = VTAM j7 = APPC Scheduler j8 = VTAM j7 = APPC Scheduler j8 = VTAM

j9 = Other CQS Client Regionj9 = Other CQS Client Regionj9 = Other CQS Client Regionj9 = Other CQS Client Region

It is recommended to use wildcards if possible to ensure jobname matches to allow for cases where one or more regions might already be down.



59

� SYS1.PARMLIB member for structures and MVS logger

�Situation Specific Diagnostics:CQS - IEADMCxx Example with Structures

SYS1.PARMLIB IEADMCIBSYS1.PARMLIB IEADMCIBSYS1.PARMLIB IEADMCIBSYS1.PARMLIB IEADMCIB

JOBNAME=(IXGLOGR),DSPNAME=('IXGLOGR'.*), JOBNAME=(IXGLOGR),DSPNAME=('IXGLOGR'.*), JOBNAME=(IXGLOGR),DSPNAME=('IXGLOGR'.*), JOBNAME=(IXGLOGR),DSPNAME=('IXGLOGR'.*),

SDATA=(ALLNUC,LPA,PSA,RGN,SQA,TRT,CSA,GRSQ,XESDATA,COUPLE), SDATA=(ALLNUC,LPA,PSA,RGN,SQA,TRT,CSA,GRSQ,XESDATA,COUPLE), SDATA=(ALLNUC,LPA,PSA,RGN,SQA,TRT,CSA,GRSQ,XESDATA,COUPLE), SDATA=(ALLNUC,LPA,PSA,RGN,SQA,TRT,CSA,GRSQ,XESDATA,COUPLE),

STRLIST=STRLIST=STRLIST=STRLIST=

(STRNAME=(STRNAME=(STRNAME=(STRNAME=imsmsgq01imsmsgq01imsmsgq01imsmsgq01,LOCKE,(LISTNUM=ALL,ADJ=CAPTURE,EDATA=UNSER), ,LOCKE,(LISTNUM=ALL,ADJ=CAPTURE,EDATA=UNSER), ,LOCKE,(LISTNUM=ALL,ADJ=CAPTURE,EDATA=UNSER), ,LOCKE,(LISTNUM=ALL,ADJ=CAPTURE,EDATA=UNSER),

STRNAME=STRNAME=STRNAME=STRNAME=imsmsgq01oflwimsmsgq01oflwimsmsgq01oflwimsmsgq01oflw,LOCKE,(LISTNUM=ALL,ADJ=CAPTURE,EDATA=UNSER),LOCKE,(LISTNUM=ALL,ADJ=CAPTURE,EDATA=UNSER),LOCKE,(LISTNUM=ALL,ADJ=CAPTURE,EDATA=UNSER),LOCKE,(LISTNUM=ALL,ADJ=CAPTURE,EDATA=UNSER)

, , , ,

STRNAME=STRNAME=STRNAME=STRNAME=mvslogqmsg01mvslogqmsg01mvslogqmsg01mvslogqmsg01,LOCKE,ACC=NOLIM,(LISTNUM=ALL,EDATA=UNSER,,LOCKE,ACC=NOLIM,(LISTNUM=ALL,EDATA=UNSER,,LOCKE,ACC=NOLIM,(LISTNUM=ALL,EDATA=UNSER,,LOCKE,ACC=NOLIM,(LISTNUM=ALL,EDATA=UNSER,

ADJ=CAPTURE))ADJ=CAPTURE))ADJ=CAPTURE))ADJ=CAPTURE))

Where:Where:Where:Where:

imsmsgq01.................is the main structure nameimsmsgq01.................is the main structure nameimsmsgq01.................is the main structure nameimsmsgq01.................is the main structure name

imsmsgq01oflw.............is the overflow structure nameimsmsgq01oflw.............is the overflow structure nameimsmsgq01oflw.............is the overflow structure nameimsmsgq01oflw.............is the overflow structure name

mvslogqmsg01..............is the associated logger structurmvslogqmsg01..............is the associated logger structurmvslogqmsg01..............is the associated logger structurmvslogqmsg01..............is the associated logger structureeee



60

�Situation Specific Diagnostics:CQS - IEADMCxx DUMP Activation

� To request a dump from the IEADMCIA and IEADDMCIB parmlib members, enter the following MVS command:

DUMP TITLE=(IMS Partners/DUMP TITLE=(IMS Partners/DUMP TITLE=(IMS Partners/DUMP TITLE=(IMS Partners/Structures),PARMLIBStructures),PARMLIBStructures),PARMLIBStructures),PARMLIB=(IA,IB)=(IA,IB)=(IA,IB)=(IA,IB)

One combined dump dataset will be created on the MVS image from which the command is entered.

One dump dataset will be created on each image in the SYSPLEX matching the REMOTE specifications for the JOBNAMEs for PARMLIB member IA.

�NOTE: MVS logger support only needs the MVS logger address space from the system which experiences the problem.



61

�Situation Specific Diagnostics:CQS - MVS Log Stream

�The merged MVS log stream can be used to examine CQS log

records.

�IEBGENER can be used along with the default log stream

subsystem exit routine, IXGSEXIT, to copy the log records at

time of failure for later analysis



62

�Situation Specific Diagnostics:CQS - MVS Log Stream, JCL Example

//CQSCPYLG JOB USERID,USERID,MSGLEVEL=1,CLASS=K

//*********************************************************************

//* This job copies a CQS log stream to a dataset (max 32K / record) *

//* *

//* - Replace the DSN on the SYSUT1 card with your CQS logstream *

//* name. *

//* *

//* - Replace the DSN on the SYSUT2 card with your desired output *

//* dataset name. You may also need to adjust the space *

//* allocations, depending on the size of your logstream. *

//* *

//* Changes: *

//* 02/24/97 ben Creation *

//*********************************************************************

//STEP1 EXEC PGM=IEBGENER,REGION=1024K


//SYSUDUMP DD SYSOUT=*

//SYSIN DD DUMMY

//SYSUT1 DD DSN=SYSLOG.MSGQ01.LOG,

// SUBSYS=(LOGR,IXGSEXIT),

// DCB=(BLKSIZE=32760)

//SYSUT2 DD DSN=CQS.LOG.COPY,

// DISP=(NEW,KEEP,DELETE),

// VOL=SER=USER05,

// SPACE=(CYL,(2,10)),

ƒ // UNIT=SYSDA



63

�Situation Specific Diagnostics:CQS - SRDS

� Save the most recent CQS SRDS (Structure Recovery Data

Set) for each structure dumped

�Use IDCAMS repro if LRECL is acceptable (less than 32761)

�If LRECL greater that 32761, use DFSMS ADRDSSU utility



64

�Situation Specific Diagnostics:MSC Related Problems

� If you use IMS MSC (Multiple Systems Coupling) and experience a related

problem

�Take a SVC dump of the coupled IMS regions (minimally the CTL

regions, but the problem may reside in any IMS related region as

discussed earlier)

� In addition, ensure that the VTAM and/or TCPIP address space is

included. Do this as close to the problem time as possible, prior to

attempting to fix the problem

� Save the IMS OLDS for both coupled systems from the time of the

message creation

�Issue DISPLAY LINK ALL and DISPLAY LINK ALL MODE commands to

show the status and queue counts for the logical link and physical link

information for partner



65

�Situation Specific Diagnostics:MSC Related Problems, Cont.

MSC related problems, continued

If the problem is recreatable,

� If using VTAM, Turn on the VTAM Internal Trace for both coupled systems

– F NET,TRACE,TYPE=VTAM,OPT=(API,PIU,MSG),DSPSIZE=5,SIZE=999

� Using these options, the VIT will be created in a VTAM dataspace. When the problem has been recreated, the dump parameters should also include the VTAM dataspace:

– DSPNAME=('NET'.ISTITDS1)

� The MSC LINK trace should also be turned on for both coupled systems (link# or ALL is acceptable for the link specification)

– TRACE SET ON LINK link# LEVEL 4 MODULE ALL

� IMS TYPE6701 records containing TM control blocks for each message at key points



66

�Situation Specific Diagnostics:CTL/DLI Region “Out of Storage”

�IMS control region “out of storage” conditions (ABEND80A ABEND878) can result in ABEND40D (Recursive errors in MVS RTM)

• Problem: Unable to diagnose problem because the dump dataset doesn’t contain private storage for the failing region.

• Solution: Set the following four SLIP traps to capture the initial “out of storage” prior to RTM gaining control



67

�Situation Specific Diagnostics:CTL/DLI Region “Out of Storage”, Continued

SLIP SET,C=80A,J=SLIP SET,C=80A,J=SLIP SET,C=80A,J=SLIP SET,C=80A,J=jobnamejobnamejobnamejobname, , , ,

SDATA=(CSA,PSA,SUM,RGN,SQA,TRT,ALLNUC,LPA),SDATA=(CSA,PSA,SUM,RGN,SQA,TRT,ALLNUC,LPA),SDATA=(CSA,PSA,SUM,RGN,SQA,TRT,ALLNUC,LPA),SDATA=(CSA,PSA,SUM,RGN,SQA,TRT,ALLNUC,LPA),

ID=ID=ID=ID=xxxx,ENDxxxx,ENDxxxx,ENDxxxx,END

SLIP SET,C=878,J=SLIP SET,C=878,J=SLIP SET,C=878,J=SLIP SET,C=878,J=jobnamejobnamejobnamejobname, , , ,

SDATA=(CSA,PSA,SUM,RGN,SQA,TRT,ALLNUC,LPA),SDATA=(CSA,PSA,SUM,RGN,SQA,TRT,ALLNUC,LPA),SDATA=(CSA,PSA,SUM,RGN,SQA,TRT,ALLNUC,LPA),SDATA=(CSA,PSA,SUM,RGN,SQA,TRT,ALLNUC,LPA),

ID=ID=ID=ID=xxxx,ENDxxxx,ENDxxxx,ENDxxxx,END

WhereWhereWhereWhere

jobnamejobnamejobnamejobname = Alternatively, = Alternatively, = Alternatively, = Alternatively, JobnameJobnameJobnameJobname of the IMSof the IMSof the IMSof the IMS

CTL region,CTL region,CTL region,CTL region,

= = = = JobnameJobnameJobnameJobname of the IMS DLI/SAS regionof the IMS DLI/SAS regionof the IMS DLI/SAS regionof the IMS DLI/SAS region

Enter the following two SLIP traps for both the IMS

Control and IMS DLI/SAS regions (4 slips total)



68

Making Hard for those IMS Problems to Hide -Summary

� MVS Setup Recommendations

� IMS Setup Recommendations

� Preservation of Standard Documentation

� Manual Dump Intervention

� Situation Specific Diagnostics



Acknowledgements and Disclaimers

Availability. References in this presentation to IBM products, programs, or services do not imply that they will be available in all

countries in which IBM operates.

The workshops, sessions and materials have been prepared by IBM or the session speakers and reflect their own views. They are provided for informational purposes only, and are neither intended to, nor shall have the effect of being, legal or other guidance or

advice to any participant. While efforts were made to verify the completeness and accuracy of the information contained in thispresentation, it is provided AS-IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this presentation or any other materials. Nothing contained in this presentation is intended to,

nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software.

All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics may vary by customer. Nothing contained in these

materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific sales, revenue growth or other results.

© Copyright IBM Corporation 2012. All rights reserved.

– U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract

with IBM Corp.

IBM, the IBM logo, ibm.com, IMS, MVS, z/OS, DB2, CICS, and WebSphere are trademarks or registered trademarks of International

Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on

their first occurrence in this information with a trademark symbol (® or ™), these symbols indicate U.S. registered or common law

trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law

trademarks in other countries. A current list of IBM trademarks is available on the Web at “Copyright and trademark information” at

www.ibm.com/legal/copytrade.shtml. Other company, product, or service names may be trademarks or service marks of others.



70

Communities

• On-line communities, User Groups, Technical Forums, Blogs,

Social networks, and more

– Find the community that interests you …

• Information Management bit.ly/InfoMgmtCommunity

• Business Analytics bit.ly/AnalyticsCommunity

• Enterprise Content Management

bit.ly/ECMCommunity

• IBM Champions

– Recognizing individuals who have made the most outstanding

contributions to Information Management, Business

Analytics, and Enterprise Content Management communities

• ibm.com/champion



71

Thank You! Your Feedback is Important!

• Access SmartSite to complete your session

surveys

– Any web or mobile browser at iodsmartsite.com

– SmartSite applications for iPhone, iPad, Droid and

Blackberry

– Any SmartSite kiosk onsite

Technology

Making It Hard for Those IMS Problems to Hide - IMS UG March 2013 DC_JAX_PA