Download PowerPoint Presentation (1.1 MB)

DB2 for z/OS Stored Procedures“In the Mainstream”

Robert Catterall, President

Michigan DB2 Users GroupDecember 6, 2007

© 2007 Catterall Consulting

Agenda

• Stored procedures – the future of server-side SQL• What happens when a stored procedure is called• Creating and coding stored procedures• Deploying stored procedures• Some stored procedure odds and ends

2

Acknowledgement

• Peggy Zagelow, Senior Technical Staff Member at the IBM Silicon Valley Lab and development lead for DB2 for z/OS stored procedures, has answered many questions I’ve sent her way.

3

Stored Procedures – The Future of Server-Side SQL

Stored Procedures – The Future of Server-Side SQL

4

Traditional z/OS Server-Side SQL (1)

• Online: embedded in CICS or IMS/TM (IMS Transaction Manager) transaction programs◦ Why?

1. A great many DB2 for z/OS sites also have CICS or IMS/TM

2. It works: fast, reliable, CPU-efficient

3. DB2 for z/OS stored procedure functionality wasn’t even available until Version 4 (mid-1990s)

4. DB2 stored procedures initially lacked key features: no ability to fetch rows from a cursor in a stored procedure (addressed with V5) and no DDL support (provided with V6)

5. Many mainframers have an early 1990s mind set when it comes to anything involving DDF (“Too much overhead,” “Can’t effectively monitor it,” etc.)

5

Traditional z/OS Server-Side SQL (2)

• Batch: SQL embedded in programs that run in JES initiator address spaces◦ Often used to process large files received by organizations

from clients and/or suppliers◦ Often thought of as the only feasible way to process bulk

DB2 database updates, especially when deadlines are tight◦ “JES comes with z/OS – why use anything else for batch

work?”

6

Stored Procedures’ Time Has Come (1)

• Off-mainframe client-side application programmers love ‘em◦ They are already familiar with the stored procedure

programming model (use of stored procedures is a best practice in SQL Server and Oracle environments)

◦ Through the DB2 client and DB2 Connect, they can invoke DB2 for z/OS stored procedures using database interfaces with which they are familiar, including:

ADO.NET JDBC (Java Database Connectivity)

◦ They’d rather deal with query result sets as such, rather than deal with something like a CICS COMMAREA

7


• For the skeptics on the z-side:◦ It is possible to run a high-volume online workload (1000+

transactions per second) using DB2 stored procedures invoked by off-mainframe programs (organizations are already doing so)

◦ Stored procedures provide the CPU efficiency and data security benefits of static SQL (authids require only the EXECUTE privilege on the stored procedure program and package)

◦ Stored procedure programs can be written in COBOL They are a great way to leverage both COBOL (on-mainframe) and

Java (or C# or whatever is used on the app server) programming skills for one application project

8


• More for the z-folks:◦ In-depth stored procedure monitoring capability is here now,

with several products available from several major vendors◦ Package-level DB2 accounting information was enhanced with

DB2 V8 to include SQL, buffer pool, and locking counters (important when stored procedures invoked via DDF – no plan)

◦ WLM-managed stored procedure address spaces provide CICS AOR-like benefits (WLM = Workload Manager, part of z/OS):

You can have several of them (useful for isolation by application, and/or to temporarily isolate new stored procedures)

You get 2-phase commit capability, just as you do with CICS DB2 attach (coordinated commit and rollback of, for example, DB2 table and MQ queue changes made in a single unit of work)

9

What About Batch?

• Of course you can call a DB2 stored procedure from a batch program (any program that can issue an SQL statement can call a stored procedure)

• Here’s an idea: think bigger – think about replacing traditional batch processing of files with a combination of MQ and DB2 stored procedures◦ First step: a program reads an input file and places individual

records on an MQ queue◦ Message hitting queue triggers stored procedure call, with

message as input to stored procedure◦ Stored procedure makes requisite database changes,

provides requisite output (if needed)

10

More on “Batch Transactionalization”

• Throughput delivered via parallelization (multiple concurrently executing stored procedure programs)

• Requires no changes on the part of organizations that send files to you or receive files from you◦ But it lets you to talk to a file sender about sending a record at a

time to a queue that you externalize via a Web service (file sender might really like that)

◦ You could similarly send an output record at a time to a company that currently receives files from you

• Implementation details beyond the scope of this session◦ But you can get a lot of good info from Chapter 33 of the DB2

for z/OS V8 Application Programming and SQL Guide

11

What Happens When a Stored Procedure is Called

What Happens When a Stored Procedure is Called

12

Stored Procedure Processing Flow (1)

• First, you need a DB2 thread◦ Local applications (on-mainframe): created when first SQL

statement issued by program (first statement could be a CALL)◦ Remote clients (off-mainframe): created when CONNECT

statement issued (application gets a DDF thread – aka a DBAT)

• Next, CALL statement is issued by program◦ Format: CALL MYPROC (:invar1, :invar2, :outvar1, :outvar2)◦ Stored procedure name, input/output parameters passed to

DB2 Remote clients: via DB2’s Distributed Data Facility (DDF) Local applications: via same Program Call (PC) used for other SQL

statements (PC is a cross-memory address space-switching function provided by z/OS)

13


• Upon receiving the CALL, DB2 looks in SYSROUTINES catalog table for information about the stored procedure, including:◦ Load module name (EXTERNAL_NAME)◦ Name of WLM application environment to be used in executing

the stored procedure program (WLM_ENVIRONMENT)◦ Once retrieved from the catalog, this information is placed in a

cache for quick access when needed again

• DB2 tells the WLM-managed address space to execute the stored procedure program

14


• WLM-managed address space uses LE (Language Environment, another part of z/OS) to load and execute the stored procedure◦ A Java stored procedure will be loaded by way of Unix System

Services, but it will be executed via LE (there is a JVM under LE)

• Control is passed to the stored procedure program, along with associated input and output parameters◦ The calling program’s task (TCB or SRB) is suspended while the

stored procedure’s TCB uses the calling program’s DB2 thread◦ Stored procedure will PC (Program Call) into DB2 address space

for SQL statement execution (same as any mainframe program)◦ Before terminating, stored procedure assigns values to output

parameters

15


• Control passes back to DB2◦ DB2 copies output parameters received from stored procedure

to client application parameter area

• Control passes back to client application◦ If the stored procedure program declares a cursor with the

option WITH RETURN, and OPENs the cursor (and doesn’t close it), the rows in the query result set will be returned to the calling program when the stored procedure program ends

16

Creating and Coding Stored Procedures

Creating and Coding Stored Procedures

17

Get the Definition Right

• A stored procedure is defined to DB2 via the CREATE PROCEDURE statement

• There are a lot of options that can be specified in a CREATE PROCEDURE statement – I’ll cover a few of these

18

CREATE PROCEDURE Options (1)

• ASUTIME◦ Specifies the total amount of processor time, in CPU service

units, that can be consumed by an invocation of the stored procedure

◦ It’s a good idea to specify a non-zero value for ASUTIME (the default value is zero)

Otherwise, if a stored procedure program started running really long, the maximum TCBs specified for a stored procedure address space could be reached, and WLM could fire up more instances of the address space

Recent z/OS enhancements will ensure that the system doesn’t choke on stored procedures as a result, but it would still be good if the system were to kill extra-long-running stored procedures

19


• STAY RESIDENT◦ Indicates whether or not a stored procedure load module is to

remain resident in memory when the stored procedure ends◦ Specify STAY RESIDENT YES to improve execution

performance (though this is ignored for a Java stored procedure) There was a time when people didn’t want to do this, due to concern

about the disruptive effect of refreshing the memory of a stored procedure address space when a new version of a load module needed to be substituted for the old one

That was (as I recall) a legitimate concern some years ago; now, refreshing the memory of a WLM-managed stored procedure address space is a non-disruptive operation if WLM is operating in goal mode

20


• PROGRAM TYPE◦ Indicates whether a stored procedure will run as a main routine

or as a subroutine◦ Specify PROGRAM TYPE SUB to boost CPU efficiency

Recall that a program written to run as a subroutine has to do some things that Language Environment would do for a main program, including initialization and clean-up work

◦ A nice DB2 V8 assist for COBOL stored procedures defined with PROGRAM TYPE SUB and STAY RESIDENT YES that use stored procedure parameters as host variables:

Set the SQL-INIT-FLAG variable (generated by the DB2 precompiler) to 0

Result: the SQLDA will be updated with the current addresses

21


• COMMIT ON RETURN◦ Indicates whether or not DB2 will commit a transaction

immediately following the end of stored procedure execution◦ Personally, I prefer COMMIT ON RETURN NO (let the calling

application control the commit scope)◦ I’d be pretty careful about using COMMIT ON RETURN YES:

Could lead to premature committing of an application’s data changes

Requires the use of the WITH HOLD option for cursors that will be used to return result sets to calling program

22

Coding Stored Procedures

• Language choices for external stored procedures: COBOL, C, C++, REXX, Assembler, PL/I, and Java◦ Keep in mind that there is no need for the language of a stored

procedure program to match the language of a calling program

• An increasingly interesting stored procedure development language option: SQL◦ A SQL procedure contains the program source within the CREATE

PROCEDURE statement◦ Statements that can be used in the body of a SQL procedure

include GOTO, IF, ITERATE, LOOP, REPEAT, and WHILE So, program logic can be fairly sophisticated

23

More on SQL Procedures

• DB2 V8 (and prior releases): DB2 generates a C program from the CREATE PROCEDURE statement when LANGUAGE SQL is specified◦ The SQL procedure executes pretty much as does an external

DB2 stored procedure

• DB2 V9 New Function Mode introduces a big change:◦ A C program is not generated from the CREATE PROCEDURE

statement when LANGUAGE SQL is specified Instead, the executable form of the SQL procedure is stored in the

DB2 Directory, and the program – when called – runs in the DB2 Database Services Address Space

24

The zIIP Factor

• DB2 for z/OS Version 9 in New Function Mode:◦ A native SQL procedure – if called from an off-mainframe

requester via DDF – can execute on a zIIP engine◦ Result could be a nice reduction in TCO, as zIIP engines –

while not free – do not factor into mainframe software pricing

• My advice: start looking into SQL procedures now, if you haven’t already done so

25

A Question of Calls

• OK, back to external stored procedures• Suppose you have a stored procedure program ABC,

written in COBOL• Suppose ABC will call program XYZ, also written in COBOL• Should program ABC:

A. CALL program XYZ as a nested stored procedure?

B. Call program ABC as a subroutine via COBOL CALL?

• If you want peak CPU efficiency, the answer is B◦ That way, you have just program ABC’s TCB, versus a TCB for the

top-level stored procedure and each nested stored procedure

26

An OPTIMIZE FOR “Gotcha?” (1)

• You’re probably familiar with the OPTIMIZE FOR n ROWS clause for a DB2 SELECT statement◦ What it does: tells the DB2 optimizer to choose the data access

path that will return the first “n” rows of the result set as quickly as possible

◦ Often, people code OPTIMIZE FOR 1 ROW to get DB2 to use an index in favor of doing a sort operation (e.g., for ORDER BY)

n=1 makes the point to DB2, so why make n>1?

• Potential problem with OPTIMIZE FOR 1 ROW in a stored procedure (see next slide)

27

An OPTIMIZE FOR “Gotcha?” (2)

• Scenario:◦ DB2 V8 Client at FixPack 15 (this is the code that runs where

DB2-bound SQL statements originate on off-mainframe servers)◦ DB2 for z/OS Version 8 in Compatibility Mode

• If OPTIMIZE FOR n ROWS specified for a cursor-based SELECT in a stored procedure, DB2 will send “n” rows in each network transmission to the client◦ “Old” behavior: DB2 would send “n” rows in first transmission to

client; after that, rows sent in 32K blocks◦ Recommendation: leave out OPTIMIZE FOR n ROWS when

coding DECLARE CURSOR in a stored procedure program DB2 will then use block fetch for network efficiency

28

Deploying Stored ProceduresDeploying Stored Procedures

29

WLM Application Environments

• Created by way of a WLM panel• One of the things that can be set on this panel is NUMTCB

◦ Specifies the maximum number of concurrently executing stored procedures for an address space associated with the WLM execution environment

◦ DB2 V7 and prior: when NUMTCB reached, WLM could dynamically fire up another instance of the address space to handle demand

◦ DB2 V8 and beyond: NUMTCB is a ceiling for an address space, but WLM manages tasks and can start additional instances of an address space without NUMTCB being reached

• Reasonable value for NUMTCB: something in the range of 40 to 50

30

A Little More on WLM Environments

• For PROGRAM TYPE SUB stored procedures, written in COBOL, that have different run-time options (specified in CREATE PROCEDURE, passed to LE)◦ Recommendation: separate the stored procedures with

different run-time options by WLM application environment

31

Some Stored Procedure Odds & Ends

Some Stored Procedure Odds & Ends

32

Stored Procedure CALL Cost

• It takes about 30,000 instructions to invoke an external stored procedure

• This number will be less (don’t yet know how much less) for a DB2 V9 NFM native SQL procedure◦ Not surprising, since this type of stored procedure will

execute in the DB2 DBM1 address space

33

Helping DB2 Find the Package

• New special register in DB2 V8, CURRENT PACKAGEPATH, can do for a stored procedure invoked via DDF what PKLIST does for a PLAN◦ Recall that a generic plan is used for all requests coming in

through DDF◦ Calling program can issue SET CURRENT PACKAGEPATH

• In DB2 for z/OS V9, PACKAGEPATH can be specified on CREATE PROCEDURE for external procedures

34

How Expensive is Java?

• A Java stored procedure will use 20-50% more CPU than a COBOL stored procedure that does the same thing◦ If that sound like a lot, consider that the ratio was probably

more like 2:1 a few years ago◦ Java performance has been significantly enhanced on the

mainframe◦ Parity with C?

35

Robert CatterallCatterall Consulting

[email protected]

36

Documents

Download PowerPoint Presentation (1.1 MB)