Serge RielauIBM Canada

Session: F13

Shifting ShapesTransforming the way you

evolve your DB schema

Oct 7, 2009 • 3:45 p.m. – 4:45 p.m.Platform: DB2 for LUW

2

Agenda

• Motivation• Task driven feature introduction• Anchoring• Auto revalidation• Forced revalidation• Object replacement• Defaults• Alter table

• Recap• Conclusion

3

The Problem

• As business needs change, applications evolve to keep up

• As the applications evolve, databases schemas must also evolve:• Columns are added, removed,

or change data type• View, routine and trigger text changes• Tables are decomposed

• In practice, these “simple” changes are not simple at all

• Requires a carefully planned change process to manage the impacts on dependent objects Chardin, Jean-Baptiste Siméon The House of Cards

4

Objective

• To describe and demonstrate how DB2 9.7 for LUW transforms the way you evolve your database schema

DB schema VnDB schema Vn+1

5

Why is changing so hard?

• So far DB2 insisted that database objects be in a consistent state at all times

• Actions that might affect objects that depend on the object being changed are either:• Restricted (the change fails), or• Cause those dependent objects to be dropped!

(CASCADE)

• Real world databases have many of these dependent objects (views, functions, triggers, etc.)

• There are some common change tasks that could be simplified

• Some dependencies are implied rather than described

6

A sample schema

• Table Emp and view Empv

• Sequence EmpId

• Module emp with hire(), terminate() procedures

• Constants MinimalWage and Workhours

• Function MinimalSalary() computes salary

• Trigger EmpValidateNew enforces minimal salary for newhires

Emp MinimalWage Workhours

MinimalSalary()

Hire() Terminate()

EmpId

EmpValidateNew

Empv

7

Tasks/Problems

1. Increasing last name domain Change base table column type Keep derived variables and parameters in synch

2. Increase minimal wage Recreate variable Manage dependencies up to the trigger

3. Change salary data type Reorg needed? Change ripples through many places

4. Running DDL scripts Determine order of creation

5. Adding a bonus parameter to a procedures Deal with numerous invokers

6. Adding a column to a view the empv view Deal with long running queries

8

Introducing anchored types

• Problem• How to keep data types in synchronized

• Solution• Distinct types

• Requires strong typing• Has never really become popular in SQL

• Anchored types• Type variable/parameter based on another object

When “root” object changes, so does derived one• Popular with some other DBMS • Can anchor to scalar types and rows

9

CREATE TABLE emp(id SMALLINT NOT NULL, lastname VARCHAR(20) NOT NULL, firstname VARCHAR(20) NOT NULL, salary DECIMAL(8, 2) NOT NULL, deptid SMALLINT)

CREATE OR REPLACE VIEW empv(id, lastname, firstname, salary, deptid) AS SELECT id, lastname, firstname, salary, deptid FROM emp

CREATE OR REPLACE SEQUENCE empid AS SMALLINT

CREATE OR REPLACE MODULE emp

ALTER MODULE emp PUBLISH PROCEDURE hire(IN lastname ANCHOR emp.lastname, IN firstname ANCHOR emp.firstname, IN salary ANCHOR emp.salary, IN deptid ANCHOR emp.deptid, OUT id ANCHOR emp.id) SELECT empid INTO id FROM NEW TABLE(INSERT INTO empv VALUES(NEXT VALUE FOR empid, lastname, firstname, salary, deptid))

ALTER MODULE emp PUBLISH PROCEDURE terminate(IN id ANCHOR emp.id) DELETE FROM vemp WHERE emp.id = terminate.id;

10

Increasing domain of lastname

• Increase length from 20 to 30 bytesALTER TABLE emp ALTER COLUMN lastname SET DATA TYPE VARCHAR(30)

• Dependent object Hire() gets invalidatedSELECT valid FROM SYSCAT.ROUTINES WHERE routinename = 'HIRE'

VALID

N

• Automatic revalidation on first usageCALL emp.hire('Jones', 'Joe', 40000, 2, ?)

Value of output parameters

Parameter Name : ID

Parameter Value : 1

SELECT valid FROM SYSCAT.ROUTINES WHERE routinename = 'HIRE'

VALID

Y

11

Introducing object replacement

• Problem• Managing conditional dropping of previous versions of

an object in DDL script• Preserve previously granted access to objects being

re-created

• Solution• CREATE OR REPLACE

• If object exists drop it first• If objects exists preserve all existing authorizations

• Supported for: Functions, procedures, triggers, types (FP1), sequences, aliases, views, variables

12

Original schema

CREATE VARIABLE minimal_wage DECIMAL(4, 2) CONSTANT 7.50

CREATE VARIABLE workhours SMALLINT CONSTANT 40 CREATE FUNCTION minimalSalary() RETURNS salary ANCHOR emp.salary RETURN minimal_wage * workhours * 52

CREATE TRIGGER emp_validate_new BEFORE INSERT ON empREFERENCING NEW AS NFOR EACH ROWBEGIN IF salary < minimalsalary() THEN SIGNAL SQLSTATE '78000'

SET MESSAGE_TEXT = 'Salary below legal minimum'; END IF;END

• Trigger uses function, uses variable..

13

Increase minimal wage• Replace the wage constantCREATE OR REPLACE VARIABLE minimal_wage DECIMAL(4, 2) CONSTANT

8.50

• Function is invalid, trigger remains untouchedSELECT valid FROM SYSCAT.ROUTINES

WHERE routinename = 'MINIMALSALARY’

VALID

N

SELECT valid FROM SYSCAT.TRIGGERS WHERE tabname = 'EMP’

VALID

Y

• Auto revalidation on usageCALL emp.hire('Smith', 'Henry', 10000, 2, ?)

SQL0438N Application raised error or warning with diagnostic text: "Salary below legal minimum". SQLSTATE=78000

14

Introducing a “make me whole” routine• Problem• Some table changes place table in reorg pending,• … others do not.• Revalidation of objects on first use can be too late

• Solution• Procedure that “does what needs to be done”

• To a base object (table, view, routine, variable, …)• A schema• A module

15

Inflation proof salary

• Change salary column type to DECFLOATALTER TABLE emp ALTER COLUMN salary SET DATA TYPE DECFLOAT(34)

• Various objects are not valid anymoreSELECT reorg_pending FROM SYSIBMADM.ADMINTABINFO

WHERE TABNAME = 'EMP'

REORG_PENDING

Y

SELECT OBJECTMODULENAME, OBJECTNAME, ROUTINENAME FROM SYSCAT.INVALIDOBJECTS

OBJECTMODULENAME OBJECTNAME ROUTINENAME

- SQL090812082112200 MINIMALSALARY

- EMP_VALIDATE_NEW -

EMP SQL090830105118000 HIRE

EMP SQL090830105127500 TERMINATE

16

..now fix it

• Fix all objects in my schemaCALL ADMIN_REVALIDATE_DB_OBJECTS(object_schema=>'MY_SCHEMA')

SELECT reorg_pending FROM SYSIBMADM.ADMINTABINFO WHERE TABNAME = 'EMP'

REORG_PENDING

N

SELECT OBJECTMODULENAME, OBJECTNAME, ROUTINENAME FROM SYSCAT.INVALIDOBJECTS

OBJECTMODULENAME OBJECTNAME ROUTINENAME

0 record(s) selected.

• Schema sure to be compiled

CALL emp.hire('Kennedy', 'Ken', 55000, 3, ?)

Value of output parameters

Parameter Name : ID

Parameter Value : 5

17

Introducing “create (with error)”• Problem• Scripts creating objects out of order• Required objects missing• Required authorization to available• Linearizing object creation not trivial or confusing

• Solution• Tolerate certain kinds of error

• Authorization• Ambiguous name• Object does not exist

• Store invalid object in the catalog• Validate on first use or via revalidation procedure

18

Adding bonus to the mix

• Add a bonus procedure without a bonus columnALTER MODULE emp PUBLISH PROCEDURE paybonus(IN id ANCHOR emp.id,

IN percent DEC(4, 2))UPDATE empv SET bonus = salary * percent / 100 WHERE emp.id = paybonus.id

SQL20480W The newly defined object "SRIELAU.EMP.PAYBONUS" is marked as invalid because it references an object "BONUS" which is not defined or is invalid, or the definer does not have privilege to access it.

SELECT ERRORMESSAGE FROM SYSCAT.INVALIDOBJECTS WHERE ROUTINENAME = 'PAYBONUS'

ERRORMESSAGE

SQL0206N "BONUS" is not valid in the context where it is used.

19

…adding bonus, really now!

• Add bonus to the employee view (also out of order)CREATE OR REPLACE VIEW empv

(id, lastname, firstname, bonus, salary, deptid)

AS SELECT id, lastname, firstname, salary, bonus, deptid FROM emp

• Finally add bonus to the employee tableALTER TABLE emp ADD column BONUS DECFLOAT

• The procedure and view fix themselves on first useCALL emp.paybonus(5, 8.5)

Return Status = 0

• NoteUPDATE DB CFG USING AUTO_REVAL DEFERRED_FORCE

required for “create (with error)”

• Supports• Views, procedures, functions and triggers

20

Introducing parameter defaulting

• Problem• Adding parameters to procedures required patches to

all callers• Procedures with many parameters get unwieldy

• Solution• Allow DEFAULT clause for procedures in the same

way as for columns (but more powerful expressions)

• Note• No defaults for function parameters yet• Defaults must be at the end• Also see parameter naming

21

Bonus for all!!!• Define default bonus for new hiresALTER MODULE emp DROP PROCEDURE hireALTER MODULE emp

PUBLISH PROCEDURE hire(IN lastname ANCHOR emp.lastname, IN firstname ANCHOR emp.firstname, IN salary ANCHOR emp.salary, IN deptid ANCHOR emp.deptid, OUT id ANCHOR emp.id, IN bonus ANCHOR emp.bonus

DEFAULT 4.5) SELECT id INTO id FROM NEW TABLE(INSERT INTO emp VALUES(NEXT VALUE FOR empid, lastname, firstname, salary,

salary * bonus / 100, deptid)

• Original invocation still worksCALL emp.hire('Miller', 'Brian', 55000, 3, ?) Value of output parameters Parameter Name : ID Parameter Value : 6

22

Introducing “soft invalidation”• Problem• Long running transactions or queries cause

partial application outages when performing DDL changes

Time

# Processing connections

IssueDDL

VIEW DDL Process

Commit

A B

23

Introducing “soft invalidation” (cont)• Solution• Allow running transactions to drain

asynchronously

Time

# Processing connections

Issue DDL = ProcessVIEW DDL

Commit

A B

24

Extend common view• Add initials without impeding running transactionsCREATE OR REPLACE VIEW empv

(id, lastname, firstname, initials, bonus, salary, deptid)

AS SELECT id, lastname, firstname,

substr(firstname, 1, 1) || '.' || substr(lastname, 1, 1) || '.',

salary, bonus, deptid FROM emp

• Also works for:• Inline triggers• Inline functions• Drop and/or create or “create or replace” works• Can even replace view with table!

25

Recap: Revalidation

• Database configuration AUTO_REVAL• DISABLE maintains previous version behavior• IMMEDIATE revalidates when marked invalid• DEFERRED Revalidate on first use• DEFERRED_FORCE allows “create (with error)”

• Deferred object revalidation options• invoke admin_revalidate_db_objects() procedure • test the object yourself• wait for next use (risks compile error for 1st user)

26

Recap: CREATE OR REPLACE

• Supported for alias, function, nickname, procedure sequence, trigger, variable, and view

• If object already exists, replace it

• Not exactly the same as DROP and CREATE

• A replaced object retains • granted privileges • dependencies with other objects

• Together with auto-revalidation, significantly less manual effort required when replacing an object (especially a view or routine used extensively)

27

Recap: ALTER TABLE• More support• Any type to any other castable type

E.g. from integer -> charALTER TABLE emp ALTER deptid SET DATA TYPE CHAR(30)

• When “down casting” table is prechecked on ALTER• RENAME column support• Unlimited alterations per transaction

• Remember• Let admin_revalidate_db_objects() decide whether

reorg is needed or not

28

Evolving your schemaBest Practices

• Pull together all the DDL changes required as a single task

• Minimize the number of transactions• especially using ALTER TABLE changing row

versions

• Last step is to run validation procedure• Avoid late surprises from revalidation errors

• Rehearse on non-production database (if possible)

• If changes are significant, process at low-usage time or in maintenance window (if possible)

29

Conclusion• DB2 9.7 transforms the way you manage schemas• Schema evolution features include:• Relaxed dependencies with automatic revalidation• Extended ALTER TABLE features• Creating objects out of order• Anchored data types• Soft invalidation to maximize uptime

• The features described are intended to:• Reduce administration to maintain the schema• Minimize planned outage windows• Provide for in flight “surgical tweaks” of the schema

30

Q&A

• ?

31

Serge RielauIBM Canada

srielau@ca.ibm.com

Session: F13

Shifting ShapesTransforming the way you evolve your DB schema