IMS XML DB - Lab Manual

Developing XML DB applications for IMSSkill Level: Intermediate to Advanced

Denis Gaebler ([email protected]) – Software Developer, IBMKiran Challapalli ([email protected]) - Software Developer, IBMEvgueni Liakhovitch ([email protected]) - Software Developer, IBM

IBM Corporation31 October 2010

Abstract

This tutorial constitutes an in-depth description of the Purchase Order(IPO) Samples for IMS. It is intended to be used as a narrative while stepping through the instructions in the walk through the tutorial.

Naming conventions are a little bit difficult for this lab.

Group 01 has assigned Userid IMPOT61. So the High Level Qualifier for JCL etc. is IMPOT61, but the PSB Name is IPOPSB01, the Database Name is IPODB01, but the Database High level Qualifier is IMS.IMSD, the Trancode name is IPOPGM01.

So for everything that is userid, unix HFS path or High Level Qualifier the xx has to be replaced with the two digit suffix of your userid and everything that is transcode, program name, psb name or dbd name has to be suffixed with the group id.

The IMS Purchase Order database

The sample Purchase Order database is a simple two segment database that stores simple information about a purchaser which each may have any number of dependent segments of information about the individual items being purchased. Each IMS record in the IPO PCB describes an individual purchase order.

The database is described by the IPODBD DBD. The application uses only a single PCB view, containing the entire database. The application PSB source is contained in the file IPOPSB.The DBDs and PSBs have already been created.

Step 1: Download project interchange and import project

Developing XML DB applications for IMS© Copyright IBM Corporation 2009, 2010. All rights reserved.

1 of 17

mailto:[email protected]



You will need the project interchange file IMSDBXMLDBLab.zip.

1.1. Switch to the z/OS PerspectiveSwitch to the default z/OS Projects perspective. Within Eclipse, there are several ways to change perspectives.

1. From the Window menu, select Open Perspective > Other.

Figure 1: Opening a perspective in Rational Developer for System z

2. Scroll down and select z/OS Projects from the Open Perspective dialog box.

1.2. Copy the Project Interchange file.

1. Expand , , and

1. Expand /usr/lpp/ims/java/project in the Remote System View and select the IMSDBXMLDBLab.zip. Right click on it and select copy. Paste it to your local directory under DrivesC:c:\temp


2 of 17

Step 2: Import the projectThe project must be imported and will be the base of the following work.

2.1. Switch to the Java PerspectiveImport of a Java project should be done from the Java perspective.

1. Switch to the Java Perspective and find the Java project explorer.

2.2. Import the projectThe project has to be imported to the workspace in order to use it.

1. In the Java project explorer, click the right mouse button and select Import.

2. In the Import Wizard, Select Other > Project Interchange

3. For the zip file click on Browse, select the IMSDBXMLDBLab.zip file that was downloaded and stored in Step 1.

4. In the project selection view, check IMSDBXMLDBLab project and click Finish to Import the project into the workspace.

5. Expand src and all other subfolder in the project.

Figure 2: Expanded IMSDBXMLDBLab

6. Make sure there are no compiler errors and all Referenced Libraries are in the list.


3 of 17

Step 3: Viewing an IMS Database as XML

Starting with IMS Version 9, the DLIModel utility is also used to generate an XML Schema based off of a DBD and PCB view of that database. The generated XML Schema creates a mapping between a PCB view of a database and an XML view of that same database. The following IPOPSB-PurchaseOrder.xsd XML Schema is used in this sample. This XML Schema can be viewed as a tree of XML Schema particles as shown here:

Figure 3: IPO schema diagram

The annotations (which are viewed as comments by XML parsers) align the XML Schema particles with corresponding IMS Segments and Fields. With this mapping defined and in place we can now view an IMS database as an XML Schema and likewise IMS Records as XML documents valid to the corresponding XML Schema.

Figure 4: IPO schema and segments


4 of 17

3.1. About the sample applications and what they doThere are several sample applications as part of the lab with each of the samples serving a different purpose. In this chapter the classes are described and some sample code is also included to better explain what needs to be done.

Sample 1: IPOSeed

Seeding the Purchase Order database using XML-DB

IMS version 9 introduced IMS XML-DB, allowing the retrieval and storage of XML documents from standard IMS databases by using the provided retrieveXML() and storeXML() User Defined Functions within an IMS JDBC application.

The IPOSeed.java application uses the storeXML() UDF to insert three example purchase orders into the sample database:

* purchaseOrder1.xml * purchaseOrder2.xml * purchaseOrder3.xml

Each of these XML documents is valid (meaning it matches a template for structure and value) to the generated XML Schema IPOPSB-PurchaseOrder.xsd. If an XML document is not valid to the generated XML Schema, an exception is thrown when trying to insert into IMS.

1. Obtaining a Database Connection

A database connection is created using the DriverManager interface:

// Create a connection Class.forName("com.ibm.ims.db.DLIDriver"); String url = new String("jdbc:dli:demo.IPOPSBDatabaseView"); connection = DriverManager.getConnection(url);

Upon calling getConnection() on the DriverManager, the DLIDriver is used to fetch a connection based on the URL prefix jdbc:dli: indicating that this is an IMS Connection and therefore why the DLIDriver must be preloaded with a Class.forName() in order to register itself as handling the jdbc:dli: prefix. The rest of the url: demo.IPOPSBDatabaseView indicates where in the classpath to find the metadata describing the database.

System.setProperty("http://www.ibm.com/ims/schema-resolver/file/path", "com/ibm/ims/xmldb/sample");

A System Property is also set, indicating how and where to find the required XML Schema if needed. In this example, the XML Schema generated by the DLIModel utility, must exist as a file named IPOPSB-PurchaseOrder.xsd (the PSB and PCB names are combined into the XML Schema filename: PSB-PCB.xsd). The System property set in the statement above indicates that this file can be found in the demo directory.

2. Emptying the Database

To empty the database, the JDBC connection is used to issue the following DELETE SQL call. This call will remove all purchaseOrder segments from the PuchaseOrder PCB. Since purchaseOrder is the root of the database and no WHERE clause is specified all IMS Records are removed.

String query = "DELETE FROM PurchaseOrder.purchaseOrder"; Statement statement = connection.createStatement(); int updates = statement.executeUpdate(query);


5 of 17

The number of deleted segments are returned and printed to System out.

System.out.println(" deleted " + updates + " IMS Records");

3. Inserting the Seed Documents

To seed the database with some initial data, the JDBC connection is again used, however this time we issue an INSERT call to insert segments into the database. The orderNumber key field is defined as well as an XML document defining the rest of the IMS Record (key values are selected arbitrarily in this example).

String query = "INSERT INTO PurchaseOrder.purchaseOrder(orderNumber, storeXML(purchaseOrder))" + "VALUES (?,?)"; PreparedStatement pstatement = connection.prepareStatement(query);

insertDocument(pstatement, "11000123", "purchaseOrder1.xml"); insertDocument(pstatement, "11002334", "purchaseOrder2.xml"); insertDocument(pstatement, "11010437", "purchaseOrder3.xml");

The insertDocument() method sets the preparedValues (as a CharacterStream in the case of the XML document) and executes the statement. The number of inserted segments is returned and printed to System Out.

Finally, the connection is closed, the transaction is committed, and the application ends.

Sample 2: IPOQuery

Querying IMS as a Virtual XML Database

The Virtual XML Garden project, together with IMS XML-DB (supported in IMS Version 9), allows you to query an IMS database as a Virtual XML Database in Java. The IPOQuery.java application uses the Virtual XML Garden runtime to run XQuery expressions against an IMS database visualized as an XML database.

The sample runs as a JMP, reading XQuery expressions off the message queue, and returning the result.

A database connection is created just as in the POSeed application.

getConnection();

XQuery expressions are retrieved from input messages, defined by com.ibm.ims.xmldb.sample.InputMessage, read off the IMS message queue using the IMSMessageQueue class.

IMSMessageQueue messageQueue = new IMSMessageQueue();

while (messageQueue.getUniqueMessage(inputMessage)) {

// Read the query from input message String xquery = inputMessage.getString("query").trim();

With the desired XQuery expression, we call a runQuery() method where we have encapsulated the logic for query evaluation.


6 of 17

To fully integrate the Virtual XML code into IMS, the retrieveXML() UDF introduced in IMS V9 has been extended to resolve XQuery expressions. The UDF optionally takes the root segment to establish an XML context as described by the IMS V9 XMLDB documentation as well as an XQuery expression. The simplest example of established XML context is a query as follows:

SELECT retrieveXML() FROM PCB

In this simple query, the XML context is established at the PCB level. Adding an XQuery expression to the retrieveXML() UDF in this expression allows evaluation over the entire database. This is what our sample does with the passed in XQuery expression.

System.out.println("Evaluating: (" + xquery + ")");

// Wrap XQuery expression in simple SQL statement String query = "SELECT retrieveXML('" + xquery + "') FROM PurchaseOrder";

Using standard JDBC syntax we next use the established connection to create a statement and execute the constructed XQuery expression.

// Create JDBC statement and execute SQL with XQuery Statement statement = connection.createStatement(); ResultSet results = statement.executeQuery(query);

The result is returned as a CLOB (Character Large Object) in the result set. From the CLOB we retrieve a Reader, and read its contents into a CharArrayWriter (this could as easily be into an XSLT transformation, a Web page, SOAP result, etc).

It is only upon reading the contents of the CLOB that IMS database calls are issued and the results are generated; only enough of a result is generated to fill the amount we read. In this sense XQuery expressions are evaluated dynamically On Demand rather than all at once. This gives the programmer control over how much output is too much, able to stop the query at anytime.

CharArrayWriter outputWriter = new CharArrayWriter(); while (results.next()) {

Clob clob = results.getClob(1); Reader reader = clob.getCharacterStream(); char[] buffer = new char[1000]; int read = reader.read(buffer); while (read != -1) {

outputWriter.write(buffer,0,read); read = reader.read(buffer);

} }

Finally, the outputWriter is converted into a String and returned as the result of the XQuery expression.

String result = outputWriter.toString();

Lastly, the resulting string is inserted into an output message, defined by com.ibm.ims.xmldb.sample.OutputMessage, and inserted back onto the IMS message queue. The work done in this transaction is committed using the IMSTransaction class, the connection is closed, and the application ends.

// Fill and insert output message outputMessage = new OutputMessage(); outputMessage.setString("result", result);


7 of 17

messageQueue.insertMessage(outputMessage);

// Commit work done IMSTransaction.getTransaction().commit();

// Close our connection closeConnection();

Sample 3: IPORetrieve

Retrieving the XML documents in the Purchase Order database using XML-DB

The IPORetrieve.java application uses the retrieveXML() UDF to retrieve all the records in the Purchase Order database and displays the records as XML documents. This application obtains a database connection the same way as it is done in IPOSeed.java. Then it sets up the query statement:

String query = "SELECT retrieveXML(purchaseOrder) FROM PurchaseOrder.purchaseOrder";

Using standard JDBC syntax we next use the established connection to create a statement and execute the constructed XQuery expression.

// Create JDBC statement and execute SQLStatement statement = connection.createStatement();ResultSet results = statement.executeQuery(query);

The result is returned as a CLOB and is processed the same way as it is done in IPOQuery.java.

Sample 4: IPOInsertDelete

Inserting and deleting an XML document using XML-DB

The IPOInsertDelete.java application inserts a new XML document(containing a purchase order) into the existing Purchase Order database and displays all the records from the database as XML documents. It then deletes the newly inserted XML document and re-displays the database. The following query is used where key is a String variable containing the orderNumber of the newly inserted purchase order.

String query = "DELETE FROM PurchaseOrder.purchaseOrder WHERE purchaseOrder.orderNumber='"+ key +"'";

Statement statement = connection.createStatement();

int updates = statement.executeUpdate(query);


8 of 17

3.2. Compile the Java classes

1. Expand the src > scripts folder in the project and select the build_xmldblab_jar.xml file, click the right mouse button and select Run As > Ant Build.

2. Verify the successful execution of the Ant Build.

Figure 5: Successful Ant Build execution

3. Select the src > scripts folder and press F5 (or select Refresh from the context menu) to refresh the view and the xmldblab.jar should appear.

3.3. Upload the jar file

1. Select the xmldblab.jar file, click the right mouse button and select Copy.

2. Switch to the z/OS projects perspective and go to the Remote Systems Explorer.

3. Expand dallas > z/OS Unix Files > My Home, select dallas > z/OS Unix Files > My Home, click the right mouse button and select Paste.

4. Verify the successful upload of the xmldblab.jar.

Step 4: Run the IPOSeed application

Before we can run the IMS Java batch job that loads XML documents into an IMS database, it is required to start the java batch region to execute the sample program.

4.1 Create z/OS Project

Settings and JCL for the IMS Java Message regionThe message region for Java programs has to be started, but first its settings have to be changed to suit the needs for your group id (e.g. name, transaction class, etc.)The following steps will be described based on userid IMPOT62, so make sure that you use 6x for all the steps, where x is your assigned group id.

1. Switch to the z/OS projects perspective.

2. On the left hand side go to the z/OS projects explorer, click the right mouse button and select File -> New -> Project.

3. The new project wizard will open. Select z/OS > z/OS Project.


9 of 17

Figure 6: New z/OS Project

4. Click Next and enter IMSXMLDBzosproject as the project name. Make sure the Create an MVS subproject is selected too and click Finish.

5. The New MVS Subproject wizard will appear. Enter MVSSubproject as name, Select Do not associate the subproject with a property group and click Finish.

Figure 7: MVS subproject name and location

Figure 8: MVS subproject property group association

6. Expand the IMSXMLDBzosproject in the project explorer.


10 of 17

7. In the Remote Systems view expand the dallas > MVS Files > My Data Sets (IMPOTxx) view and Select the following dataset: IMPOT6x.XMLDB.JCL

8. Drag the selected files or the whole dataset over to the MVSSubproject. They should then appear under MVSSubproject.

4.2 Execute the Java Batch Program

The IPOSeed application will be executed as batch, which will actually insert data from XML documents into the database.

1. Double click the DFSJVMAP.JCL proclib member from the JCL library.

2. If the class name (end of line 1) is not IPOSeed, change it to IPOSeed. Save the changes.

3. Right click the JBPRUN1.JCL member and click Submit. Through the JCL changes it will be submitted with XMLxxJBP as the jobname.

4. Check for the Job submission confirmation window.

5. Check the job output of the XMLxxJBP job that was just submitted. Go to the Remote Systems Explorer to do that.

Figure 9: Refresh JES > My Jobs view in the Remote Systems Explorer


11 of 17

6. Double click the XMLxxJBP Job Output to check for the job output.

Figure 10: Job output of the successful executed JBP

4.3 Check the JAVAOUT and JAVAERR output files

1. Go back to Rational Developer for System z, into the z/OS Projects perspective, Remote Systems view at the right hand side and expand z/OS Unix Files > My Home.

2. Double click the JVM.ERR file to see if there were any errors.

Figure 11: JVM.ERR output file on successful execution

3. Double click the JVM.OUT file to see if there were any errors.

Figure 12: JVM.OUT file on successful execution

The IMS Java Batch Program that inserted XML documents as IMS Data using the decomposed XML feature finished successfully.

Step 5: Run the IPOQuery application


12 of 17

Before we can run the IMS Java Message program that allows to query XML documents from the IMS database that we created, it is required to start the java message region to execute the sample program.

5.1 Start the JCL for the IMS Java Message region

The IPOQuery application will be executed as online message program, which will actually query XML documents from the segments in the IMS database.

1. Double click the DFSJVMAP.JCL proclib member from the JCL library.

2. The IPOSeed class name has to be changed to IPOQuery in order to execute the XMLDB query program. Save the changes.

3. Right click the JMPRUN1.JCL member and click Submit. With the JCL changes it will be submitted as XMLxxJMP as the jobname.

4. The job submission confirmation will appear. In the Remote Systems explorer expand JES > My Jobs. If it is already expanded, select My Jobs, click the right mouse button and click Refresh (as an alternative the F5 button can be pressed instead using the context menu option).

Figure 13: Refresh JES > My Jobs view in the Remote Systems Explorer

5. Double click the XML6xJMP Job Output to check that the message region is running. A running job will be marked by a slightly different icon than a finished job.

Figure 14: Verification of running XML6xJMP Java Message Processing Region

6. Verify the job output of the running Java Message Processing Region.

Figure 15: Job output of running Java Message Processing Region


13 of 17

5.2 Test the transaction using a 3270 client

Test the transactionFor a quick test, the IMS transaction will be tested using a 3270 client.

1. In the Remote Systems view, right click on dallas and select Host Connection Emulator.

Figure 16: Launching the Host Connection Emulator

The Host Connection Emulator

The host connection emulator is a terminal emulator that automatically connects to a remote z/OS system for which you have an active connection in the Remote Systems view. You can use this feature to log on and send commands to a remote z/OS system.

If the Host Connection Emulator fails to connect due to security restrictions, please notify your class instructor. You will be given alternate instructions.


14 of 17

2. At the Welcome screen, enter IMSD and use the <CTRL> key to send the data to the mainframe.

Figure 17: Welcome screen of the dallas remote z/OS system


15 of 17

3. At the IMS Signon Screen, enter your userid and password, again use the <CTRL> key to send the data to the mainframe.

Figure 18: IMS Signon Screen

4. After successful signon to IMS enter the transaction code (IPOPGMxx) which includes to replace the xx with your assigned group number to adhere to the naming conventions, followed by a space and at least one more character which serves as input message to the IMS Java transaction.

Figure 19: IMS Transaction code with input data

5. Double click the XML6xJMP Job Output to check that the message region is running. A running job will be marked by a slightly different icon than a finished job.

Figure 20: IMS Transaction output data

The output here will only be a subset of the output, since only the first 80 characters will be displayed at the 3270 screen.

5.3 Check the JAVAOUT and JAVAERR output files


16 of 17

1. Go back to Rational Developer for System z, into the z/OS Projects perspective, Remote Systems view at the right hand side and expand z/OS Unix Files > My Home.

Figure 21: Expanded z/OS Unix Files > My Home

2. Double click the JVM.ERR file to see if there were any errors.

Figure 22: JVM.ERR output file on successful execution

3. Double click the JVM.OUT file to see if there were any errors.

Figure 23: JVM.OUT file on successful execution

The IMS Java Transaction with access to IMS DB through the IMS XML DB function runs. Feel free to scroll through JVM.OUT to see to complete output.

Appended are some more suggestions on how XQuery can be used with that IOP0xPGM transaction:o IPOPGM0x /purchaseOrdero IPOPGM0x for $x in /purchaseOrder where $x/@orderDate > '2006' return $x/billTo/name

o IPOPGM0x <names>{for $x in /purchaseOrder return $x/billTo/name}</names>

o IPOPGM0x for $x in /purchaseOrder/items/item return $xo IPOPGM0x <customers>{for $x in /purchaseOrder let $y := count($x/items/item) return <customer name="{$x/billTo/name/text()}">{$y}</customer>}</customers>

o IPOPGM0x <comment>Even this is a valid XQuery expression</comment>o IPOPGM0x <calculate>{8+7}</calculate>


17 of 17

Technology

IMS XML DB - Lab Manual