Web Services Integration and the SAS® Drug Development ...Web Services Integration and the SAS® Drug Development Platform Chris Olinger, d-Wise Technologies, Raleigh, NC John Leveille,

1

Web Services Integration and the SAS® Drug Development Platform Chris Olinger, d-Wise Technologies, Raleigh, NC John Leveille, d-Wise Technologies, Raleigh, NC

ABSTRACT The SAS ® Drug Development (SDD) platform is an extensible, web-based application for the hosting, processing, and management of clinical and pre-clinical data. The system is 21CFR Part 11 compliant, runs over the Internet, integrates with SAS, and provides direct interfaces for controlling the various components of the system. The system employs Java EJB technology, Web Services, and a WEBDAV interface to deliver Internet-based clinical functionality, and to provide integration points for controlling basic functionality such as auditing, uploading, downloading, and versioning. This presentation discusses the application of Web Services and reviews the primary methods for integrating with SAS Drug Development (as there are several). The paper is intended for individuals with Java/Web experience, and for SAS programmers that would like to see what you can do with the platform.

INTRODUCTION con·nec·tiv·i·ty

1. The quality or condition of being connected or connective.

Since the advent of the Internet years ago, people have touted the Web as providing the framework for enhanced connectivity between people, systems, and institutions. In general, this claim has been true – the Internet has enabled information flow in a way that no other invention before it has ever done. With the click of a button, a person can visit a locale on the other side of the world. Universities share data, course work, and research across wide expanses. Business systems, if designed properly, interact and share information in a flash. However, in general, getting disparate systems to talk to each other over the internet is problematic. Systems must be designed in such a way as to allow connective behavior. If they are not designed correctly then getting system A to talk to system B is about as easy as it used to be – not very.

WEB SERVICES AND XML What is a Web Service? A Web Service is the answer to the following question: “How will we connect our business systems across the Internet in a general manner”? A few courageous souls wrote programs in the early days that attempted to communicate with web-based systems using HTML – the language of the web. However, the language of the web (HTML) was for people, and for web pages. The very thing that made the web ubiquitous – its heterogeneous nature – worked against these efforts to write reliable inter-company software systems. Web technologists realized that a more flexible, meaningful, and uniform markup language could answer the question of how a program could talk to the web. Enter XML, the eXtensible Markup Language, which looks a lot like HTML, but allows computer systems to reliably read and write data in a standard format. With XML as the building block, programmers began to link systems together across the enterprise using web servers. These server components were designed to service web requests with a dynamic XML response, and the term Web Service was coined. The idea behind web services is connectivity – being able to connect disparate systems together in a cohesive framework. SDD uses these ideas to surface its architecture to any client that wishes to integrate with it. The SDD is Java-based, but its web services can be used by Java, Microsoft .NET, and other client environments with relative ease.

SAS DRUG DEVELOPMENT SAS Drug Development (SDD) is part of a class of new applications from SAS that are vertical in nature and exploit technologies beyond the traditional SAS notion of data step and procedure. SDD was designed from the beginning to host clinical data in a web environment, provide analysis capabilities, and to be 21CFR Part 11 compliant. A traditional SAS user may look at SDD and say “where are the proc’s?” They are there, but they are only a small part of the overall solution. Some of the features of SDD include the following:

• Internet collaboration framework • SAS Analysis applications and framework • Auditing • Data storage, management, and versioning support • Part 11 compliance

ApplicationsNESUG 17

2

• Extensive API support for integration and customization

It’s the last point that we are going to talk about in this paper. The SDD team realized long ago that creating a Part 11 solution for Clinical Warehousing was only part of the picture. It doesn’t matter how good of a product the platform is if clients cannot integrate SDD with their existing legacy business systems. You must be able to use data from multiple sources, retrieve data from SDD and transfer it to other systems, and launch existing software on data stored in SDD. So, how do you integrate with SDD? There are actually various ways to connect and interact with SDD. It all depends on what you want to do and the level of interaction desired. In general, there are six ways that a user can directly integrate with SDD. A client can:

• Attach a web drive that interacts with SDD’s WEBDAV server • Call the native EJB interfaces that SDD exposes • Call the SDD Extension Servlet to invoke server-side functionality • Call the SDD EJB interfaces through a web service • Build what’s known as a GAI (Global Accessor Interface). A GAI is used by SDD Applications to read and

write data to and from other external systems. • Install custom applet applications and customize the SDD toolbars

We will go over the first four of these scenarios later in this paper.1

SDD ARCHETECTURE SDD is constructed as an N-tiered web application. There are usually at least three physical servers contained in any given implementation: a web server, a data server, and a SAS server. A pictorial view of the architecture is show in Figure 1. The diagram is relatively complex. You can simplify this by thinking of the platform as three distinct partitions: a server running BEA Weblogic (application server), a server running SAS (analytics, reporting, data transformations), and a data server running both Xythos (for the WEBDAV implementation ) and Oracle (for data storage). SDD can be configured in a variety of ways. SAS provides an ASP solution where all data is hosted on a secure, private network at SAS. You can run on virtual hosts (for smaller implementations), or a set of dedicated servers. You can also host the implementation at your site directly on your own hardware. Your SDD integration options are largely determined by the SDD setup that you have chosen. If you are interacting with SDD from outside of the SDD firewall then your options are more limited. If you are behind the firewall on an internal network then your options are more flexible. The sections that follow describe various integration scenarios and some of the pros and cons of each.

1 Building a GAI interface is fairly complex, and is left as an exercise to the reader ;) Installing client apps is also fairly involved. Call the authors if you are interested in knowing more about it.


3

Figure 1.

SCENARIO 1: WEBDAV AND WEB DRIVES WEBDAV stands for “Web-based Distributed Authoring and Versioning”. The protocol defines extensions to HTTP that allow users to collaboratively edit and manage files on remote web servers. It provides mechanisms for writing files and directories (HTTP gave us the ability to read files), and for versioning of files. SDD uses an implementation of WEBDAV that is produced by a vendor named Xythos. Xythos provides a complete implementation of the WEBDAV protocol, and functions as the SDD data layer. What does this mean to the standard SDD user? For starters, it means that you can version files in SDD. It also means that you can use WEBDAV to access and store files in SDD without going through the standard browser-based upload and download procedures. Usually, you must upload and download data directly from SDD to your desktop before you can do something with it that is outside of SDD. With a web drive, you can map a directory folder to SDD directly and access the data (read and write if you so choose) without having to physically download the data to your desktop first. There are probably a few people who, when hearing this, scratch their heads and say “but that’s not Part 11 compliant!” However, in this case it is. All data that is read from and written to SDD via a web drive is filtered, and audited. So, even though it looks like you are just writing to a folder, SDD is really behind the scenes auditing the reading and writing of the file. In this regard, there is no difference between using a web drive or physically uploading and downloading the data from SDD using a web browser. What isn’t Part 11 compliant is what you do with the data once you have copied it to your desktop. If you copy down a data file from SDD using either a web drive or through the export facility, and you then manipulate the data using a desktop tool, only the uploads and downloads are audited. What you do on your desktop is not tracked by SDD. If you want to track desktop client behavior or external system behavior then you’ll have to use another integration method.

EJB Based Services

Scheduling

Object Model

Audit Trail

Meta-Data

Remote Data Access

Remote Doc Access

Filter

WEBDAV Server

P21 Web Server

JSP / HTML

SOAP / WSDL

DE/PPV Com

Servlet

Extension Servlet

Desktop Clients /

Web Folder

Browser

SOAP Enabled Clients

Firewall / HTTPS / 443

SDD Metadata Database

SAS Server(s)

Central User Server

Data Access O*C

Clintrial EDC

Documentum Intraspect

…


4

How does one connect a web drive? On the PC (Windows XP), this is trivial. Simply open Network Connections:

Select My Network Connections,


5

Select Add a network place, and follow the dialog to Choose another network location. Add the URL of the WEBDAV server, and you’re off and running:

You can then treat the newly mounted web drive just like any other file system. You can use Desktop Explorer, Microsoft Word, or any other application that can read files from the file system.


6

Synopsis: The benefits of this approach are:

• It’s simple – anyone can do it. No need for programming expertise.

• Auditing is maintained throughout the session.

• It’s much easier than downloading or uploading files to SDD via a web browser.

• Any 3rd party tool can use data contained in SDD.

The downsides to this approach are:

• Applications that run on the desktop are outside of the SDD Part 11 compliance infrastructure. Tight integration of 3rd party to SDD applications is not possible.

• There is no way to add customized audit messages to the SDD audit trails.

SCENARIO 2: THE EJB INTERFACES If Scenario 1 does not meet your needs then you can use the SDD EJB interfaces to integrate directly with the SDD Object Model. This approach is more complex, and you will need someone with Java and XML expertise. What is an EJB? EJB stands for “Enterprise Java Beans”. In Sun’s words: “Enterprise Java Beans (EJB) is a server-side component architecture for the development and deployment of distributed object systems for the Java platform. Applications written using the EJB architecture are scalable, transactional, and multi-user secure.” What does this mean for you? It means you can make Java calls to directly manipulate objects in SDD. The SDD API consists of the following Java packages:

• The Object Model – allows manipulation (add, remove …) of objects in the SDD

• Platform – allows you to set and get SDD system properties

• Auditing – allows for viewing and writing audit messages for objects in SDD

• Archiving – provides the SDD archival interfaces

• Logging – provides tools for logging messages to the SDD logs

• Searching – provides an interface for executing a document search

• Security – allows for interaction with the SDD security model

• WEBDAV– surfaces the underlying WEBDAV interfaces For this scenario, we are going to concentrate on the auditing package. In fact, the example we are going show does nothing more than query an object in SDD for its audit records and then display them.

EJB OVERVIEW EJB’s are designed to allow the invocation of methods on objects on remote servers. To facilitate this, you must think of an EJB as an object with a client “part” and a server “part”. A client wishing to use an EJB must interact with the client part. The client part is responsible for communicating to the server part. Sometimes, the server part is on the same machine as the client part, and sometimes it isn’t. The client part is referred to as the “Remote” interface. The server part is referred to as the “EJB Object”: Figure 2.

Server B Server A

Your Application

Back-end Business Systems

EJB Remote Interface

EJB Object

Internet


7

In order for the Remote Interface to talk to the EJB Object there must be a JNDI server running on an accessible port. JNDI stands for “Java Naming and Directory Interface”. JNDI is a generic set of interfaces in Java that manage name and object lookup. All EJB’s are registered in a JNDI tree so that they may be located by potential callers. For instance, if Server A wants to call a method on an EJB in Server B, it must first “locate” the EJB in Server B’s JNDI tree. Once the object has been located, it can be instantiated and the method can be invoked.

EXAMPLE In this example, we are going to write code that lets you pass in the path of an object in SDD. We will then query the object’s audit records and display them. The general algorithm is:

1) Connect to the JNDI server using what’s known as an InitialContext 2) Query for the Audit EJB on the InitialContext 3) Search for the audit records 4) Produce HTML from the returned XML document containing the audit records

Step 1: Create the InitialContext The InitialContext must be configured with a valid SDD userid and password and must also be configured with the URL and port of the JNDI factory: / / our connect i nf or mat i on f or EJB' s. I n r eal i t y , t hi s i nf o woul d be st or ed i n / / a dat abase or a pr oper t i es f i l e somewher e St r i ng HOST = " 12. 345. 67. 89" ; St r i ng USER = " admi n" ; St r i ng PASS = " I amAPasswor d" ; Hasht abl e ht = new j ava. ut i l . Hasht abl e( ) ; / / f or WebLogi c ht . put ( Cont ext . I NI TI AL_CONTEXT_FACTORY, " webl ogi c. j ndi . WLI ni t i al Cont ext Fact or y" ) ; ht . put ( Cont ext . PROVI DER_URL, " t 3: / / " +HOST+" : 800" ) ; / / user / passwor d t hat has access t o audi t s ht . put ( Cont ext . SECURI TY_PRI NCI PAL, USER) ; ht . put ( Cont ext . SECURI TY_CREDENTI ALS, PASS) ; Cont ext c t x = nul l ; / / her e we get a cont ext , based on our cr edent i al s , t hat poi nt s t o t he SDD / / JNDI f act or y t r y { c t x = new I ni t i al Cont ext ( ht ) ; } cat ch( Except i on e) {

… } Step 2: Get the Audit Bean The InitialContext allows us to look up the Home object as a general object. You look up an EJB by giving its path in the JNDI tree. The path in the JNDI tree is determined by the author of the EJB. After the lookup returns a generic object we must down cast it to the specific Home Interface type. Next, we must create the EJB Object by calling the create method on the Home Interface. This returns to us the Remote Interface of the EJB Object.. Audi t audi t = nul l ;

t r y { / / Get a r ef er ence t o t he Audi t Bean Obj ect r ef = ct x. l ookup( " ej b/ p21/ Audi t " ) ; / / Get a r ef er ence f r om t hi s t o t he Bean' s Home ( l ocal ) i nt er f ace Audi t Home home = ( Audi t Home) Por t abl eRemot eObj ect . nar r ow( r ef , Audi t Home. cl ass ) ;


8

/ / Cr eat e an I nt er est obj ect f r om t he Home i nt er f ace audi t = home. cr eat e( ) ; } cat ch ( Except i on e) {

… }

Step 3: Get the Audit Records from SDD You should now be able to talk to SDD (assuming that everything went right in the previous steps). This step just calls a method on the Remote Interface. The resulting audit records are returned as an XML document:

/ / get t he audi t r ecor ds i n JDOM XML f or mat St r i ng pat h = “ / SDD/ MyDi r ect or y/ MyObj ect ” ; Document d = nul l ; t r y { d = audi t . sear chByPat h( pat h) ; } cat ch ( Except i on e1) {

… }

Step 4: Render the XML Audit Records as HTML There are various ways that a person can render HTML from XML. The example below uses JDOM to do it the brute force way: Ser vl et Out put St r eam out = r esponse. get Out put St r eam( ) ;

out . pr i nt l n( " <ht ml ><body>" ) ; / / t he r et ur ned adui t r ecor ds ar e i n a JDOM document . Sur f i t . El ement r oot = d. get Root El ement ( ) ; Li s t chi l dr en = r oot . get Chi l dr en( " Audi t Recor d" ) ; I t er at or i = chi l dr en. i t er at or ( ) ; i nt count = 0; / / i t er at e over t he r et ur ned el ement s whi l e( i . hasNext ( ) ) { El ement e = ( El ement ) i . next ( ) ; St r i ng dat e = e. get At t r i but eVal ue( " r ecor dDat e" ) ; St r i ng ver bat i m = e. get Chi l dText ( " Ver bat i m" ) ; / / f i r s t t i me t hr ough, set up t he t abl e i f ( count == 0 ) { out . pr i nt l n( " <h3>Audi t Recor ds f or : " + pat h + " </ h3>" ) ; out . pr i nt l n( " <t abl e bor der =1><t r ><t d wi dt h=20%><b>Dat e</ b></ t d>” ) ;

out . pr i nt l n( “ <t d><b>Audi t Message</ b></ t d></ t r >" ) ; } / / wr i t e a r ow out out . pr i nt l n( " <t r ><t d>" +dat e+" </ t d><t d>" +ver bat i m+" </ t d></ t r >" ) ; count ++; } / / end t hi ngs up i f ( count == 0 ) out . pr i nt l n( " <h3>No r ecor ds f ound f or : " + pat h + " </ h3>" ) ; el se out . pr i nt l n( " </ t abl e>" ) ; / / we' r e done out . pr i nt l n( " </ body></ ht ml >" ) ;

The Results:


9

If everything went well then you should see a table of audit messages for an SDD object. For instance:


• Standard Java calls to interact with SDD

• Interaction over an Intranet is easy

• EJB handles all of the issues regarding marshalling of data, and remote function calls

• Highly controlled technique is good for integrating external systems with SDD

The downsides to this approach are:

• EJB requires an open port (in our example, port 800). If your client code is on the opposite side of the firewall from the SDD then this will not work.

• EJB communicates using a protocol called RMI over IIOP. Forget the acronyms; currently, you cannot do EJB over a secure channel. Everything is unencrypted. This may or may not be a problem given your setup.


10

SCENARIO 3: THE EXTENSION SERVLET Chances are that in any given SDD customization project you will not have an open port to work with. It is also likely that an unencrypted option is not appropriate. Unless your IT department has granted you access to the ports that you need, you are more than likely out of luck. What do we do in this case? The only port open by default on an SDD instance is the one that provides the HTTPS service. This means that in order for us to interoperate with the SDD interfaces we must be able to do it with a URL. This can be accomplished in two different ways. We can use a Web Service (more on that later) or we can use the SDD Extension Servlet. The Extension Servlet is a Servlet that is running in SDD whose only goal in life is to forward to “extensions” that have been installed on the SDD server. What is an SDD extension? An SDD extension is a set of code, written as a Servlet, which is stored and run on the SDD server. This means that extensions can call any of the SDD EJB interfaces without having to have a special open port. This also means that any network traffic between the EJB’s and SDD is local traffic only. Authentication is handled by the same login form as a normal SDD URL request would use. Figure 3.

In this scenario, your application calls (or forwards) to a Servlet running in the SDD server. The Servlet then performs the tasks that need doing with SDD. There is no need for an open port other than HTTPS.

EXAMPLE In this example, we are going to write a Servlet that lets you pass in the path of an object in SDD. We will then query the object’s audit records and display them. The general algorithm for the extension Servlet is:

1) Connect to the JNDI server using an InitialContext with default properties 2) Query for the Audit EJB on the InitialContext 3) Search for the audit records 4) Produce HTML from the returned XML document containing he audit records

This looks remarkably similar to the EJB example. In fact, the only difference in this algorithm and the EJB algorithm is how we instantiate the IntialContext. In this example we do not provide a port or credentials. This will force the InitialContext to use the default JNDI tree, and the SDD local EJB implementations. To install the Extension Servlet you must do the following:

1) Create the extension as a Servlet and add it to a Jar file 2) Drop the Jar file in a directory on the SDD server 3) Configure SDD to look in the directory to resolve extension requests2

2 Installation of an extension involves configuring the SDD registry to specify the class of the extension and the name it will be recognized by. If you are interested in learning about this step then please contact the authors.

Server A

Server B

Your Application

Back-end

Business Systems

EJB URL / HTTPS

Internet

Extension Servlet


11

Step 1: Create the InitialContext The InitialContext must be configured with the defaults only. The port and host, userid and password fields are left out: Cont ext c t x = nul l ; / / her e we get a cont ext , based on our cr edent i al s , t hat poi nt s t o t he SDD / / JNDI f act or y t r y { c t x = new I ni t i al Cont ext ( ) ; } cat ch( Except i on e) {

… } From this point on, the example is exactly the same as the previous EJB example. To run the example, invoke the SDD URL: https://mysddhost.com/p21/extender/AuditExtension?path=/SDD/MyFolder/MyData. AuditExtension is the name of the extension that we used when we installed it in the SDD registry.3 One difference that needs to be highlighted is the lack of credentials. If you run this URL then you would first be routed to the SDD login page. After you provided your userid and password you would be forwarded to the extension Servlet. The results of the query would be the same – a listing of audit messages for the object that you passed in:

3 The name of extension in the SDD registry maps the name AuditExtension to the class of the extension to be run.


12


• EJB calls are restricted to the SDD server eliminating unencrypted network traffic and additional public ports

• Easy to call a URL

• Easy to add a new extension to the system The downsides to this approach are:

• If you don’t want to log in each time you call the URL then you must pass credentials in the URL. Adding credentials to form based authentication can sometimes be tricky

• Requires that you install the extension. If SDD has been IQ/OQ’ed and is locked down then this might prove to be a barrier

SCENARIO 4: WEB SERVICES What is a Web Service? One definition defines a Web service as a mechanism for discovering and executing functionality (methods, procedures) over the Internet. In this regard, a Web Service is much like a remote procedure call. In our case, a Web Service is a mechanism for calling the SDD EJB interfaces from both Java and non-Java clients. For instance, I want to call the SDD API from VB or .Net. How do I accomplish this? How do I go about connecting two totally disparate systems? The answer lies in a plethora of acronyms: XML, SOAP, WSDL, and UDDI. From Webopedia.com:

• XML (eXtensible Markup Language) defines the structure of a human readable markup language enabling a regular message structure that is easily parsed by computer systems.

• SOAP (Simple Object Access Protocol) defines “a lightweight XML-based messaging protocol used to encode information in Web Service request and response messages before sending them over a network. SOAP messages are independent of any operating system or protocol and may be transported using a variety of Internet protocols, including SMTP, MIME, and HTTP.”

• WSDL (Web Service Description Language) specifies “an XML-formatted language used to describe a Web service's capabilities as collections of communication endpoints capable of exchanging messages. WSDL is an integral part of UDDI, an XML-based worldwide business registry. WSDL is the language that UDDI uses. WSDL was developed jointly by Microsoft and IBM” .

• UDDI (Universal, Description, Discovery and Integration) defines “a Web-based distributed directory that enables businesses to list themselves on the Internet and discover each other, similar to a traditional phone book's yellow and white pages.”

Figure 4 depicts how the component pieces fit together in a Web Services scenario. Figure 4.

Server A

Server B

Your VB App

Back-end Business Systems

EJB

Internet

Web Service Servlet

WSDL

SOAP/XML over HTTPS

EJB SOAP Proxy

.NET SOAP Proxy

Generate


13

UDDI is mentioned here because it is an important part of the complete picture of Web Services, although you would probably not use it in your SDD integration projects. When the architects behind Web Services imagined different companies interacting with each other’s computer systems they figured that there should be a provision in Web Services for helping business partners find one another. It can be thought of as an attempt at creating a directory for the virtual market place. Once a client locates a service on the web it has the need to describe the service in detail. This is where the WSDL file comes in. The WSDL file can be used by clients to create a properly formed SOAP XML message to send to the Web Service. In the case of SDD, one option for generating the WSDL file for the Audit Bean is to use the tools that come with the Weblogic application server (the default application server for SDD). Servicegen4 is a powerful tool that allows you to point at an EJB and then generate the component parts of the Web Service (the Web Service Servlet, the EJB SOAP Proxy, and a WSDL file for the EJB). After you run the Servicegen task one time you will get an EAR5 file. The Web Services servlet and the EJB SOAP proxy are contained inside the ear file along with various helper classes for marshalling data between the SOAP message and the Java environment on the server. The next step in this scenario is to deploy the ear file to the server. After you have deployed the file you can use your web browser to query the Web Service for the WSDL that describes it. Simply visit the URL

https://mysddhost.com/mywarfile/SDDAudit?WSDL and the server returns the WSDL describing the Audit Bean. At this point you can take the WSDL file to a variety of client programs and use it as a programming interface to the SDD Audit Trail. Microsoft Visual Studio .NET is a good example of a client programming environment that is Web Services aware. All you have to do is include the Audit Bean WSDL file in your .NET project and start programming. The .NET toolset automatically generates the .NET SOAP Proxy component so that calls to the Audit Bean appear just like regular calls to another object in your C# or VB program. Of course, under the covers, the proxy on the client side is taking each of your calls, translating it into a SOAP message, and sending it off to the server for processing.

EXAMPLE The example for this scenario is fairly complex. For brevity’s sake, we are going to omit the code samples in favor of an interactive demo.

4 Servicegen is an Ant task. Ant is an open source build utility that utilizes XML configuration files to execute build related tasks in Java. 5 Enterprise Application Archive. This is a zip file that contains all the component parts of a web application, and then some.


14

CONCLUSION The SAS Drug Development platform is one of a set of new applications from SAS Institute. It incorporates a suite of technology to provide an open and secure environment for the hosting and analysis of clinical data, beyond the typical notion of SAS procedures. Using the techniques described in the paper enables you to extend and integrate the platform with external client software systems.

ACKNOWLEDGMENTS Special thanks to the SDD development group for putting up with many pesky questions. Y’all are the best.

CONTACT INFORMATION Examples for this paper can be downloaded at: http://www.d-wise.com Your comments and questions are valued and encouraged. Contact the authors at: Chris Olinger d-Wise Technologies, Inc. 3115 Belspring Lane Raleigh, NC 27612 919 696 3276 (W) 919 510 8391 (F) [email protected] http://www.d-wise.com

John Leveille d-Wise Technologies, Inc. 3115 Belspring Lane Raleigh, NC 27612 919 880 9068 (W) 919 510 8391 (F) [email protected] http://www.d-wise.com

SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration. Other brand and product names are trademarks of their respective companies.


Documents

Web Services Integration and the SAS® Drug Development ...Web Services Integration and the SAS® Drug Development Platform Chris Olinger, d-Wise Technologies, Raleigh, NC John Leveille,