ENABLING KERBEROS SSO FOR

EMC® DOCUMENTUM® D2 APPLICATIONS IN A DISTRIBUTED

WEBLOGIC/WEBSERVER/CONTENT SERVER CLUSTER ENABLED FOR

HIGH AVAILABILITY

ABSTRACT

This white paper explains the process of enabling Kerberos Single Sign On in a distributed Content Server, WebLogic, and WebServer cluster and in an environment where load balance and high availability are enabled. This abstract appears as the online abstract for EMC.com/Powerlink.

April 2015

Copyright © 2015 EMC Corporation. All Rights Reserved. EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice. The information in this publication is provided “as is.” EMC Corporation makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license. For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com. Part Number H12274

`

2

TABLE OF CONTENTS

EXECUTIVE SUMMARY ............................................................................................................................3

AUDIENCE ..............................................................................................................................................3

TERMINOLOGY .......................................................................................................................................3

PREREQUISITES .....................................................................................................................................3

DEFINING SPN .......................................................................................................................................4

Active Directory ................................................................................................................................... 4

ACTIVE DIRECTORY SETUP .....................................................................................................................5

Apache WebServer Load Balancer Configuration ...................................................................................... 5

WEBLOGIC APPLICATION SERVER CLUSTER CONFIGURATION ...............................................................8

Configuring Kerberos SSO in a WebLogic cluster and Apache WebServer load balance environment ............... 8

Configuring Kerberos SSO on WebLogic application server instance ........................................................... 8

CONTENT SERVER CLUSTER CONFIGURATION ........................................................................................9

Configuring Kerberos SSO on Content Server Load Balance and Failover Environment .................................. 9

Configuring the Content Server for Single Sign On ................................................................................... 9

CLIENT MACHINE SETUP ......................................................................................................................10

Client machine setup using IE ............................................................................................................. 10

ENABLING SSO WHEN CO-EXISTENCE OF WDK CLIENTS AND D2 ON THE APPLICATION SERVER

CLUSTER ...............................................................................................................................................12

CONCLUSION ........................................................................................................................................13

REFERENCES ........................................................................................................................................13

`

3

EXECUTIVE SUMMARY

Kerberos is a computer network authentication protocol that allows individuals communicating over a non-secure network to prove their identity to one another in a secure manner. It is designed to provide strong authentication for client/server applications using secret-key cryptography. After using Kerberos to prove their identity, the client and server can also encrypt all communications to ensure privacy and data integrity as they go about their business.

In summary, Kerberos is a solution to network security problems. It provides the tools of authentication and strong cryptography over the network to secure information systems across entire enterprise.

The other most visible benefit of Kerberos for end-users is Single Sign On. The end user can sign on to the computer once and be automatically signed on to all applications on the computer. Kerberos accomplishes single sign on by storing credentials in a secure manner.

With Documentum D2 adding support for Kerberos on a WebLogic/WebServer/Content Server cluster enabled with load balance and high availability, the end users are automatically logged in to the repository using credentials stored in the Windows private credential area. Unlike other SSO solutions, Kerberos SSO does not present any authentication challenge to the user. The only authentication challenge that the user will encounter is when a user logs in to the desktop using Windows domain credentials.

This document also covers enabling SSO when wdk clients and D2 applications coexist in same repository.

AUDIENCE

This white paper is intended for engineers, support professionals, and customers. It provides a basic understanding of enabling Kerberos SSO in distributed cluster and load balance environments.

TERMINOLOGY

SSO: Single Sign On

Single sign-on (SSO) is a method of access control that enables a user to log in once and gain access to the resources of multiple software systems without being prompted to log in again.

KDC: Key Distribution Center

The KDC is a domain service that uses a directory to hold its account database and global catalog for referral to KDCs in other domains.

SPN: Service Principal Name

The SPN is a mechanism to communicate with the application server.

The service principal name (SPN) is the name by which a client uniquely identifies an instance of a service.

Before a client can use an SPN to authenticate an instance of a service, the SPN must be registered on the user or computer account that the service instance will use to log in. Typically, service principal names are unique identifiers for services in a domain.

PREREQUISITES This section lists environment details where the setup was configured and tested. This is a single working scenario, and not the requirement operating system.

Active Directory machine Windows 2008 Server R2 version that supports Kerberos v5 authentication protocol. Refer to Microsoft help for setting up Active Directory and Domain for Kerberos. Add the required computers and users to the Active Directory.

Content server machines

`

4

Windows Server 2008 R2 – Requires two virtual machines Install Documentum 6.7 SP2 or above version of Content Server after adding the machine to the

Kerberos domain.

Application server machines Windows Server 2008 R2 - Requires two virtual machines Install the relevant application server version after adding the machine to the Kerberos domain.

The D2 application is deployed on the application server machine.

Apache WebServer machines Windows Server 2008 R2 – Requires two virtual machines

Install the relevant Apache WebServer version after adding the machine to the Kerberos domain.

Load Balancer server machine Windows Server 2008 R2 – Requires one virtual machine Install the relevant Apache WebServer and configure it as a load balancer between the two

Apache WebServer machines after adding the machine to the Kerberos domain.

Client machine Windows XP or Windows 7 Install the supported browser (Internet Explorer or Mozilla Firefox) configured to access D2 with

Kerberos SSO support.

Note: The Kerberos SSO must also work for D2-based applications deployed on Web Logic/Webserver load

balancer in Linux and UNIX environments. The procedure for enabling the Kerberos SSO is the same.

DEFINING SPN

Active Directory

An SPN (Service Principal Name) is a unique name that identifies an instance of a service and is associated with the logon account under which the service instance runs. Windows 2003/2008 account names are not multipart as Kerberos principal names. Therefore, it is not possible to directly create an account of the name HTTP/hostname.dns.com.

Such a principal instance is created using service principal name mappings. In this case, an account is created with a meaningful name hostname and a service principal name mapping is added for HTTP/hostname.dns.com.

D2 uses browser SPNEGO support to implement Kerberos SSO. In this case, the browser requests for a service token from the KDC for the WebServer Load Balancer server. The browser prepares the Service Principal Name (SPN) in the following format: HTTP/fully qualified URL@REALM. For example, if the load balancer server Hostname is cs5-lb.ssotest.loc and the realm is SSOTEST.LOC, the browser framed SPN will be HTTP/cs5-lb.ssotest.loc@SSOTEST.LOC

AppServer1 1

AppServer1 1

AppServer2

WebServr2

WebServr1

Load Balancer

End-user

CS1

AppServer1 1

CS2

`

5

Figure 1.Deployment where Kerberos SSO is enabled (Each node represents a virtual machine where D2 is deployed on AppServer1 and AppServer2)

ACTIVE DIRECTORY SETUP

After setting up the Active Directory services and designating the domain to use Kerberos, perform the following setup.

Apache WebServer Load Balancer Configuration To create a user account for WebServer Load Balancer in the Active Directory: 1. Create a new user for Apache WebServer Load Balancer (e.g. wlbuser1)

2. On the Active Directory machine, select Start > Programs > Administrative Tools > Active Directory Users and Computers.

3. Expand the Kerberos domain, right-click the Users node, and select New > User.

4. Enter the user credentials and logon name details, and click Next.

Note: In the screenshot displayed at Step 3, SSOTEST.LOC is the Kerberos domain to which all the machines and users are added.

`

6

5. Enter the Password.

6. Click Finish.

`

7

7. Right-click the user and select Properties.

8. Set the following property for the user:

Use DES encryption type for this account

Generating keytab file for Apache WebServer Load Balancer

1. Open the command prompt and change the directory to C:\Program Files\Support Tools.

2. Run the following command: ktpass /pass <password> -out <keytab-file> -princ <SPN> -crypto DES-CBC-MD5 +DumpSalt -ptype KRB5_NT_PRINCIPAL +desOnly /mapOp set /mapUser <user-name>

where password - password of the user keytab-file - location to save the keytab file SPN - The SPN framed by the browser in the HTTP/hostname.dns.com@REALM format. For example, HTTP://cs5-lb.ssotest.loc@SSOTEST.LOC user-name - User name (Example: wlbuser1)

For example, you can run the ktpass command using the following parameters

ktpass /pass Password123 -out c:\wlbuser1.123.keytab -princ HTTP/cs5-lb.ssotest.loc@SSOTEST.LOC -crypto DES-CBC-MD5 +DumpSalt -ptype KRB5_NT_PRINCIPAL +desOnly /mapOp set /mapUser wlbuser1

3. After generating the keytab, go to User Properties to verify the SPN registered to the user. The user logon name now reflects the registered SPN.

4. After you register the SPN for the user, the Delegation tab is displayed when you view the properties of the new user in the User properties dialog box.

5. Set the following properties for the user

Trust this user for delegation to any service(Kerberos Only)

6. Copy the Load Balancer keytab file to the application server machines (See the next section for details on the location to copy).

`

8

WEBLOGIC APPLICATION SERVER CLUSTER CONFIGURATION

Configuring Kerberos SSO in a WebLogic cluster and Apache

WebServer load balance environment 1. Perform the following steps to set up the WebLogic cluster and Apache webserver load balancing:

a. Install WebLogic application server instance 1 on machine 1

b. Install WebLogic application server instance 2 on machine 2

c. Install Apache WebServer instance 1 on machine 3

d. Install Apache WebServer instance 2 on machine 4

e. Install Apache WebServer Load Balancer on machine 5

Note: For detailed information about setting up the WebLogic cluster and Apache webserver load

balancing, refer to the white paper on WebLogic Cluster.

2. Deploy D2 Application on the WebLogic cluster and set the relevant values in the dfc.properties file.

Configuring Kerberos SSO on WebLogic application server instance 1. Copy the Load Balancer keytab file(wlbuser1.123.keytab) generated using the ktpass command to the

<web-app-root>/WEB-INF folder.

2. If d2fs-trust.properties does not exist, create the file in the <Deployed_loc>/D2/WEB-INF/classes/ folder using a text editor. Open d2fs-trust.properties in the folder webapps/D2/WEB-INF/classes/ and add the following lines:

*.user=<administrator user> *.password=<encoded password> *.domain=<your domain> [not mandatory] #or for each repository <repository>.user=<administrator user> <repository>.password=<encoded password> <repository>.domain=<your domain> where:

repository corresponds to the the name of repository using Kerberos. user and password are the username and password of an inline Super User account in the repository.

domain is the domain of the repository and can be left blank for inline accounts. 3. To encrypt a password, type the following in your command-line window:

set classpath=%classpath%;<path>/d2.jar where <path> is the path to d2.jar. The command enables the encryption command on the application server. Next, type the encryption command as follows to output the encrypted password: java com.emc.d2.api.utils.GetCryptedPassword <password>

4. Navigate to <Deployed loc>/D2/WEB-INF/classes/ and open shiro.ini. If shiro.ini does not exist, create a copy of shiro_base.ini and rename it as shiro.ini. Make the following changes to shiro.ini:

a. Find the line D2–Kerberos.keyTabLocation and append =<location>, where <location> is the path to the keytab you copied to the machine. E.g. c:/kerberos

b. Add the lines: [main] X3–Kerberos=com.emc.x3.portal.server.filters.authc.

X3KerberosHttpAuthenticationFilter

X3–Kerberos.defaultRepository=<default repository> X3–Kerberos.servicePrincipal=HTTP/lb.d2ccsa.com

X3–Kerberos.krbConfLocation=c:/Windows/ KRB5.ini X3–Kerberos.keyTabLocation=<web-app-root>/WEB-INF/ wlbuser1.123.keytab X3–Kerberos.debug=true [urls] ##Authentication type

`

9

/** = X3–Kerberos 5. Modify the krb5.ini file with the following details:

default_realm - Specify the Kerberos domain name realms section must point to the KDC server

6. Restart application server instance 1.

7. Follow the same procedure to configure Kerberos on both application server instances.

CONTENT SERVER CLUSTER CONFIGURATION

Configuring Kerberos SSO on Content Server Load Balance and Failover Environment

Perform the following steps to set up the Content Server load balance and failover environment:

a. Install Content Server on Machine1.

a. Configure the docbroker and repository on Machine1.

b. Install Content Server on Machine2.

c. Configure the docbroker on Machine2.

d. Run the CFSConfiguration utility on Machine2.

Note : For detailed instructions about setting up Content Server for Load Balancing and Failover cluster,

refer to the EMC Documentum Content Server installation guide located at https://support.emc.com

Configuring the Content Server for Single Sign On D2 do not use the Kerberos part of the Content Server. We only have Kerberos authentication at Application server

level.D2 implements Kerberos SSO authentication using DFC Principal Authentication. In principal authentication, there are 2 sets of user credentials involved. One is for the principal, and the other is for the user we will impersonate. This allows D2 to use a specific inline repository super user account to authenticate itself via Kerberos and then impersonate the inline password defined user accounts. Hence we need to create a user with the same name wlbuser1 as in Step1 (Page6) in your repository. Follow the below steps to create the user in the repository.

Creating users in the repository 1. Start Documentum Administrator and connect to the repository where you want to create users.

2. Select Administration > User Management > Users.

3. Perform one of the following steps:

To create a new user, select File > New > User. The New User page is displayed. To modify an existing user, select the user, then select View > Properties > Info. The

User Properties page is displayed.

4. Enter the user information for the user in the User Properties page. For a detailed description of all fields, see the Documentum Administrator User Guide.

For Kerberos users, specify different values for the User Login Domain and the User Source fields. Do not use the values defined for accounts that do not use Kerberos authentication.

User Login Domain - The domain in which the user is authenticated. This is a Windows domain.

User Source - Specifies how the server authenticates the username and password. In our scenario, the value must be set to inline password.

5. Click OK.

Creating User in the KDC

`

10

Create a user account in the Kerberos Active Directory for users who log into the end user machine. For information about creating such a user account, see the Create User Account for WebServer Load Balancer in the Active Directory section.

CLIENT MACHINE SETUP Edit the %WINDIR%/system32/drivers/etc/hosts file and map appserver ip to its fully qualified domain name.

Note: This step is required only if the DNS is not configured to resolve the fully qualified hostname to the

application server IP address.

Client machine setup using IE 1. Launch Internet Explorer.

2. Select Tools > Internet Options and click the Security tab.

3. In the IE7 and IE8 browsers, clear the selection of the Enable Protected Mode option for Internet and Local Intranet.

4. Select the Local intranet icon and click Sites.

`

11

5. In the Local intranet dialog box, ensure that the Include all local (intranet) sites not listed in other zones option is selected, and click Advanced.

6. Add the Load Balancer URL used to access Documentum D2, to the Websites list (Example:

http://cs5-lb.ssotest.loc ).

7. Click Close

8. In the Internet Options dialog box, click the Advanced tab and navigate to Security settings.

`

12

9. Select the Enable Integrated Windows Authentication (requires restart) option.

10. Click OK and restart the browser.

ENABLING SSO WHEN CO-EXISTENCE OF WDK CLIENTS AND D2 ON THE APPLICATION SERVER CLUSTER

With the current implementation of Kerberos SSO for D2, if a customer wants to migrate to D2 from webtop or any other wdk client, Kerberos SSO will not work for the existing accounts which are SSO enabled. In other words, we cannot support co-existence of D2 & WDK APP's if customers using WDK Apps have "User Source" associated to LDAP for the User Accounts in the docbase. D2 does not use the Content Server Kerberos plugins. We support Kerberos authentication at Application server level. D2 implements Kerberos SSO authentication using DFC Principal Authentication. In principal authentication, there are 2 sets of user credentials involved. One is for the principal, and the other is for the user we will impersonate. This allows D2 to use a specific inline repository super user account (or principal) to authenticate itself and then impersonate the

inline password defined user accounts. Hence we need to create a user with the same name wlbuser1 as in Step1 (Page6) in your repository.Follow the below steps to create the user in the repository

Scenario where Customer using WDK's App's with SSO enabled:

1. Imagine customer using WDK App's like Webtop has enabled Kerberos SSO.

2. While creating user account in the docbase, selected User Source as either LDAP or dm_krb or Run the LDAP job to

sync the user accounts from AD to Docbase which is an ideal scenario for customers.

3. Kerberos SSO work's fine for WDK App's in the above scenario.

`

13

Co-existence of D2 with WDK Applications

If customers want to deploy D2 on the above environment and enable SSO for D2, it will not work because D2 doesn't

use the Content Server Kerberos plugins. It will return an authentication error, because DFC Principal Authentication

only works for users with “User Source” set to “Inline Password.”

Work Around:

For D2 to work, the "User Source" of the user accounts in the docbase should be manually updated as "Inline Password"

and set the password.

CONCLUSION

This white paper provides detailed steps on enabling Kerberos SSO in a complex Content Server, WebLogic, and WebServer cluster that is configured for Load Balance and High Availability.

References

Use the following references to access articles on enabling Kerberos sso authentication

http://docs.oracle.com/javase/6/docs/technotes/guides/security/jgss/tutorials/Troubleshooting.html http://support.microsoft.com/kb/262177

`

14

CONTACT US To learn more about how

EMC products, services, and

solutions can help solve your

business and IT challenges,

contact your local

representative or authorized

reseller—or visit us at

www.EMC.com.

www.EMC.com

Copyright © 2015 EMC Corporation. All Rights Reserved.

EMC believes the information in this publication is accurate as of its publication date. The information

is subject to change without notice.

The information in this publication is provided “as is.” EMC Corporation makes no representations or

warranties of any kind with respect to the information in this publication, and specifically disclaims

implied warranties of merchantability or fitness for a particular purpose.

Use, copying, and distribution of any EMC software described in this publication requires an

applicable software license.

For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on

EMC.com.

EMC2, EMC, the EMC logo, and the RSA logo are registered trademarks or trademarks of EMC

Corporation in the United States and other countries. All other trademarks used herein are the

property of their respective owners.