1

Dynamic Application Installation(Case of CMS on OSG)

• Introduction

• CMS Software Installation Overview

• Software Installation Issues

• Validation Considerations

• Publication of Software Installation

• Considering User Request

• SummaryBockjoo KimU of Florida

EGEE/OSG OPS. Workshop-3Sep. 28, 2005

2

Introduction

• Application software can be installed without publication and notification

• If software installation could be organized and managed in a consistent manner, one could share the responsibility and efficiently use the grid resources.

• This presentation shows how CMS software that is used by the CMS collaboration for physics analyses can be installed, maintained, and published on OSG in a managed way

• LCG counter part is doing this on LCG

3

An LCG/OSG Comparison

LCG/OSG LCG 2.6 OSG 0.2

GPT Vers. 3.1 3.2

Monitors MDS+GIP+BDII... MDS+(GIP),MIS-CI,ML

RB EDG (Condor-G)-everyone has one

Globus /opt/globus $GLOBUS_LOCATION

Software /opt/exp_soft/$vo $APP/$vo_user

WNTMP GlueSubClusterWNTmpDir $WNTMP

Data GlueCEInfoDataDir $DATA

4

An Overview of CMS Software Deployment on OSG

Local Scripts

A Remote Site

$app

Discovery Service

CMS Grid Users

Grid Middle Ware

GridCat

GridCat Client

CMSI Tool

RPMs

CMSI Tool RPMsCMS CMSI Repository

CSD-GUIOSG IS CMS IS

CSD-GUI : X509 Persistent Interface

MySQL DB

Deploy Scripts

Submit Condor-G

Local Scripts

5

CMS Software Installation Tool

• is used to install CMS softwares

• xcmsi is a tool to install CMS software written in PERL

• Arbitrary installation directory $CMS_PATH to relocate RPM packages

• No root privileges required • Typically 1 GCC compiler, 1 SCRAM (Soft.

Management Tool), ~40 CMS RPMs, ~30 LCG RPMs. ~4 GB space

• Essential ingredients to the installation are the RPMs

6

A Schematic of OSG Authorization and Resources

$app

MDS+GIP

Publish

OSG Monitoring

7

Implementation of CSD-GUI (CMS Software Deployment Grid User Interface for USCMS)

OSG Client/Condor-G

check arch

Execution Scripts

voms proxy globus clientGUI script(python)

GridCat Client

MySQLcondor_submit/condor_q

HTTPSCGI

Site Info.

Installation Table

condor_q/rmcsd upload

csd install.

csd verifycreate info file

undo upload.

undo install.

undo verify

8

CSD-GUI Features

• Managed Software Installation• Persistent Installation Task• Installation Control Using X509 Proxy• Installation Bookkeeping Using MySQL• Publish Installation Info in Text ->

– GIP and CMSSoftDB

• Switch Between OSG-ITB and OSG• Undo in CSD-GUI MySQL

Operation/GIP/CMSSoftDB• Non-DAG/Single Job Submission for

Success

9

CSD-GUI

10

Software Installation Table

11

Software Installation Issues

• Different Linux Flavors

• 64-bit at 32-bit mode

• Network Insulation

• Missing Client Tools on WN, e.g., wget, pm

• Different Shared File Systems– Some applications are sensitive to them

• Issues have to be fed back

12

Site Specific Issues• Dedicated Slot (Busy Site)

– Dedicated CPU slot– Dedicated Time slot

• Disk Space Issue : – Problem disk– Soft-linking

• Problem resolution occurs :– IM– Phone– Email– GOC

13

Validation

• A series of particular applications should run successfully

• Only partial validation is possible• An iterative process from user

feedback and testing• Different validation procedures can

provide redundant

14

Publishing Software Installation

• Three different types of publications– GIP publication : Job submission time

discovery– OSG publication (grid3 style) favored by

USAtlas : Job submission time discovery

– CMSSoftDB : for just in time discovery

15

Considering User Demand

• What if a user wants a particular version of software to be installed on weekends or holidays.

• A request submission interface is necessary ?

• How to authorize the request ?• What if site admin wants to install it ?--> Everything is done through

emails/meetings at the moment

16

Summary (1)

• Managed/Persistent CMS software installation on OSG is performed

• Interoperability is considered for both OSG and LCG job submission in mind

• Local Installation Information for Discovery Service

• Non-trivial Sites to be handled with

different methods and special arrangement

17

Summary (2)

• Use Real Physics Analysis for Validation• More validation is required as more grid

users will try to use the installation • Versatile deployment (retiring old and

deploying new) options should be considered

• With a careful planning and communications with counter part, dynamic software installation can be used for interoperable job submission