Securing Science Gateways

July 2011

Victor HazlewoodNational Institute for Computational Sciences

University of Tennesseevictor@utk.edu

Matthew WoitaszekComputational and Information Systems Laboratory

National Center for Atmospheric Researchmattheww@ucar.edu

2

Securing Science Gateways Overview· Work initiated by Science Gateway Group of TeraGrid,

led by Nancy Wilkins-Diehr· Goal to investigate securing science gateways that

balance security, developer ease-of-use, end user ease-of-use and improves security posture

· Present survey of TeraGrid science gateway implementations and security models

· Selected commsh integration with Globus GRAM· Performed pilot study for securing science gateways

with GridAMP science gateway on Kraken at NICS· Recommendations for science gateway security

beyond TeraGrid (XSEDE)· Paper at http://www.nics.tennessee.edu/~victor/SecuringGateways-TG11.pdf

3

Investigate Securing Science Gateways· Gateways have evolved from a web portal requesting

service on behalf of a user to using a “community account” shared by a number of portal end users

· Gateway developers want to employ a reasonably easy-to-use method to authenticate in order to gain access to compute resources

· Resource providers want gateways to use a secure and auditable method for submitting work and jobs for the gateway end users

· Additionally, TeraGrid has security and accounting requirements for Science Gateways

· How to strike the correct balance between development, use and security?

4

TeraGrid Science Gateways Survey· Over twenty-five TeraGrid science gateways are

registered and integrated with the TeraGrid. See https://www.teragrid.org/web/science-gateways/gateway_list

· Implementation details of twenty science gateways are reported in the survey table

· AuthN– Nineteen reported using community account– One reported using only end user account

· Middleware used– Thirteen reported using GRAM2– Two reported using GRAM4– Two reported using GSISSH– Two reported using both GRAM2 and GRAM4– One did not report the grid middleware service used

5

TeraGrid Science Gateways SurveyGateway

NameGateway

InfrastructureAuthN CredentialCommunity User

Execution Data Movement

RENCI SciencePortal

JBoss;Axis2;JMS;CondorGApache; Django

X GRAM2; Condor Prestaged and Condor staging

CIG Java Web Start/WS Stack X X GSISSH; Custom LRM scripts

GridFTP; UberFTP

GridGhem - X X - -

ESG GridSphere; CoG; GAMA X GRAM2 GridFTP

GEON GridSphere; trebuchet; CoG X GSISSH; MRD GridFTP

LEAD GridSphere; CoG; GAMA; Opal X GRAM2;GRAM4 GridFTP; SCP; rsync

NBCR Condor-G; Stork; X GRAM2 GridFTP

NanoHub Portal; GAMA2; CoG X GRAM2 HTTP

NEES GridSphere; GRAM CLI X GRAM2 GridFTP

Neutron GridSphere; CoG X GRAM2 GridFTP

OLSG GridSphere; OGCE; Condor-G X GRAM2 GRAM2/GASS

QuakeSim WS; Axis2; Condor-G X GRAM2

Swarm Pegasus; Condor-G; RLS X GRAM2 GridFTP

SCEC Portal; Swift X GRAM2 GridFTP

SIDGrid GridSphere; CoG X GRAM2; GRAM4 GridFTP

GISolve GridSphere; OGCE; CoG X X GRAM2 GridFTP

TG Viz Portal; globusrun; G-U-C X GRAM4 GridFTP

AMP Condor-G; JGlobus X GRAM4 GridFTP; UberFTP

TeraDRE Condor-G; JGlobus; GRAM X GRAM2 Custom JAVA

C4E4 JGlobus X GRAM2 GridFTP

6

The Risks with Community Accounts· Compromise of the science gateway server· Compromise of the science gateway’s

community account short-lived GSI credential· Compromise of the science gateway’s

community account long-lived GSI credential· Unauthorized use of the science gateway

community account· Unauthorized command and/or application

execution on a resource· Unauthorized replacement of gateway

executables

7

Risk Mitigation Techniques

1. Separate the user and developer accounts– Personal accounts used for development– Gateway software installed in shared software

area via group permissions– ‘sudo’ available if required for testing

The gateway community user should only be able to execute authorized commands and software.

Risk Mitigation Techniques

2. Use the NCSA “commsh” restricted shell for community accounts– When set as a shell, filters all commands

according to rules (like sudo or using regular expressions)

– Integrates easily with GRAM4 and GRAM5, also works for accounts using gsissh and qsub

3. Disambiguate and audit community account users using SAML assertions– Currently informational; may be mandatory

8

9

Mitigation with commsh a New Tool for Securing GRAM· NCSA commsh tool

– When set as a shell, filters all commands according to rules (like sudo or using regular expressions)

– Integrates easily with GRAM4 and GRAM5, also works for accounts using gsissh and qsub

· Decided to investigate use of commsh· Found willing accomplice to pilot the use of

commsh with GRAM for AMP at NCAR using Kraken at NICS

10

Pilot Requirements· Use of Kraken as the computational resource· Installation and configuration of Science Gateway Kit

on Kraken with GRAM4 (eventually upgraded to GRAM5)

· Report gateway end user attributes (gramAudit)· Use commsh to restrict community user access· Use a managed gateway directory

scripts, executables and static datasets go here· Use GSI authentication· AMP must get productive work done

11

AMP portal – http://amp.ucar.edu/

13

Kraken Compute Resource at NICS

· 1.17 PetaFlop Cray XT5· 9408 nodes with 112,896 compute cores· Two 2.6 GHz AMD processors with 12 cores

per node· 147 TBs memory

14

AMP Architecture

· Users interact with a web server· Community certificate on the daemon server· All communication through database

– SQL recordset role controls– All input files parsed, stored in database using

native types, and regenerated by template before transmission to remote resource

GridAMP Daemon

SQL DB

Web Interface Remote Resource

SQL SQL

GRAM4GRAM5

15

GridAMP Simulation Processing· Every simulation uses a

unique X.509 certificate with a SAML assertion

· Simulation-specific certificates cached and regenerated automatically

Community Certificate

Simulation Certificate

Simulation Web Auth. Record

+ SAMLassertion

16

GridAMP Simulation Processing· Developer must register

commands with commsh· GRAM4/5 on resource:

– Checks command with commsh– Extracts SAML for audit log

Community Certificate

Simulation Certificate

Remote Resource

GRAM4GRAM5

Simulation Web Auth. Record

+ SAMLassertion

ProcessingCommand

+commsh

developer makes sure these match

Fork or Queue

GRAMAudit Log& TGCDB

SAML

17

Pilot Study Experiences· Gateway Developer

– Must understand commsh (and work with RP)– Must understand and implement SAML assertions

(may require gateway user interface modification!)– Must understand use of gateway managed directory

· Gateway User– No end user visible changes

· System Administrator– GRAM4 and GRAM5 have steep learning curve to

configure and test (provided feedback to Globus dev)– Commsh and gramAudit configuration needs better

installation and example documentation– Found a commsh bug [GRAM-206]

18

Recommendations· Continue mandatory TeraGrid security and

accounting requirements· Use of community accounts should be limited

to GRAM and GSISSH implementations· Restrict commands that may be executed on

a host by a community account with commsh or an equivalent

· Require use of a “managed gateway directory” for community accounts scripts, executables and static data

19

Recommendations· Do not allow use of the community account

for gateway development. Use a gateway developer account

· Limit community account user certificates to be only from short-lived certificate authorities

· Audit user DNs periodically to make sure multiple end users do not use the same DNs

· When user or community accounts are compromised revoke all related user certs

· Improve the commsh configuration documentation

20

Recommendations

· Fix the commsh bug [GRAM-206]Note: this has been reported as fixed.

21

Questions or Comments?