Upload
hailey-suarez
View
221
Download
0
Tags:
Embed Size (px)
Citation preview
1
The LDCM Grid Prototype
Jeff Lubelczyk& Beth Weinstein January 4, 2005
2Sponsored by NASA LDCM, NASA/GSFC Code 580 Team: 586/585/SGT/QSS/Aerospace Corp/USGS EDC
Prototype Introduction
A Grid infrastructure allows scientists at resource-poor sites access to remote resource-rich sites• Enables greater scientific research• Maximizes existing resources• Limits the expense of building new facilities
The objective of the LDCM Grid Prototype (LGP) is to assess the applicability and effectiveness of a data grid to serve as the infrastructure for research scientists to generate virtual Landsat-like data products
3Sponsored by NASA LDCM, NASA/GSFC Code 580 Team: 586/585/SGT/QSS/Aerospace Corp/USGS EDC
LGP Key POCs
Sponsors• LDCM - Bill Ochs, Matt Schwaller• Code 500/580 - Peter Hughes,
Julie LoftisLGP Team members• Jeff Lubelczyk (Lead)• Gail McConaughy (SDS Lead
Technologist)• Beth Weinstein (Software Lead)• Ben Kobler (Hardware, Networks)• Eunice Eng (Software Dev, Data)• Valerie Ward (Software Dev,
Apps)• Ananth Rao ([SGT] Software
Arch/Dev, Grid Expertise)• Brooks Davis ([Aerospace Corp]
Globus/Grid Admin Expert)• Glenn Zenker ([QSS] System
Admin)
USGS• Stu Doescher (Mgmt)• Chris Doescher (POC)• Mike Neiers (Systems
Support)Science Input• Jeff Masek, 923 (Blender)• Robert Wolfe, 922
(Blender, Data)• Ed Masuoka, 922 (MODIS,
Grid)LDCM Prototype Liaison• Harper Prior (SAIC)
CEOS grid working group (CA)• Ken McDonald• Yonsook Enloe [SGT]
4Sponsored by NASA LDCM, NASA/GSFC Code 580 Team: 586/585/SGT/QSS/Aerospace Corp/USGS EDC
Grid - A Layer of Abstraction
Application
User Client
Grid Middleware•Security (Authentication, Authorization)
•Resource Discovery•Storage Management
•Scheduling and Job Management
• Grid Middleware packages the underlying infrastructure into defined APIs•A common package is the Globus Toolkit
–Open source, low cost, flexible solution
StorageCompute
West Coast/Platform A
StorageCompute
On Campus/Platform A
StorageCompute
East Coast/Platform C
5Sponsored by NASA LDCM, NASA/GSFC Code 580 Team: 586/585/SGT/QSS/Aerospace Corp/USGS EDC
What the current data grid provides
Security Infrastructure• Globus Gate Keeper
• Authentication (PKI)• Authorization
Resource Discovery• Monitoring and Discovery Service (MDS)
[LDAP like] Storage Management and Brokering
• Metadata catalogs• Replica Location Service
• Allows use of logical file names– Physical locations are hidden
• Storage Resource Management• GridFTP
– Retrieves data using physical file names• Data formats and subsetting
Job Scheduling and Resource Allocation• GRAM (Globus Resource Allocation Manager)
-- Provides a single common API for requesting and using remote system resources
Globus Tookit 2.4.2
GlobusGate
keeperGRAMGridFTP
Note: Portions of the Globus Toolkit used in Capability 1
6Sponsored by NASA LDCM, NASA/GSFC Code 580 Team: 586/585/SGT/QSS/Aerospace Corp/USGS EDC
High Level Schedule
Major Milestones• 12/03 - Prototype start• 6/04 - Demo of Capability 1 grid infrastructure
•Demonstrate simple file transfers and remote application execution at multiple GSFC labs and USGS EDC
•Ready to build application on top of basic infrastructure• 12/04 - Demo of Capability 1
•Provide and demonstrate a grid infrastructure that enables a user program to access and process remote heterogeneous instrument data at multiple GSFC labs and USGS EDC
• 3/05 - Demo of Capability 2 grid infrastructure•Demonstrate file transfers and remote application
execution at multiple GSFC labs, USGS EDC, and ARC/GSFC commodity resources to assess scaleability
• 6/05 - Demo of Capability 2•Enable the data fusion (blender) algorithm to obtain
datasets, execute, and store the results on any resource within the Virtual Organization (GSFC labs, USGS EDC, ARC/GSFC)
7Sponsored by NASA LDCM, NASA/GSFC Code 580 Team: 586/585/SGT/QSS/Aerospace Corp/USGS EDC
The LDCM Demonstration …
Prepares two heterogeneous data sets at different remote locations for like “footprint” comparison from a science user’s home site• The MODIS Reprojection Tool (MRT) serves as
our “typical science application” developed at the science users site (Building 32 in demo)•mrtmosaic and resample (subset and reproject)•Operates on MODIS and LEDAPS (Landsat) surface
reflectance scenes• Data distributed at remote facilities
•Building 23 (MODIS scenes)•USGS/EDC (LEDAPS scenes)
Solves a realistic scientific scenario using grid-enabled resources
8Sponsored by NASA LDCM, NASA/GSFC Code 580 Team: 586/585/SGT/QSS/Aerospace Corp/USGS EDC
Capability 1 Virtual Organization
GSFC SEN1Gbps
Backbone
edclxs66USGS EDC
Sioux Falls, SDLGP23GSFC
B23/W316
LGP32Science User_1
GSFCB32/C101
Capability 1
Installed Equipment
Dell/Linux Server•Dual Xeon Processors•8 GB Memory•438GB Disk Storage
Dell/Linux Server•Quad Xeon Processors•16 GB Memory•438GB Disk Storage
Dell/Linux Server•Dual Xeon Processors•8 GB Memory•438GB Disk Storage
MAX (College Park)OC48, 2.4Gbps
Backbone
USGS/EDC1 Gbps
Backbone
vBNS+ (Chicago)OC48, 2.4Gbps
Backbone
1 Gbps
1 Gbps
SEN: Science and Engineering NetworkMAX: Mid-Atlantic CrossroadsDREN: Defense Research and Engineering NetworkvBNS+: Very high Performance Network Service
OC12, 622 Mbps OC12, 622 MbpsShared with DREN
1 GbpsUSGS/EDC
GSFC
9Sponsored by NASA LDCM, NASA/GSFC Code 580 Team: 586/585/SGT/QSS/Aerospace Corp/USGS EDC
A “Typical Science Application”
MODIS Reprojection Tool (MRT)• Software suite distributed by LP DAAC • Applications used include
•mrtmosaic.exe– Create 1 scene from adjacent scenes
•resample.exe (Subset)– Geographic– Band/Channel– Projection
• Each operate on MODIS and LEDAPS scene data
Visualization Tool -- Software to display scenes • HDFLook
10Sponsored by NASA LDCM, NASA/GSFC Code 580 Team: 586/585/SGT/QSS/Aerospace Corp/USGS EDC
Data
MODIS - MOD09GHK• MODIS/Terra Surface Reflectance Daily L2G Global 500m SIN
Grid V004 • Sinusoidal projection• 7 Scenes
•Washington D.C. (H = 11,12, V = 5)•Pacific NW (H = 9, V = 4)
• Obtained from LP DAAC ECS Data PoolLEDAPS - L7ESR• LEDAPS Landsat-7 Corrected Surface Reflectance• UTM projection • 2 Scenes
•Washington D.C. (Path = 15, Row = 33)•Pacific NW areas (Path = 48, Row = 26)
• Obtained from LEDAPS websiteBoth compatible with the MRTAll like-area scenes are as temporally coincident as possible
11Sponsored by NASA LDCM, NASA/GSFC Code 580 Team: 586/585/SGT/QSS/Aerospace Corp/USGS EDC
4 Scenarios to Illustrate Grid Flexibility
Data Services (Move application to data)• Transfer the MRT to the remote hosts and process the
data remotely, sending the results back to the science facility
Batch Execution (Parallel computing)• Demonstrate the execution of the MRT in a parallel
batch environmentLocal Processing (User prefers to process locally)• Transfer the selected data sets to the science user site
for processing Third Party Processing (No local resource usage)• Perform a third party data transfer and process the
data remotely
Grid flexibility maximizes science resources
12Sponsored by NASA LDCM, NASA/GSFC Code 580 Team: 586/585/SGT/QSS/Aerospace Corp/USGS EDC
How we make this happen
Command line interface to execute the LDCM Grid Prototype (LGP) driver programThe LGP Driver• Manages the execution of a specified
application•Transfers the application and data as needed
• Uses configuration files as inputs to describe:•The executable and its location•The data sets and their location•The location of the resulting output file(s)
13Sponsored by NASA LDCM, NASA/GSFC Code 580 Team: 586/585/SGT/QSS/Aerospace Corp/USGS EDC
LGP Driver(Java 1.4.2)
Data
Capability 1 Software Framework
LDCM Grid Prototype (LGP) Driver• Provides a generic software system
architecture based on Globus services
• LGP Driver high-level services•Session Manager – grid session
initiation and user authentication using proxy certificates
•Data Manager – file transfer using GridFTP
• Job Manager – job submission and status in a grid environment
Utilizes the Java Commodity Grid Kits (CoGs)
•Supplies a layer of abstraction from underlying Globus services
•Simplifies the programming interface
Java CoG 1.1
Session Job
Globus Tookit 2.4.2
GlobusGate
keeperGRAMGridFTP
14Sponsored by NASA LDCM, NASA/GSFC Code 580 Team: 586/585/SGT/QSS/Aerospace Corp/USGS EDC
Demo Scenario 1: Input Data
LEDAPS - L7ESR MODIS - MOD09GHK
15Sponsored by NASA LDCM, NASA/GSFC Code 580 Team: 586/585/SGT/QSS/Aerospace Corp/USGS EDC
data.txt
LDCM VO
Demo Scenario 1: Data Services
USGSEDC
LEDAPS
GSFCB23
MOD09GHK
4) Move mrtmosaic.exe from B32 to B23
6) Move resample.exe from B32 to B23
5) Run mrtmosaic.exe with 2 MOD09GHK files
7) Run resample.exe on mrtmosaic output
1) Move resample.exe from B32 to EDC
2) Run resample.exe on 1 LEDAPS file
3) Move LEDAPS resample output from EDC to B32
GSFCB32MRTGrid Node
GT 2.4.3GridFTP Server
8) Move mrtmosaic resample output from B23 to B32
9) Display LEDAPS and MODIS resampled output using HDFLook
16Sponsored by NASA LDCM, NASA/GSFC Code 580 Team: 586/585/SGT/QSS/Aerospace Corp/USGS EDC
MRT
MO
DIS
Mos
aic
resampleresample3) Move
LEDAPS resample output from EDC to B32
2) Run resample on 1 LEDAPS file
mrtmosaic
1) Move resample from B32 to EDC
8) Move mrtmosaic resample output from B23 to B32
data.txt
LDCM VO
Demo Scenario 1: Data Services
GSFCB23
MOD09GHK
4) Move mrtmosaic from B32 to B23
5) Run mrtmosaic with 2 MOD09GHK files
MOD09GHK M
OD
ISM
osai
c
MOD09GHK
mrtmosaic
USGSEDC
LEDAPS
MODISMosaic
LEDAPS
LE
DA
PS
Su
bse
t
MO
DIS
Mos
aic
Su
bse
t
MRT
MO
DIS
Mos
aic
resample
MRT
MO
DIS
Mos
aic
resample
GSFCB32MRT
LEDAPS – L7ESR MODIS – MOD09GHK
7) Run resample on mrtmosaic output
6) Move resample from B32 to B23
9) Display LEDAPS and MODIS resampled output using HDFLook
17Sponsored by NASA LDCM, NASA/GSFC Code 580 Team: 586/585/SGT/QSS/Aerospace Corp/USGS EDC
Capability 1 Task RequirementsCompleted
Science user is at B32 and the data is at EDC and B23
2 - 3 instrument types10 - 20 scenesSpatially and temporally coincident dataAlgorithm must run on B23, B32, and EDCCommand-line invocation from client sidePerform distributed computationShare distributed data
Verified by executing the 4 scenarios
18Sponsored by NASA LDCM, NASA/GSFC Code 580 Team: 586/585/SGT/QSS/Aerospace Corp/USGS EDC
Next Steps -- Capability 2
Capability 2 (C2)• Integrate with the Blender team
•Collaborate to identify meaningful C2 data sets•Demonstrate blender algorithm
• Assess Grid performance•Expand the VO to include ARC supercomputing if
available•Performance Goals
– Demonstrate the processing of 1 day’s worth of data in the grid environment (~250 scenes)
• Grid Workflow -- increase automation
19Sponsored by NASA LDCM, NASA/GSFC Code 580 Team: 586/585/SGT/QSS/Aerospace Corp/USGS EDC
Grid Workflow
Our current capabilities allow us to submit jobs only to a specified resourceThe goal of the next phase will be to provide the ability to submit a job to the “Grid” Virtual Organization• Grid resource management• Scheduling policy
•Maximize grid resources•Manage sub tasks
• Reliable job completion• Checkpointing and job migration• Leverage wasted cpu cycles
Next step: Examine Condor and Pegasus open source Globus toolkit workflow extensions
20Sponsored by NASA LDCM, NASA/GSFC Code 580 Team: 586/585/SGT/QSS/Aerospace Corp/USGS EDC
Univ. of MD
Grid Workflow Engine
Reflectance PdtsUSGS/EDC
VO Grid Operator’s Interface
Concept of a Future Grid Architecture - LDCM example
DAAC
Landsat Data MODISVIRS& other
ResearchArchive
Data Product Distribution
Existing C1 Grid Infrastructure
Proposed C2 Grid Infrastructure
Future Grid Components
ResearchCommunity
VO Grid Config
VO Grid Resource
Status
Failure Recovery
Science Product Interface
Data Manager
Data Node/Manager
NASA/GSFC
Job Manager
GridResourceManager
Data Node/Manager
SessionManager
ProductStatus &Recovery
Overall V0 Grid Management
Scientist Data Site
Research Community Data Server
ProductDef.
<Blender>
21Sponsored by NASA LDCM, NASA/GSFC Code 580 Team: 586/585/SGT/QSS/Aerospace Corp/USGS EDC
Acronym List
FTP File Transfer ProtocolLDCMLandsat Data Continuity MissionLEDAPS Landsat Ecosystem Disturbance Analysis Adaptive Processing System LGP LDCM Grid PrototypeLP DAAC Land Processes Distributed Active
Archive CenterMODIS Moderate Resolution Imaging
Spectroradiometer
MRT MODIS Reprojection Tool
22Sponsored by NASA LDCM, NASA/GSFC Code 580 Team: 586/585/SGT/QSS/Aerospace Corp/USGS EDC
Condor, Condor-G, DAGman
Condor addresses many workflow challenges for Grid applications.• Managing sets of subtasks• Getting the tasks done reliably and efficiently• Managing computational resources
Similar to a distributed batch processing system, but with some interesting twists.• Scheduling policy• ClassAds• DAGman• Checkpointing and Migration• Grid-aware & Grid-enabled• Flocking (linking pools of resources) & Glide-ins
See http://www.cs.wisc.edu/condor/ for more details
Chart author: lee limingargonne national laboratory
23Sponsored by NASA LDCM, NASA/GSFC Code 580 Team: 586/585/SGT/QSS/Aerospace Corp/USGS EDC
Pegasus Workflow Transformation
Converts Abstract Workflow (AW) into Concrete Workflow (CW).
• Uses Metadata to convert user request to logical data sources
• Obtains AW from Chimera• Uses replication data to locate
physical files• Delivers CW to DAGman• Executes using Condor• Publishes new replication and
derivation data in RLS and Chimera (optional)
See http://pegasus.isi.edu/ for details
ChimeraVirtual Data
Catalog
ReplicaLocationService
MetadataCatalog
StorageSystemStorageSystemStorageSystem
ComputeServerCompute
ServerComputeServerCompute
Server
DAGman
Condor
t
Chart author: lee limingargonne national laboratory