Upload
leduong
View
213
Download
0
Embed Size (px)
Citation preview
1
Grid ComputingJean-Marc Pierson
October 2008pierson at irit.fr
2
And comparisons must bemade among many
We need to get to one micron to know location of every cell. We’re just now starting to get to 10 microns
A Brain is a Lot of Data!(Mark Ellisman, UCSD)
3
Outline A very short introduction to Grids A brief introduction to parallelism A not so short introduction to Grids “Gridification” of a sequential program Some grid middlewares and grid projects In-depth study of the Globus middleware
4
Grid concepts : an analogy
Electric power distribution :the electric network and high voltage
2
5
Grids concepts
Computer power distribution :Internet network and high performance (parallelism and distribution)
6
A Definition ofGrid Computing
Ian Foster: “Resource sharing & coordinatedproblem solving in dynamic, multi-institutionalvirtual organizations”. http://www-fp.mcs.anl.gov/~foster/
Current views: see for instance the survey fromHeinz Stockinger in 2006 :http://hst.web.cern.ch/hst/publications/DefiningTheGrid-1.1.pdf
Oldest: Corbato in 1965:http://www.multicians.org/fjcc3.html
See http://en.wikipedia.org/wiki/Grid_computingfor more or alternate definitions
7
Concrete examplesof GridS
Europe: EGEEwww.eu-egee.org/240 institutions, 45 countries72,000 CPU, 20 PB disk, 200,000 concurrent jobsHigh Energy Physics, BioInformatics, Astrophysics
Several projects in US LabsJapan’s Naregi, GridBus in Australia, …France:
8
An Example VirtualOrganization: CERN’sLarge Hadron Collider
1800 Physicists, 150 Institutes, 32 Countries
100 PB of data by 2010; 70,000 CPUs
3
9
A Market or a niche?
Only a scientist thing? Market: 4 Billions $ by 2008 (www.gridtoday.com)
Major vendors are on the hype: IBM, Sun, Intel,Microsoft, Oracle, HP, Hitachi, …
About 30 big companies involved in the Open GridForum: www.gridforum.org
+ myriad of smaller companies. + services/experts companies (CS-SI, IBM, Atos,
Accenture, Cap Gemini, …) IN2P3, CEA, CNES, Metéo France, Airbus, IFP,
Banques, Peugeot, FT, … Simulations (Physics, Environment, Finance),
Genetics, Chemistry, Games, Data centers, … 10
Where does all thiscome from???
11
Parallelism : anintroduction
Grids dates back only 1996 Parallelism is older ! (first classification in 1972)
Motivations :need more computing power (weather forecast,
atomic simulation, genetics…)need more storage capacity (petabytes and more)in a word : improve performance ! 3 ways ...Work harder --> Use faster hardwareWork smarter --> Optimize algorithmsGet help --> Use more computers !
12
Parallelism : the oldclassification
Flynn (1972) Parallel architectures classified by the
number of instructions (single/multiple)performed and the number of data(single/multiple) treated at same time
SISD : Single Instruction, Single Data SIMD : Single Instruction, Multiple Data MISD : Multiple Instructions, Single Data MIMD : Multiple Instructions, Multiple Data
4
13
SIMD architectures
in decline since 97 (disappeared frommarket place)
concept : same instruction performed onseveral CPU (as much as 16384) on differentdata
data are treated in parallel
14
MIMD architectures
different instructions are performed inparallel on different data
divide and conquer : many subtasks inparallel to shorten global execution time
large heterogeneity of systems
15
Another taxonomy
based on how memories and processorsinterconnect
SMP : Symmetric Multiprocessors MPP : Massively Parallel Processors Constellations Clusters Distributed systems
16
SymmetricMulti-Processors (1/3)
Small number of identical processors (2-64)Share-everything architecture
single memory (shared memory architecture)single I/Osingle OSequal access to resources
CPU CPU CPU
HDMemory
network
5
17
Pro :Easy to program :
only one address space to exchange data(but programmer must take care ofsynchronization in memory access : criticalsection)
SymmetricMulti-Processors (2/3)
18
SymmetricMulti-Processors (3/3)
Cons :poor scalability : when the number of
processors increases, the cost to transfer databecomes too high; more CPUs = more accessmemory by the network = more need in memorybandwidth !
Direct transfer from proc. to proc. (->MPP)Different interconnection schema (full
impossible !, growing in O(n2) when nb of procsincreases by O(n)) : bus, crossbar, multistagecrossbar, ...
19
Massively ParallelProcessors (1/2)
Several hundred nodes with a high speedinterconnection network/switch
A share-nothing architectureeach node owns a memory (distributed memory),
one or more processors, each runs an OS copy
CPU
CPU
CPUMemory
network
Memory
Memory20
Pros :good scalability
Cons :communication between nodes longer than in shared
memory; improve interconnection schema : hypercube,(2D or 3D), torus, fat-tree, multistage crossbars
harder to program :data and/or tasks have to be explicitly distributed to nodesremote procedure calls (RPC, JavaRMI)message passing between nodes (PVM, MPI), synchronous or
asynchronous communications
DSM : Distributed Shared Memory : a virtual memoryupgrade : processors and/or communication ?
Massively ParallelProcessors (2/2)
6
21
Constellations
a small number of processors (up to 16) clusteredin SMP nodes (fast connection)
SMPs are connected through a less costlynetwork with “poorer” performance
With DSM, memory may be addressed globally :each CPU has a global memory view, memory andcache coherence is guaranteed (ccNuma)
CPUCPU
Memorynetwork
CPU
CPU CPUCPU
Memorynetwork
CPU
CPU CPUCPU
Memorynetwork
CPU
CPU
Interconnection network Periph.22
Clusters a collection of workstations (PC for
instance) interconnected through highspeed network, acting as a MPP/DSM withnetwork RAM and software RAID(redundant storage, // IO)
clusters = specialized version of NOW :Network Of Workstation
Pros :low coststandard componentstake advantage of unused computing power
23
Distributed systems
interconnection of independent computers each node runs its own OS each node might be any of SMPs, MPPs,
constellations, clusters, individualcomputer …
the heart of the Grid ! «A distributed system is a collection of independent
computers that appear to the users of the system as a singlecomputer » Distributed Operating System. A. Tanenbaum,Prentice Hall, 1994
24
Where are we today(Oct 20, 2008) ?
a source for efficient and up-to-dateinformation : www.top500.org
the 500 best architectures ! we are at 1 petaflops=1000 Teraflops 1 Flops = 1 floating point operation per
second 1 TeraFlop = 1000 GigaFlops = 100 000
MegaFlops = 1 000 000 000 000 flops = onethousand billion operations per second
7
25
Today's bests comparison on a similar matrix maths test (Linpack) :Ax=b
26
NEC earth simulatorSingle stage crossbar : 2700 km of cables
700 TB disk space1.6 PB mass storagearea : 4 tennis court,
3 floors
- a MIMD withDistributed Memory
27
How it grows ? in 1993 (14 years ago!)
n°1 : 59.7 GFlopsn°500 : 0.4 GflopsSum = 1.17 TFlops
in 2007 (yesterday?)n°1 : 280 TFlops (x4666)n°500 : 4 TFlops (x10000)Sum = 4920 Tflops
28
8
29 30
31 32
Problems of the parallelismTwo models of parallelism :
driven by data flow : how to distribute data ?driven by control flow : how to distribute tasks ?
Scheduling :which task to execute, on which data, when ?how to insure highest compute time (overlap
communication/computation?) ? Communication
using shared memory ?using explicit node to node communication ?what about the network ?
Concurrent accessto memory (in shared memory systems)to input/output (parallel Input/Output)
9
33
The performance ? Ideally grows linearly Speed-up :
if TS is the best time to treat a problem in sequential,its time should be TP=TS/P with P processors !
Speedup = TS/TP
limited (Amdhal law): any program has a sequentialand a parallel part : TS=F+T//, thus the speedup islimited : S = (F+T//)/(F+T///P)<1/F
Scale-up :if TPS is the time to treat a problem of size S with P
processors, then TPS should also be the time to treat aproblem of size n*S with n*P processors
34
Network performance analysis scalability : can the network be extended ?
limited wire length, physical problems
fault tolerance : if one node is down ?for instance in an hypercube
multiple access to media ? inter-blocking ? The metrics :
latency : time to connectbandwidth : measured in MB/s
35
Tools/environment for parallelism (1/2)
Communication between nodes :By global memory ! (if possible, plain or virtual)Otherwise :low-level communication : sockets
s = socket(AF_INET, SOCK_STREAM, 0 );
mid-level communication library (PVM, MPI)info = pvm_initsend( PvmDataDefault );info = pvm_pkint( array, 10, 1 );info = pvm_send( tid, 3 );remote service/object call (RPC, RMI, CORBA)service runs on distant node, only its name and
parameters (in, out) have to be known
36
Tools/environment for parallelism (2/2)Programming tools
threads : small processesdata parallel language (for DM archi.):
HPF (High Performance Fortran)say how data (arrays) are placed, the system will infer the
best placement of computation (to minimize totalcomputation time (e.g. further communications)
task parallel language (for SM archi.):OpenMP : compiler directives and library routines; based
on threads. The parallel program is close to sequential; it isa step by step transformParallel loop directives (PARALLEL DO)Task parallel constructs (PARALLEL SECTIONS)PRIVATE and SHARED data declarations
10
37
Bibliography / WebographyG.C Fox, R.D William and P.C Messina"Parallel Computing Works !"Morgan Kaufmann publisher, 1994, ISBN 1-55860-253-4
M. Cosnard and D Trystram"Parallel Algorithms and Architectures"Thomson Learning publisher, 1994, ISBN 1-85032-125-6
M. Gengler, S. Ubéda and F. Desprez,
"Initiation au parallélisme : concepts, architectures et algorithmes"
Masson, 1995, ISBN 2-225-85014-3
Parallelism: www.ens-lyon.fr/~desprez/SCHEDULE/tutorials.html www.buyya.com/cluster
Grids : www.lri.fr/~fci/Hammamet/Cosnard-Hammamet-9-4-02.ppt
TOP 500 : www.top500.org PVM:www.csm.ornl.gov/pvm
OpenMP : www.openmp.org HPF : www.crpc.rice.edu/HPFF38
Computational grid “HW and SW infrastructure that provides
dependable, consistent, pervasive and inexpensiveaccess to high-end computational capabilities”
Performance criteria : security reliability computing power latency services throughput
39
Grid Definition RefinedUse Open ProtocolsIs decentralizedDeliver non-trivial QoS
40
Levels of cooperation End system (computer, disk, sensor…)
multithreading, local I/O
Cluster (heterogeneous) synchronous communications, DSM, parallel I/O parallel processing
Intranet heterogeneity, distributed admin, distributed FS and
databases low supervision, resource discovery high throughput
Internet no control, collaborative systems, (international) WAN brokers, negotiation
11
41
Grid Characteristics Large Scale Heterogeneity Multiple Domains of Administration Autonomy… but coordination Dynamicity Flexibility Extensibility Security
42
Basic servicesAuthentication/Authorization/TraceabilityActivity control (monitoring)Resource informationResource brokeringSchedulingJob submission, data access/migration and
executionAccounting
43
Layered Grid Architecture(By Analogy to Internet Architecture)
Application
Fabric“Controlling things locally”: Accessto, & control of, resources
Connectivity“Talking to things”: communication(Internet protocols) & security
Resource“Sharing single resources”:negotiating access, controlling use
Collective“Coordinating multiple resources”:ubiquitous infrastructure services,app-specific distributed services
InternetTransport
Application
Link
Intern
et Protocol Arch
itecture
From I. Foster 44
Elements of theProblem Resource sharing
Computers, storage, sensors, networks, … Heterogeneity of device, mechanism, policy Sharing conditional: negotiation, payment, …
Coordinated problem solving Integration of distributed resources Compound quality of service requirements
Dynamic, multi-institutional virtual orgs Dynamic overlays on classic org structures Map to underlying control mechanisms
From I. Foster
12
45
Aspects of the Problem Need for interoperability when different groups want
to share resources Diverse components, policies, mechanisms E.g., standard notions of identity, means of communication,
resource descriptions
Need for shared infrastructure services to avoidrepeated development, installation E.g., one port/service/protocol for remote access to
computing, not one per tool/application E.g., Certificate Authorities: expensive to run
A common need for protocols & services
From I. Foster 46
ResourcesDescriptionAdvertisingCatalogingMatchingClaimingReservingCheckpointing
47
Resource layers Application layer
tasks, resource requests
Application resource management layer intertask resource management, execution environment
System layer resource matching, global brokering
Owner layer owner policy : who may use what
End-resource layer end-resource policy (e.g. O.S.)
48
Resource management(1)
Services and protocols depend on theinfrastructure
Some parameters stability of the infrastructure (same set of resources or
not) freshness of the resource availability information reservation facilities multiple resource or single resource brokering
Example request : I need from 10 to 100 CE eachwith at least 128 MB RAM and a computing powerof 50 Mips
13
49
Resource managementand scheduling (1)
Levels of scheduling job scheduling (global level ; perf : throughput) resource scheduling (perf : fairness, utilization) application scheduling (perf : response time, speedup,
produced data…)
Mapping/scheduling resource discovery and selection assignment of tasks to computing resources data distribution task scheduling on the computing resources (communication scheduling)
Individual perfs are not necessarily consistent withthe global (system) perf !
50
Resource managementand scheduling (2)
Grid problemspredictions are not definitive : dynamicity !Heterogeneous platformsCheckpointing and migration
51
GRAM GRAM GRAM
LSF Condor NQE
Application
RSL
Simple ground RSL
InformationService
Localresourcemanagers
RSLspecialization
Broker
Ground RSL
Co-allocator
Queries& Info
A ResourceManagement System example (Globus)
52
Resource information (1) What is to be stored ?
Organization, people, computing resources, softwarepackages, communication resources, event producers,devices…
what about data ???
A key issue in such dynamics environments A first approach : (distributed) directory (LDAP)
easy to use tree structure distribution static mostly read ; not efficient updating hierarchical poor procedural language
14
53
Resource information(2)
But :dynamicitycomplex relationshipsfrequent updatescomplex queries
A second approach : (relational) database
54
Programming the grid:potential programming
modelsMessage passing (PVM, MPI)Distributed Shared MemoryData Parallelism (HPF, HPC++)Task Parallelism (Condor)Client/server - RPCAgentsIntegration system (Corba, DCOM, RMI)
55
Program execution :issues
Parallelize the program with the right jobstructure, communication patterns/procedures,algorithms
Discover the available resources Select the suitable resources Allocate or reserve these resources Migrate the data (or the code) Initiate computations Monitor the executions ; checkpoints ? React to changes Collect results
56
Data management It was long forgotten !!! Though it is a key issue ! Issues :
indexing retrieval replication caching traceability (auditing)
And security !!!
15
57
From computing grids toinformation grids
58
From computing gridsto information grids (1)
Grids for long was lacking most of the tools mandatory to share(index, search, access), analyze, secure, monitor semanticdata (information)
Several reasons : history money difficulty
Why is it so difficult ? Sensitivity but openness Multiple administrative domains, multiple actors,
heterogeneousness but a single global architecture/view/system Dynamicity and unpredictability but robustness Wideness but high performance
59
From computing grids to information grids (2)ex : the Replica Management Problem
Maintain a mapping between logical names for files andcollections and one or more physical locations
Decide where and when a piece of data must be replicated Important for many applications Example: CERN high-level trigger data
Multiple petabytes of data per year Copy of everything at CERN (Tier 0) Subsets at national centers (Tier 1) Smaller regional centers (Tier 2) Individual researchers have copies of pieces of data
Much more complex with sensitive and complex data likemedical data !!!
60
From computing grids toinformation grids (3)
some (still…) open issues
Security, security, security (incl. privacy,monitoring, traceability…)) at a semantic level
Access protocols (incl. replication, caching,migration…)
Indexing tools Brokering of data (incl. accounting) (Content-based) Query optimization and execution Mediation of data Data integration, data warehousing and analysis
tools Knowledge discovery and data mining
16
61
Functional View of GridData Management
Location based ondata attributes
Location of one ormore physical replicas
State of grid resources, performance measurements and predictions
Metadata Service
Application
Replica LocationService
Information Services
Planner:Data location, Replica selection,Selection of compute and storage nodes
Security and Policy
Executor:Initiatesdata transfers andcomputations
Data Movement
Data Access
Compute Resources Storage Resources62
Security:Why Grid Security is Hard
Resources being used may be extremely valuable & theproblems being solved extremely sensitive
Resources are often located in distinct administrative domains Each resource may have own policies & procedures
Users may be different The set of resources used by a single computation may be
large, dynamic, and/or unpredictable Not just client/server
The security service must be broadly available & applicable Standard, well-tested, well-understood protocols Integration with wide variety of tools
63
1) Easy to use
2) Single sign-on
3) Run applicationsftp,ssh,MPI,Condor,Web,…
4) User based trust model
5) Proxies/agents (delegation)
1) Specify local access control
2) Auditing, accounting, etc.
3) Integration w/ local systemKerberos, AFS, license mgr.
4) Protection from compromisedresources
API/SDK with authentication, flexible message protection,
flexible communication, delegation, ...Direct calls to various security functions (e.g. GSS-API)Or security integrated into higher-level SDKs:
E.g. GlobusIO, Condor
User View Resource Owner View
Developer View
Grid Security : various views
64
Grid security :requirements
AuthenticationAuthorization and delegation of authorityAssuranceAccountingAuditing and monitoringTraceabilityIntegrity and confidentiality
17
65
Query optimization andexecution
Old wine in new bottles ? Yes and no : it seems the problem has not changed
but the operational context has so changed thatclassical heuristics and methods are not morepertinent
Key issues : dynamicity, unpredictability,adaptability
Very few works have specifically addressed thisproblem
Use mobile agents ?
66
Service OrientedArchitecture
Open Grid Service ArchitectureWSRF : Web Service Resource FrameworkEverything is resources, WS-ResourcesAccess through services
67
© Globus Tutorial
68
BibliographyThe Grid 2: Blueprint for a New Computing
Architecture. Ian Foster, Carl Kesselman
Grid Computing: The Savvy Manager’sGuide. Pawel Plaszczak, Richard Wellner Jr