Components for Scientific Computing

Components for Scientific Computing

Randall Bramley

Computer Science, Indiana University

Center for Advanced Computation Research, California Institute of Technology

CAT Team Members

• Dennis Gannon: CAT architecture design• Randall Bramley: CAT/LSA design• Benjamin Temko: Globus wrangler• Fabian Breg: NexusRMI author• Prafulla Deuskar: parallel HPC++ components • Shridhar Diwan: HPC++ Manager • Madhusudhan Govindaraju: HPC++ framework, GRAM

server • David Stern: HPC++ framework design, LSA components • Juan Villacis: CAT design, GUI workspace, Java Manager • Andrew Whitaker: information subsystem, GUI InfoBrowser• Esra Akman, Deepa Viswanathan, Thomas Stuckey: alums

Large Sparse Linear Systems

• 105-107 or more unknowns• Irregular sparsity structure • Nonbanded, non-structured

– Not diagonally dominant

• Nonsymmetric in values– Causes convergence problems

The Good News for Linear Systems

• Sparse direct solvers readily available:– Umfpack, SuperLU, Y12M

– HPC banded solvers in Scalapack

• Preconditioned iterative solvers:– Aztec, SPLIB, BPKIT, ITPACK, ...

• Matrix manipulation packages: – Sparskit, SMMS, Metis, Chaco, ...

The Bad News for Linear Systems

• All methods are parameterized– enormous design space to navigate.

• Black arts abound– matrix reordering, scalings, filtering

• No existing theory provides a practical guide to crafting a solution strategy– experimentation required.

Goals of Linear System Analyzer (LSA)

• Develop “solution strategies” for linear systems• High-level rapid prototyping for combining sparse

matrix manipulations and solvers• Hot-wiring capabilities: grab linear systems from

running applications, return solutions/strategies dynamically

• High-performance capabilities: big systems, fast network protocols.

• We chose a distributed component architecture approach...

Classes of Components in the LSA

• I/O components: getting linear system data in and out of the framework

• Filter components: manipulating the system to have more favorable properties for a solver

• Solve components• Information components

LSA I/O Components

• Read from file: H/B or Matrix Market format• Write out solution vector to a file• Take input from URL on network• Take input from active object (running program)• Feed solution back to an active object• Extract linear system from framework to file

LSA Filter Components

• Reorder linear system: MinDeg, RCM, ND, MaxTraversal, BBD, Weighted Diag Block

• Scale system: max element, row, column, relative scalings

• Filter out elements based on magnitude• Parallel partitioning reorderings: Chaco, Metis

LSA Solver Components

• LAPACK dense solver (extracted)• LAPACK banded solver (extracted)• SuperLU sparse direct from Xiaoye Li• SPLIB preconditioned iterative solvers

– Thirteen iterative methods (only two are really useful)

– Seven preconditioners (four are of use)

– Heavily instrumented

• Aztec parallel iterative solvers• BLOCK

LSA Info Components

• BasicInfo: analysis of coefficient matrix (beefed-up version of Sparskit’s infofun)

• VizInfo: emily sparse matrix visualization• Wannabes:

– IterativeInfo: specialized analysis for selecting iterative solvers, preconditioners

– DirectInfo: estimates storage required, computational rate for sparse direct solvers

– Spectral: eigenvalue/singular value decomposition

Component Systems Definition

• Characteristics of “component”– Encapsulated software object providing specific functionality or

service, used with other components to build complete applications

– Peer-to-peer interactions vs. client-server.

– Applications built by connecting binaries - not via compilation/linking of sources

– Components interact via well-defined interfaces or ports.

– Connected as software IC’s in a framework

– Services need not be computational: visualization, object store, ...

• Components have swept desktop, business worlds– Microsoft COM, CORBA, Java Beans, Iris Explorer, Khoros, ...

Distributed Component Framework Services

• Component identification and handling– LDAP component information system

• Interface registry– Java RMI mechanism

• Connection management; exception handling– Combination of Java, HPC++, Nexus

• User interface / collaboration mechanisms– Java GUIs

• Hardware resource management– Globus system

DatabaseCATCATCompositionCompositionGUIGUI

Java Composition GUI

Visualizer

Local Java proxies to remote objectsRemote objects inHPC++, Java orwrapped MPI-Fortran

Intercomponent comm with Java RMI/HPC++remote invocation semantics implemented over Globus/Nexus

EvolverInitializer

Globus authentication and

instantiation of components

CAT Framework Architecture

Standard Metacomputing Model

• Application codes at top,

written in C, C++, Fortran, …

• HPC libraries (ScalaPack),

runtimes (MPI, Tulip)

• Infrastructure (Globus, Nexus,

network protocols)

• Hardware (Origins, SP2s,

CAVES, …)

Hardware: computers,visualization equipt

Basic infrastructuresoftware

Software libraries, runtime systems

Application codes

CAT Metacomputing Model

• Applications … not code (physical process modeling)

• Composition tools for organizing resources (CAT Java GUIs, managers)

• Resource (both SW and HW) location/management (Globus, LDAP DB’s)

• Actual resources: code, computers, visualization engines, databases, instruments

HardwareResources

SoftwareResources

Component Resource Management

Component Resource Composition Tools

Applications

CAT User Interfaces

Software componentfinder: LDAP based

Detail info aboutselected comp

Compositionwindow showingcomp connects

Common Component Architecture Group

• Existing component frameworks not suitable for HPC• Participants from all national labs, few universities• Key goal: develop component interface standards.

– Allow interchanging of components between frameworks, labs

– Mechanisms for a component to make its interfaces known

– Allow for parallel links, different network protocols.

• Meetings:– Unfunded meetings six times last year

– BOF meeting at SC98 (debutante party)

– Last meeting has implementable goal

Usage of Component Architectures

Init

DB Solve

DiscretizeComposition phase: instantiate and tie together comps

Init

DB Solve

DiscretizeRun phase: start comps executing

Mixed phase: addcomps dynamically

Visualize

A Possible CCA Framework System

entific IDL

Any CCA Compliant Framework

FrameworkServices

Component 1 Component 2

proxygenerator

Builder

Repository/Registry

GPorts Part of Gports specific to the framework

Abstract ConfigurationAPI

Repository API

Scientific IDL

(From RobArmstrong,

SNL)

Basic CCA mechanism: gPorts

• gPorts = generalized Ports– Borrows ideas from CORBA, COM, visual programming environments

like Iris Explorer, AVS, Khoros

– Current prototype based on CORBA-like user/provider model

– Uses type of COM-like IUnknown pattern

– Defined using linked interfaces.

• Source-target, start pt - end pt, source-sink, output-input, event source - listener, user-provider terms all used to describe the communicating ports


• Output ports can be multiplexed, but not input ports• gPorts directly* invoke methods on each other.• Allows

– standard data-flow model

– shared-memory or “data stays, components move” models

* Using underlying run-time system


outputGport inputGport

Source Target

Event Source“uses” remote methods

Event listener“provides” methods

Typed Channel

outputGport implements- methods defined by typed channel- addListener() used to connect target to outputGport

Examples:- outputGport is list of “global pointers” in HPC++- logic for argument marshaling and RPC- n x m collective communications as in MPI

Each Gport provides information about its types via strings for name, type:

Framework responsible for mapping from strings to internal representation of types


class gPortInfo { public gPortInfo(String portname, String porttype); public String getType(); public String getName();};


Channel types can be defined via an IDL spec:interface MyChannelType{ int row(in float, in array<int,3>); void mat_order(in int); };

Becomes in Java or in C++

Interface MyChannelType{ int row( float, array_int_3); void mat_order(int); };

class MyChannelType{ public: virtual int row(float, array_int_3)=0; virtual void mat_order(int)= 0; };

Defining input port requires implementing Channel type generated by IDL (or by hand)interface InputGPort extends GPort{};

Defining output port requires implementingChannel type

Mechanisms also will allow hierarchical compositionSee web pages for details, examples.


interface OutputGPort extends GPort{ public void addInputListener(InputGPort ); public void removeInputListener(InputGPort ); public Enumeration getInputPorts(); };

CCA and IDL

• CCA standard does not require using IDL. • CAT/LSA allows seamless interaction between Java and

HPC++ method invocations; no IDL• Groups can define own “glue” mechanisms• Must support C/C++, Java, Fortran77, Python, Perl

– Fortran 90 support allowed but not required.

• Reflection capabilities required for interfaces.– Run-time discovery of objects by frameworks

CCA Dynamic Registry

• First Level: how to register components so that frameworks can locate and instantiate them

• Second Level: how a component makes its interfaces known to the framework

• Meta-Level: how to describe the internal functionality of a component in the global marketplace

• Less of a problem in DOE applications: relatively few components and interfaces to handle.

• Will be/is a major problem for academic applications

Current CCA Status

• Scrupulous avoidance of word “standards”. • draft gPort spec

http://z.ca.sandia.gov/~cca-forum/gport-spec

• draft IDL spechttp://www.llnl.gov/CASC/babel

• Multiple test frameworks being developed– InDEPS (Sandia, Armstrong and Melius)

– CAT/LSA (IU, Gannon and Bramley)

– PAWS (LANL, Beckman and Reynders)

• Existing frameworks likely to be made CCA-compliant– POOMA, PetSC, SciRun, ISIS++, ...

Current CCA Participants

• Sandia: Rob Armstrong, Robert Clay, Bill Mason,Carl Melius • Indiana University: Dennis Gannon • CACR, Indiana: Randall Bramley • LLNL: Andy Cleary, Scott Kohn, Brent Smolinski• LANL: Pete Beckman, Pat Fasel, Bill Humphrey, Kate

Keahey• ORNL: Al Geist, Noel Nachtigal • ANL: Satish Balay, Lori Freitag, Paul Hovland, Lois Curfman

McInnes, Barry Smith • Utah: Steve Parker, Chris Johnson• Lawrence Berkeley: Brent Milne• Pacific Northwest Labs: Jarek Nieplocha

An Idiosyncratic View

• Components are the future of scientific/eng computing– desktop, business computing always leads the way

– is the right level of abstraction for object-orientation

– maps naturally to collaborating groups distributed across the globe

– integrates weird stuff: symbolic computing, vizualization, DBs

• We have to address scalability in terms of numbers of components: 105 components, 108 interface instances

• Software agents may be right tool to find, challenge/test, and connect software components

Documents

Components for Scientific Computing