Amesos Interfaces to sparse direct solvers

October 15, 2003

8:30 – 9:30 a.m.

Ken Stanley

Overview

•Background• Simplifying AX = B •Getting started: Amesos_Umfpack•Amesos options•Other Amesos interfaces• Status • Future Work and Discussion• Summary

Background

• Solvers• Sandia Users•Challenges

Background - Solvers

• PSPACES [Gupta]; f90; mpi; SPD; unsupported• WSSMP [Gupta]; binary; mpi; general; • WGSMP [Gupta]; binary; mpi; SPD• SPOOLES [Ashcraft];public domain;c; unsupported• SuperLU [Li,Demmel];serial;c;general• UMFPACK [Davis];serial;c;general;robust• Unnamed [Ng];parallel;f77;SPD;• MUMPS [Amestoy];f90;general/SPD;• Ksparse [Kundert];• SNL_MPFRONT [Benner];parallel• OTHERS: y12M, Boeing, MA57, MA43, DPSLDLT, DPSLDU,

PARDISO, Shen, Dearholt, SPLU, Lusol, LUSOL, XYT

Background – Sandia Users

• Tuminaro – ML, etc.• Shadid – MP Salsa•Williams – Sierra•Hoekstra – Xyce• Pierson – •Dohrman – • Schultz - GOMA•Many – Aztec•Many more

Background - Challenges

•No single solver dominates•Different interfaces– Unique data distribution requirements

•Academic quality– Sporadic development, bugs, untested features,

changing interfaces

– Sandia modifications not consistently adopted

• Imperfect heuristics • Interface changes between revisions

Simplifying AX = B

• Example•Clean, consistent interface• Single point of contact • Evaluation

Epetra_LinearProblem Problem( A, x, b);

Amesos_Umfpack A_direct( Problem, ParamList ) ;

A_direct.SymbolicFactorization( ) ;

A_direct.NumericFactorization( ) ;

A_direct.Solve( ) ;

Simplifying AX = BExample

Simplifying AX = B Single point of contact

• Isolate; report bugs; follow-up• Prompt bug fixes without maintaining third party

code• Encourage developers to adopt Trilinos practices– Autotools; CVS; regression testing; bug tracking; etc.

•Regression testing•Check for memory leaks

Simplifying AX = B Evaluation

• Solvers• Platforms•Matrices• Features•Our goal is to find the bugs before you do

• Amesos User Guide: http://software.sandia.gov/trilinos/packages/amesos

• Build UMFPACK – Download from UMFPACK website– Modify make.inc (see: Trilinos3PL/config/UMFPACKv4.1)– “make” ; Check output of example run

• Future Work – autotool UMFPACK; – Trilinos/packages/UMFPACK ?– -lumfpack?

Getting started: Building Umfpack

• Amesos User Guide:http://software.sandia.gov/trilinos/packages/amesos

• Build Amesos_Umfpack– “./configure –enable-amesos –enable-amesos-umfpack –with-

trilinos3PLdir=/home/userid/Trilinos3PL”– “make” – “cd amesos/test; source AmesosUmfpackShort.exe”

Getting started: Building Amesos_Umfpack

Epetra_CrsMatrix A; Epetra_Vector x, b; AMESOS::Parameter::List ParamList ; Epetra_LinearProblem Problem( A, x, b);

Amesos_Umfpack A_direct( Problem, ParamList ) ; A_direct.SymbolicFactorization( ) ;

A_direct.NumericFactorization( ) ; A_direct.Solve( ) ;

Future Work: Factory Method, Trilinos::Parameter::List

Getting started: Calling Amesos_Umfpack

• Same structure different data• Multiple solves• Blocked right hand sides• Parameter control• Different solver• Matrix format

Amesos options

Epetra_LinearProblem Problem( A, x, b);

Amesos_Umfpack A_direct( Problem, ParamList ) ; A_direct.SymbolicFactorization( ) ;

for ( ) {

// Modifications to the non-zero values of A

A_direct.NumericFactorization( ) ;

A_direct.Solve( ) ;

}

Amesos options: Same Structure Different Data

Epetra_LinearProblem Problem( A);

Amesos_Umfpack A_direct( Problem, ParamList ) ; A_direct.SymbolicFactorization( ) ;

A_direct.NumericFactorization( ) ;

for ( ) {

Problem.SetLHS(x);

Problem.SetRHS(b);

A_direct.Solve( ) ;

}

Amesos options: Multiple Solves

Epetra_CrsMatrix A;

Epetra_MultiVector x, b;

AMESOS::Parameter::List ParamList ;

Epetra_LinearProblem Problem( A, x, b);

Amesos_Umfpack A_direct( Problem, ParamList ) ; A_direct.SymbolicFactorization( ) ;

A_direct.NumericFactorization( ) ;

A_direct.Solve( ) ;

Amesos Options: Blocked right hand sides

Epetra_LinearProblem Problem( A, x, b);

Amesos_Umfpack A_direct( Problem, ParamList ) ; A_direct.SymbolicFactorization( ) ;

ParamList.SetParam( “UmfpackOrder”, 2 );

for ( ) {

// Modifications to the non-zero values of A

A_direct.NumericFactorization( ) ;

A_direct.Solve( ) ;

}

Amesos Options: Parameter control

• Interface accepts any Epetra_RowMatrix• Current implementation accepts any

Epetra_CrsMatrix

Amesos Options: Matrix format

•Klu – Serial, unblocked, for circuit matrices • SuperLUdist – Distributed; limited Amesos release• MUMPS – Distributed; F90; limited Amesos release• DSCPACK – Distributed; Symmetric

•Use distributed code for problems– that are too large to solve on one processor -or-

– take more than one minute to factor

•Others – WSMP, Benner, Ksparse, Ng

Other Amesos Interfaces

•Nightly testing on four platforms: – Atlantis/IRIX,

– Stratus/OSF,

– Herouxsmp/Linux,

– Paunchy/SunOS

• Full support for three packages: Amesos_Umfpack, Amesos_Klu and Amesos_Dscpack•Other packages: Epetra_SLU, SuperludistOO,

Amesos_Mumps, Amesos_Superludist•One example code: example/AmesosDscpack

Status

•Capabilities• Ease of use• Single point of contact

Future Work

•Amesos_Superludist• Amesos_Mumps, Amesos_SuperLU, Amesos_Ksparse,

Amesos_Benner, Amesos_Ng, Amesos_Wsmp • Expose capabilities to users • Support transpose for all solvers• Support row matrix interface • Support Epetra_VbrMatrix in DSCPACK• Extra precision iterative refinement

Future Work - Capabilities

• Examples (TSF extended; TSF core; etc.) • Pre-built libraries? (-lamesos, -lumfpack, -lsuperludist)•Autotools use by third party developers• Factory Method

Future Work – Ease of use

• Encourage developers to adopt Trilinos practices– Autotools; CVS; regression testing; bug tracking; nightly

testing; etc.

– Trilinos package?

• Isolate, report bugs, follow-up•Migration to latest revisions•Add testing based on the Amesos_BaseSolver class– More features

– More matrices

– Parameter control

– Test error conditions

Future Work – Single point of contact

Discussion

• Your needs?– Platforms

– Packages

– Features

– Interface

Amesos Value Added

• Simple, consistent interface • Short learning curve• Easy to switch solvers• Interface to third party developers• Evaluation•Bug workarounds

Summary

•Amesos is driven by your needs– Tell me what you need

– Give me your matrices

• Simple, consistent interface • Start with Amesos_Umfpack • Switch to parallel codes for large matrices– Matrices that don’t fit in memory

– Factorizations that take over a minute

– Experiment with different solvers