Parallel I/O

Parallel I/O

A. PatraMAE 609/CE667 ...

What is Parallel I/O ?

Parallel processes need parallel input/output

Ideal: Processor consuming/producing data reads/writes it directly

Not practical for large numbers of processors ...



R. Lusk : Multiple Processes Participate in the I/O Application level parallelism “File” is stored on multiple disks on a

parallel file system Additional Interfaces for I/O


File is stored on multiple disks on a parallel file system


I/O should be parallel at both endsApplication program end -- with access to

single logical file that is distributed across physical disks

I/O should be physically parallel so that parallel performance scales with no. of processors etc.

Parallel File Systems

Provide users with a consistent name space across the

machine,aids programmers in accessing file data on

multiple nodes

physical distribution of data across disks and network entities, and,eliminates bottlenecks both at the disk

interface and the network, providing more effective bandwidth to the I/O resources

Parallel File Systems

Example Systems PFS -- Intel Paragon XFS -- SGI Origin PIOFS-- IBM SP PVFS -- Linux cluster

Parallel Virtual File System(PVFS)

PVFS system consists of three components:

• the manager daemon, which runs on a single node,

handles permission checking for file creation, open, close, and remove operations

•the I/O daemons, one of which runs on each I/O nodes, and handle all file I/O

•application library, through which applications communicate with the PVFS daemons.

Parallel Virtual File System(PVFS)

File striping File partitioning Application-oriented interfaces Operation with existing binaries

When using PVFS, nodes who perform computation (compute nodes) must communicate with nodes who perform I/O operations (I/O nodes) in order for file system operations to take place. Application tasks use one of the interfaces available in the PVFS libraries to communicate with the I/O daemons, who use UNIX read() and write() calls to perform I/O operations on the local disks

PVFS

small data transfers tend to lead to very poor throughput.

Streams-based approach to data transfer is an attempt to improve overall network throughput by: reducing the number of control messages removing stripe and partition dependence on

message sizes

PVFS

I/O stream between an application and an I/O node resulting from a

strided request. Each side calculates the intersection of physical stripe

and the strided request. The data is always passed in ascending byte order

and is packed into TCP packets by the underlying networking software.

PIOFS -- IBM

An RS/6000 SP with several client nodes accessing data at server nodes.

The Parallel I/O File System supports simultaneous access of server nodes by multiple

client nodes.

PIOFS -- IBM

PIOFS lets you create files as large as 128 Terabytes that span

multiple server nodes. With PIOFS file partitioning, you can parallelize access to your data

without the inconvenience and administrative overhead of maintaining multiple data files.

PIOFS files can be dynamically partitioned into subfiles many

different ways, all without altering or moving the contents of the file. PIOFS supports parallelism in two complementary ways: physically

and logically: A file can be divided physically over multiple disks and servers.

A file can be divided logically into multiple subfiles.

File Partitioning

1 2 3 4 5 6 7 8 9

10 11 12 13 14 15 16 17 18

19 20 21 22 23 24 25 26 27

. . . . . . . . . . . . . .

file with numbers

File split into 8 subfiles by columns

File Partitioning

File split into 8 subfiles by rows

File split in 3 with wrapping

Matrix-Matrix Multiply

Tasks 0 and 1 process the first N/2 rows of matrix A

Tasks 2 and 3 process the last N/2 rows of matrix A

Tasks 0 and 2 process the first N/2 columns of matrix B

Tasks 1 and 3 process the last N/2 columns of matrix B

Parallel I/O is active researchOnly 1 complete MPI-IO implementation availablePicture will stabilize over the next few years (1-2)

Documents

Parallel I/O