View
221
Download
1
Tags:
Embed Size (px)
Citation preview
Research ComputingUNC - Chapel Hill Instructor: Mark Reed Email: [email protected]
Group and Communicator
Management Routines
its.unc.edu 2
GroupsGroups
A group is an ordered set of processes.
Each process in a group is associated with a unique integer rank.
Rank values start at 0 and go to N-1, where N is the number of processes in the group.
In MPI, a group is represented within system memory as an object
• accessible to programmer only by a "handle".
A group is always associated with a communicator object.
its.unc.edu 3
CommunicatorsCommunicators
A communicator encompasses a group of processes that may communicate with each other.
All messages must specify a communicator.
Like groups, communicators are represented within system memory as objects, accessible to the programmer only by "handles".
E.G., handle of the communicator comprising all tasks is MPI_COMM_WORLD
its.unc.edu 4
Groups - Communicators
Groups - Communicators
Communicators specify a communication domain, i.e. communicators provide a self-contained, communication “world” in which to exchange messages
• typically they bind process groups and contexts together to form a safe communication space within the group
intracommunicators are used for communication within a group
intercommunicators are used for communication between disjoint groups
its.unc.edu 5
Groups - Communicators
Groups - Communicators
From the programmer's perspective, a group and a communicator often appear the same.
The group routines are primarily used to specify which processes should be used to construct a communicator
its.unc.edu 6
Group and Communicator
Objects
Group and Communicator
Objects Primary purposes
•Allow you to organize tasks, based upon function, into task groups.
•Enable Collective Communications operations across a subset of related tasks.
•Provide basis for implementing user defined virtual topologies
•Provide for safe communications
its.unc.edu 7
CommunicatorsCommunicators
Groups/communicators are dynamic - they can be created and destroyed during program execution.
Processes may be in more than one group/communicator.
•They will have a unique rank within each group/communicator.
MPI provides over 40 routines related to groups, communicators, and virtual topologies
its.unc.edu 8
Typical usage: Typical usage:
Extract handle of global group from MPI_COMM_WORLD using MPI_Comm_group
Form new group as a subset of global group using MPI_Group_incl or one of the many group constructors
Create new communicator for new group using MPI_Comm_create
Determine new rank in new communicator using MPI_Comm_rank
its.unc.edu 9
Typical usage cont. : Typical usage cont. :
Conduct communications using any MPI message passing routine
When finished, free up new communicator and group (optional) using MPI_Comm_free and MPI_Group_free
its.unc.edu 10
its.unc.edu 11
Group AccessorsGroup Accessors
its.unc.edu 12
MPI_Group_rank MPI_Group_rank
int MPI_Group_rank (MPI_Group group,int *rank)
Returns the rank of this process in the given group or MPI_UNDEFINED if the process is not a member.
its.unc.edu 13
MPI_Group_size MPI_Group_size
int MPI_Group_size(MPI_Group group, int
*size)
Returns the size of a group - number of processes in the group.
its.unc.edu 14
MPI_Group_compare MPI_Group_compare
int MPI_Group_compare (MPI_Group group1, MPI_Group group2, int *result)
•result - returned result of comparison
Compares two groups and returns an integer result which is MPI_IDENT if the order and members of the two groups are the same, MPI_SIMILAR if only the members are the same, and MPI_UNEQUAL otherwise.
its.unc.edu 15
Group Constructors
Group Constructors
its.unc.edu 16
MPI_Comm_group MPI_Comm_group
int MPI_Comm_group (MPI_Comm comm, MPI_Group *group)
•group - returned value is the handle associated with comm
Determines the group associated with the given communicator.
its.unc.edu 17
MPI_Group_exclMPI_Group_excl
int MPI_Group_excl (MPI_Group group, int n,int *ranks, MPI_Group *newgroup)
Produces a group by reordering an existing group and taking only unlisted members
• n - the size of the ranks array
• ranks - array with list of ranks to exclude from new group , each should be valid and distinct
• newgroup - new group derived from above, preserving the order defined by group (handle)
See also MPI_Group_range_excl
its.unc.edu 18
MPI_Group_inclMPI_Group_incl
int MPI_Group_incl (MPI_Group group, int n, int *ranks, MPI_Group *newgroup)
Produces a group by reordering an existing group and taking only listed members.
•n - the size of the ranks array
•ranks - array with list of ranks to include in the new group, each should be valid and distinct
•newgroup - new group derived from above, preserving the order defined by group (handle)
See also MPI_Group_range_incl
its.unc.edu 19
MPI_Group_intersection
MPI_Group_intersection
int MPI_Group_intersection (MPI_Group group1, MPI_Group group2, MPI_Group *newgroup)
Produces a group as the intersection of two existing groups
•group1 - handle of first group
•group2 - handle of second group
•newgroup - handle of intersection group
its.unc.edu 20
MPI_Group_union MPI_Group_union
int MPI_Group_union (MPI_Group group1, MPI_Group group2, MPI_Group *newgroup)
Produces a group by combining two groups.
•group1 - handle of first group
•group2 - handle of second group
•newgroup - handle of union group
its.unc.edu 21
MPI_Group_difference MPI_Group_difference
int MPI_Group_difference (MPI_Group group1, MPI_Group group2, MPI_Group *newgroup)
Creates a group from the difference of two groups.
•group1 - handle of first group
•group2 - handle of second group
•newgroup - handle of difference group
its.unc.edu 22
set-like operations: set-like operations:
union
• All elements of the first group, followed by all elements of second group, not in first.
intersect
• all elements of the first group that are also in the second group, ordered as in first group.
difference
• all elements of the first group that are not in the second group, ordered as in the first group.
its.unc.edu 23
set-like operations cont. :
set-like operations cont. :
Note that for these operations the order of processes in the output group is determined primarily by order in the first group (if possible) and then, if necessary, by order in the second group.
Neither union nor intersection are commutative, but both are associative.
The new group can be empty, that is, equal to MPI_GROUP_EMPTY.
its.unc.edu 2424
Group DestructorsGroup Destructors
its.unc.edu 25
MPI_Group_freeMPI_Group_free
int MPI_Group_free (MPI_Group *group)
Frees a group
This operation marks a group object for deallocation.
The handle group is set to MPI_GROUP_NULL by the call.
Any on-going operation using this group will complete normally.
its.unc.edu 26
Manipulating CommunicatorsManipulating
Communicators
Accessors, Constructors, Destructors
its.unc.edu 27
MPI_Comm_compare MPI_Comm_compare
int MPI_Comm_compare (MPI_Comm comm1, MPI_Comm comm2, int
*result) Compares two communicators and returns
integer result • MPI_IDENT - contexts and groups are the same
• MPI_CONGRUENT - different contexts but identical groups
• MPI_SIMILAR - different contexts but similar groups
• MPI_UNEQUAL otherwise.
its.unc.edu 28
MPI_Comm_create MPI_Comm_create
int MPI_Comm_create (MPI_Comm comm, MPI_Group group, MPI_Comm *newcomm)
Creates a new communicator from the old communicator and the new group.
• comm - communicator associated with old group
• group - new group to create a communicator for
• newcomm - returns new communicator (handle)
Note: call is executed by all processes in comm (even if they’re not in new group)
returns MPI_COMM_NULL to non-members
its.unc.edu 29
MPI_Comm_splitMPI_Comm_split
Partitions the group into disjoint subgroups
arguments include 2 control arguments
•color - nonnegative integer selects process subset
•key - ranks in order by integer key value, tiebreaker is original rank
A new group for each distinct “color” is created
Use MPI_UNDEFINED as the color argument to be excluded from all groups
its.unc.edu 30
MPI_Comm_dup MPI_Comm_dup
int MPI_Comm_dup (MPI_Comm comm, MPI_Comm
*newcomm)
Duplicates an existing communicator with all its associated information.
This is useful for building safe parallel libraries.
its.unc.edu 31
MPI_Comm_free MPI_Comm_free
int MPI_Comm_free (MPI_Comm *comm)
Marks the communicator object for deallocation. The handle is set to MPI_COMM_NULL
Any pending operations that use this communicator will complete normally; the object is actually deallocated only if there are no other active references to it
its.unc.edu 32
Group and Communicator
Routines Example
Group and Communicator
Routines Example/* NOTE: This does not work on all systems - buggy! */
/* Create two different process groups for separate collective communications exchange. Requires creating new communicators*/
#include "mpi.h"
#include <stdio.h>
#define NPROCS 8
int main(argc,argv)
int argc;
char *argv[]; {
int rank, new_rank, sendbuf, recvbuf,
ranks1[4]={0,1,2,3}, ranks2[4]={4,5,6,7};
MPI_Group orig_group, new_group;
MPI_Comm new_comm;
its.unc.edu 33
Group and Communicator
Routines Example
Group and Communicator
Routines ExampleMPI_Init(&argc,&argv);
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
sendbuf = rank;
/* Extract the original group handle */
MPI_Comm_group(MPI_COMM_WORLD, &orig_group);
/* Divide tasks into two groups based upon rank */
/* Note new_group has a different value on each PE */
if (rank < NPROCS/2) {
MPI_Group_incl(orig_group,NPROCS/2,ranks1,&new_group);
} else {
MPI_Group_incl(orig_group,NPROCS/2,ranks2,&new_group);
}
its.unc.edu 34
Group and Communicator
Routines Example
Group and Communicator
Routines Example/* Create new new communicator and then perform collective
communications */
MPI_Comm_create(MPI_COMM_WORLD, new_group, &new_comm);
MPI_Allreduce(&sendbuf, &recvbuf,1,MPI_INT, MPI_SUM, new_comm);
MPI_Group_rank (new_group, &new_rank);
printf("rank= %d newrank= %d recvbuf= %d\n", rank, new_rank, recvbuf);
MPI_Finalize();
}
its.unc.edu 35
Sample program output:
Sample program output:
rank= 7 newrank= 3 recvbuf= 22
rank= 0 newrank= 0 recvbuf= 6
rank= 1 newrank= 1 recvbuf= 6
rank= 2 newrank= 2 recvbuf= 6
rank= 6 newrank= 2 recvbuf= 22
rank= 3 newrank= 3 recvbuf= 6
rank= 4 newrank= 0 recvbuf= 22
rank= 5 newrank= 1 recvbuf= 22
its.unc.edu 36
Previous Example Done with
MPI_Comm_split
Previous Example Done with
MPI_Comm_split/* this fixes the buggy Maui code by using MPI_Comm_split */
#include "mpi.h"
#include <stdio.h>
#define NPROCS 8
#define MASTER 0
#define MSGSIZE 7
int main(argc,argv)
int argc;
char *argv[]; {
int rank, new_rank,sendbuf, recvbuf,color;
char msg[MSGSIZE+1]=" ";
MPI_Comm new_comm;
its.unc.edu 37
Split example, Cont.Split example, Cont.
MPI_Init(&argc,&argv);
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
sendbuf = rank;
/* Divide tasks into two distinct groups. First */
/* create new group and then a new communicator.*/
/* Find new rank in new group and setup for the */
/* collective communication broadcast if MASTER.*/
/* use integer division to split group into 2 "colors“ */
/* 0 and 1 */
color = (2*rank)/NPROCS;
MPI_Comm_split(MPI_COMM_WORLD,color,rank,&new_comm);
MPI_Comm_rank(new_comm, &new_rank)
its.unc.edu 38
Split ConcludedSplit Concluded
if (new_rank == MASTER) sprintf(msg,"Group %d",color+1);
MPI_Bcast (&msg, MSGSIZE, MPI_CHAR, MASTER, new_comm);
MPI_Allreduce (&sendbuf, &recvbuf, 1,MPI_INT,MPI_SUM, new_comm);
printf("rank= %d newrank= %d msg= %s sum=%d\n", rank, new_rank, msg, recvbuf);
MPI_Finalize();
}