MATLAB Parallel Computing ToolboxA.HosseiniEmail: s.a.hosseini86@gmail.comCourse : Professional Architecture

Of 372

Product Overview

• solve computationally and data intensive problems using

multicore processors, GPUs, and computer clusters.

• The toolbox provides twelve workers to execute

applications locally without changing the code

• MDCS software allows you to run as many MATLAB

workers on a remote cluster

Of 373

Key Problems Addressed by Parallel Computing

• Running Parallel for-Loops (parfor)• Executing Batch Jobs in Parallel• Partitioning Large Data Sets

Of 374

Running Parallel for-Loops (parfor)

• Many applications involve multiple repetitive segments of code.

• Often you can use for-loops to solve these cases.

• The ability to execute code in parallel, on one computer or on a cluster of computers, can significantly improve performance in many cases

▫ Many iterations▫ Long iterations

Of 375

Executing Batch Jobs in Parallel

• When working interactively in a MATLAB session, you can offload work to a MATLAB worker session to run as a batch job.

• MATLAB session is not blocked.

• The MATLAB worker can run either on the same machine as the client, or if using MATLAB Distributed Computing Server, on a remote cluster machine.

Of 376

Partitioning Large Data Sets

• If you have an array that is too large for your computer's memory, it cannot be easily handled in a single MATLAB session. Parallel Computing Toolbox software allows you to distribute that array among multiple MATLAB workers, so that each worker contains only a part of the array.

Of 377

Introduction to Parallel Solutions

•Interactively Run a Loop in Parallel

•Run a Batch Job

•Run a Batch Parallel Loop

•Distributing Arrays and Running SPMD

Of 378

Interactively Run a Loop in Parallel

• This section shows how to modify a simple for-loop so that it runs in parallel.

for i=1:1024 A(i) = sin(i*2*pi/1024);Endplot(A)

matlabpool open local 3parfor i=1:1024 A(i) = sin(i*2*pi/1024);Endplot(A)matlabpool close

Of 379

Run a Batch Job• To offload work from your MATLAB session to another

session, you can use the batch command. This example uses the for-loop from the last section inside a script.

edit mywave

for i=1:1024

A(i) = sin(i*2*pi/1024);

End

job = batch('mywave')

wait(job)

load(job, 'A')

plot(A)

destroy(job)

Of 3710

Run a Batch Parallel Loop

• combine the abilities to offload a job and run a parallel loop

Of 3711

Distributing Arrays and Running SPMD

•Distributed Arrays (see page 9 on

document)

•Single Program Multiple Data(SPMD) (see

page 34 on document)

Of 3712

Interactive Parallel Computation with pmode(P45)

•pmode lets you work interactively with a parallel job running simultaneously on several labs.

•provides a desktop with a display for each lab running the job.

•When you exit your pmode session, its job is effectively destroyed, and all information and data on the labs is lost. Starting another pmode session always begins from a clean state.

Of 3713

Run Parallel Jobs Interactively Using pmode

Of 3714

Profiling Parallel Code

• The parallel profiler provides an extension of the profile command and the profile viewer specifically for parallel jobs, to enable you to see how much time each lab spends evaluating each function and how much time communicating or waiting for communications with the other labs.

• mpiprofile on

• Execution time of each function on each lab• Execution time of each line of code in each function• Amount of data transferred between each lab• Amount of time each lab spends waiting for communications

Of 3715

Viewing Parallel Profile Data

• P>> R1 = rand(16, codistributor())• P>> R2 = rand(16, codistributor())• P>> mpiprofile on• P>> P = R1*R2• P>> mpiprofile off• P>> mpiprofile viewer

Of 3716

The function summary report Column Header Description

Calls How many times the function was called on this lab

Total Time The total amount of time this lab spent executing this function

Self Time The time this lab spent inside this function, not within children or subfunctions

Total Comm Time The total time this lab spent transferring data with other labs, including waiting time to receive data

Self Comm Waiting Time The time this lab spent during this function waiting to receive data from other labs

Total Interlab Data The amount of data transferred to and from this lab for this function

Computation Time Ratio The ratio of time spent in computation for this function vs. total time (which includes communication time) for this function

Total Time Plot Bar graph showing relative size of Self Time, Self Comm Waiting Time, and Total Time for this function on this lab

Of 3717

Parallel computing with MATLAB

•Parallel Computing Toolbox (formerly Distributed Computing Toolbox) should be installed on the computer where you write your applications.

•MATLAB Distributed Computing Server (formerly MATLAB Distributed Computing Engine) should be installed on each computer of the cluster that performs the computation.

Of 3718

job

•A job is some large operation that you need to perform in your MATLAB session. A job is broken down into segments called tasks. You decide how best to divide your job into tasks. You could divide your job into identical tasks, but tasks do not have to be identical.

Of 3719

client

•this is on the machine where you program MATLAB. The client uses Parallel Computing Toolbox software to perform the definition of jobs and tasks. MATLAB Distributed Computing Server software is the product that performs the execution of your job by evaluating each of its tasks and returning the result to your client session.

Of 3720

job manager

•the part of the engine that coordinates the execution of jobs and the evaluation of their tasks. The job manager distributes the tasks for evaluation to the server's individual MATLAB sessions called workers.

Of 3721

Basic Parallel Computing Configuration

Of 3722

Life Cycle of a Job

Of 3723

Job Stage Description

Pending You a job on the scheduler with the createJob function in your client session of Parallel Computing Toolbox software. The job's first state is pending. This is when you define the job by adding tasks to it. create

Queued When you execute the submit function on a job, the scheduler places the job in the queue, and the job's state is queued. The scheduler executes jobs in the queue in the sequence in which they are submitted, all jobs moving up the queue as the jobs before them are finished. You can change the order of the jobs in the queue with the promote and demote functions.

Running When a job reaches the top of the queue, the scheduler distributes the job's tasks to worker sessions for evaluation. The job's state is running. If more workers are available than necessary for a job's tasks, the scheduler begins executing the next job. In this way, there can be more than one job running at a time.

Finished When all of a job's tasks have been evaluated, a job is moved to the finished state. At this time, you can retrieve the results from all the tasks in the job with the function getAllOutputArguments.

Failed When using a third-party scheduler, a job might fail if the scheduler encounters an error when attempting to execute its commands or access necessary files.

Destroyed When a job's data has been removed from its data location or from the job manager, the state of the job in the client is destroyed. This state is available only as long as the job object remains in the client.

Of 3724

Parallel Configurations for Cluster Access

• You create and modify configurations through the Configurations Manager. You access the Configurations Manager using the Parallel pull-down menu on the MATLAB desktop. Select Parallel > Manage Configurations to open the Configurations Manger.

Of 3725

• The first time you open the Configurations Manager, it lists only one configuration called local, which at first is the default configuration and has only default settings.

Of 3726

Creating and Modifying User Configurations

• In the Configurations Manager, select New > jobmanager. This specifies that you want a new configuration whose type of scheduler is a job manager

Of 3727

Creating and Modifying User Configurations

• Enter a configuration name MyJMconfig1 and a description as shown in the following figure. In the Scheduler tab, enter the host name for the machine on which the job manager is running and the name of the job manager.

Of 3728

Creating and Modifying User Configurations

• In the Jobs tab, enter the maximum and minimum number of workers. This specifies that for jobs using this configuration, they require at least four workers and use no more than four workers. Therefore, the job runs on exactly four workers, even if it has to wait until four workers are available before starting.

Of 3729

Creating and Modifying User Configurations

• To create a similar configuration with just a few differences, you can duplicate an existing configuration and modify only the parts you need to change:▫ In the Configurations Manager, right-click the

configuration MyJMconfig1 in the list and select Duplicate.

Of 3730

Exporting and Importing Configurations• To export a parallel configuration: In the Configurations Manager, select (highlight) the

configuration you want to export.▫ Click File > Export.

• Configurations saved in this way can then be imported by other MATLAB software users:

In the Configuration Manager, ▫ click File > Import.

Of 3731

Validate Configurations• Open the Configurations Manager by selecting on the

desktop Parallel > Manage Configurations.• In the Configurations Manager, click the name of the

configuration you want to test in the list of those available. • Click Start Validation.

Of 3732

• The configuration listing displays the overall validation result for each configuration. The following figure shows overall validation results for one configuration that passed and one that failed. The selected configuration is the one that failed.

Of 3733

How to submit program for run?

• Create a Scheduler Object

• Create a Job

• Create Tasks

• Submit a Job to the Scheduler

• Retrieve the Job's Results

Of 3734

Creating Jobs• sched = findResource('scheduler','type','local'); (create an object in your local MATLAB session representing the local

scheduler.)• job1 = createJob(sched) (This statement creates a job in the scheduler's data

location)• get(job1) ( see all the properties of this job object)• Sched (The scheduler's display now indicates the existence of your job)• createTask(job1, @rand, 1, {{3,3} {3,3} {3,3} {3,3} {3,3}}); (Tasks define the functions to be evaluated by the workers during the running

of the job)• get(job1,'Tasks')• submit(job1) (To run your job and have its tasks evaluated)• waitForState(job1)• results = getAllOutputArguments(job1); (to retrieve the results from all the tasks in the job)• sched• job1• get(job1,'Tasks')

Of 3735

Books

Of 3736

Books(Cont.)

Of 3737

Thanks for your attentionAny Question?