Azure Batch and MPI

Azure Batch and MPIMartin Abbott@martinabbott

What is MPI?• Message Passing Interface• Communication protocol for

parallel computing• Scalable and portable• Good for HPC• On Azure, two main

implementations• Open MPI - Linux• Intel MPI – Linux and Windows• A8, A9, H16r, H16mr VMs• RDMA networking

Some scenarios• Computational Fluid

Dynamics• OpenFOAM

• Free on Linux• Also available on Windows

• Fire Simulation• FDS (Fire Dynamics Software)

• Free on both Windows and Linux

Approach• Create input files in a preprocessor• Copy scripts and applications to storage• Create pool

• Enable internode communication• Set startup task to install pre-requisites if required

• Create Job• Create Task

• Mount blob container• Unzip code, libraries and input files• Run code in parallel mode (mpirun)

• Download results for post processing

Demo

Automation possibilities• Creating task can be achieved through PowerShell• Azure Functions supports PowerShell• Potential Approach• Upload input files and push message to

Service Bus Queue• Use QueueTrigger to execute a

PowerShell command• Use BlobTrigger to execute a Function

command to zip results

More information• Scenarios and run throughs (NOTE: Lots of errors in the script

so mostly trial and error)https://blogs.technet.microsoft.com/windowshpc/2016/07/20/introducing-mpi-support-for-linux-on-azure-batch/

• GitHub repository for sampleshttps://github.com/Azure/azure-batch-samples

• OpenFOAMhttp://www.openfoam.com/

• ParaViewhttp://www.paraview.org/

• Fire Dynamics Simulatorhttps://www.nist.gov/services-resources/software/fds-and-smokeview

Thank you!