using MPI with docker containers

revolutionary picture revolutionary · Jan 3, 2018 · Viewed 7.8k times · Source

I have created a docker image based on Ubuntu 16.04 and with all the dependencies needed to run MPI.

It is public on docker-hub at: https://hub.docker.com/r/orwel84/ubuntu-16-mpi/

I use this image to create an MPI container. I can also compile a simple mpi-hello-world.c (which comes inside the container) and run it with mpirun.

These are the steps I use and, (if you have Docker installed you can reproduce them too) :

  1. docker run -it orwel84/ubuntu-16-mpi bash
  2. (on container's shell) mpirun -np 4 --allow-run-as-root ./mpi_hello_world

You will see Output:

Hello world from processor 6f9b11cef939, rank 0 out of 4 processors

Hello world from processor 6f9b11cef939, rank 1 out of 4 processors

Hello world from processor 6f9b11cef939, rank 2 out of 4 processors

Hello world from processor 6f9b11cef939, rank 3 out of 4 processors

Question:

Right now all the above four mpi processes run inside a single container.

How can I use mpirun to run on multiple containers on a single host? And also how can I use Docker swarm to run on multiple nodes of the swarm?

Please help. Thankyou.

Answer

psaha4 picture psaha4 · Mar 7, 2018

for multiple containers in a single machine:

1.You can create multiple containers on a single host machine and inspect their ip address (under docker bridge network).

docker inspect -f "{{ .NetworkSettings.IPAddress }}" containerName

2. Now attach to one of the containers and create a host file listing ip-addresses of the containers you have created.

3.Now run the application:

mpirun -np 4 -hostfile hostfile_you_created ./mpi_hello_world