How does CUDA assign device IDs to GPUs?

cuda gpu gpgpu nvidia
solvingPuzzles picture solvingPuzzles · Dec 8, 2012 · Viewed 23.6k times · Source

When a computer has multiple CUDA-capable GPUs, each GPU is assigned a device ID. By default, CUDA kernels execute on device ID 0. You can use cudaSetDevice(int device) to select a different device.

Let's say I have two GPUs in my machine: a GTX 480 and a GTX 670. How does CUDA decide which GPU is device ID 0 and which GPU is device ID 1?

Ideas for how CUDA might assign device IDs (just brainstorming):

  • descending order of compute capability
  • PCI slot number
  • date/time when the device was added to system (device that was just added to computer is higher ID number)

Motivation: I'm working on some HPC algorithms, and I'm benchmarking and autotuning them for several GPUs. My processor has enough PCIe lanes to drive cudaMemcpys to 3 GPUs at full bandwidth. So, instead of constantly swapping GPUs in and out of my machine, I'm planning to just keep 3 GPUs in my computer. I'd like to be able to predict what will happen when I add or replace some GPUs in the computer.

Answer

Liang Xiao picture Liang Xiao · Mar 31, 2017

Set the environment variable CUDA_DEVICE_ORDER as:

export CUDA_DEVICE_ORDER=PCI_BUS_ID

Then the GPU IDs will be ordered by pci bus IDs.

Related questions

Is it possible to run CUDA on AMD GPUs?

I'd like to extend my skill set into GPU computing. I am familiar with raytracing and realtime graphics(OpenGL), but the next generation of graphics and high performance computing seems to be in GPU computing or something like it. I …

Read More
nvidia-smi Volatile GPU-Utilization explanation?

I know that nvidia-smi -l 1 will give the GPU usage every one second (similarly to the following). However, I would appreciate an explanation on what Volatile GPU-Util really means. Is that the number of used SMs over total SMs, or …

Read More
CUDA 7.5 installation: Unsupported compiler error

I just tried installing CUDA 7.5 on my laptop. I disabled lightdm and did sudo sh cuda7.5.run. The driver installation passed but then I got an error Unsupported compiler ... and the installation fails. How can I resolve this issue?

Read More