TensorFlow 从多个 GPU 中选择要使用的 GPU

Question

我是 TensorFlow 的新手，已经按照 TensorFlow 网站上的说明安装了 CUDA-7.5 和 cudnn-v4。调整 TensorFlow 配置文件并尝试运行来自网站的以下示例后：

python -m tensorflow.models.image.mnist.convolutional

我很确定 TensorFlow 使用其中一个 GPU 而不是另一个，但是，我希望它使用速度更快的那个。我想知道这个示例代码是否只是默认使用它找到的第一个 GPU。如果是这样，我如何选择在 python 中的 TensorFlow 代码中使用哪个 GPU？

我在运行运行示例代码时收到的消息是：

ldt-tesla:~$ python -m tensorflow.models.image.mnist.convolutional
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcublas.so locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcudnn.so locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcufft.so locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcuda.so locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcurand.so locally
Extracting data/train-images-idx3-ubyte.gz
Extracting data/train-labels-idx1-ubyte.gz
Extracting data/t10k-images-idx3-ubyte.gz
Extracting data/t10k-labels-idx1-ubyte.gz
I tensorflow/core/common_runtime/gpu/gpu_init.cc:102] Found device 0 with properties:
name: Tesla K20c
major: 3 minor: 5 memoryClockRate (GHz) 0.7055
pciBusID 0000:03:00.0
Total memory: 4.63GiB
Free memory: 4.57GiB
W tensorflow/stream_executor/cuda/cuda_driver.cc:572] creating context when one is currently active; existing: 0x2f27390
I tensorflow/core/common_runtime/gpu/gpu_init.cc:102] Found device 1 with properties:
name: Quadro K2200
major: 5 minor: 0 memoryClockRate (GHz) 1.124
pciBusID 0000:02:00.0
Total memory: 3.95GiB
Free memory: 3.62GiB
I tensorflow/core/common_runtime/gpu/gpu_init.cc:59] cannot enable peer access from device ordinal 0 to device ordinal 1
I tensorflow/core/common_runtime/gpu/gpu_init.cc:59] cannot enable peer access from device ordinal 1 to device ordinal 0
I tensorflow/core/common_runtime/gpu/gpu_init.cc:126] DMA: 0 1
I tensorflow/core/common_runtime/gpu/gpu_init.cc:136] 0:   Y N
I tensorflow/core/common_runtime/gpu/gpu_init.cc:136] 1:   N Y
I tensorflow/core/common_runtime/gpu/gpu_device.cc:806] Creating TensorFlow device (/gpu:0) -> (device: 0, name: Tesla K20c, pci bus id: 0000:03:00.0)
I tensorflow/core/common_runtime/gpu/gpu_device.cc:793] Ignoring gpu device (device: 1, name: Quadro K2200, pci bus id: 0000:02:00.0) with Cuda multiprocessor count: 5. The minimum required count is 8. You can adjust this requirement with the env var TF_MIN_GPU_MULTIPROCESSOR_COUNT.
Initialized!

Answer 1

您可以设置 CUDA_VISIBLE_DEVICES 环境变量以仅公开您想要的变量，在 masking gpus 上引用此示例：

CUDA_VISIBLE_DEVICES=1  Only device 1 will be seen
CUDA_VISIBLE_DEVICES=0,1    Devices 0 and 1 will be visible
CUDA_VISIBLE_DEVICES=”0,1”  Same as above, quotation marks are optional
CUDA_VISIBLE_DEVICES=0,2,3  Devices 0, 2, 3 will be visible; device 1 is masked

Answer 2

您可以设置要在运行时运行程序启动的 GPU，而不是将其硬编码到您的脚本中。这将防止运行ning 在没有多个 GPU 或没有太多 GPU 的设备上出现问题。

假设您想运行在 GPU #3 上，您可以这样做：

CUDA_VISIBLE_DEVICES=3, python -m tensorflow.models.image.mnist.convolutional

TensorFlow 从多个 GPU 中选择要使用的 GPU

TensorFlow Choose GPU to use from multiple GPUs

python

cuda

gpu

tensorflow