在 C++ 版本的 tensorflow 上使用多个 gpus
Use multiple gpus on the C++ version of tensorflow
先说明一下我的运行环境：
win10x64
cuda9.1 and cudnn7
gtx1080Ti x2
i7-6850k
我用tensorflow的c++版本写了一个读取pb文件然后输入图像进行预测的程序。我的目标是在一个线程或一个线程一个gpu中使用tensorflow时可以调用所有gpu。
首先使用windows下的python调用tensorflow slim训练，然后使用freeze_graph.py.
将保存的模型文件转为freeze文件
但是我发现在使用session->运行()函数时只调用了一个gpu。无论是创建多线程还是单线程，我都是用下面的方法调用多个gpu：
tensorflow::graph::SetDefaultDevice("0", &graphdef);
或
GraphDef graphdef; //Graph Definition for current model
Status status_load = ReadBinaryProto(Env::Default(), model_path, &graphdef); //read graph from pb_file
if (!status_load.ok()) {
std::cout << " ERROR: Loading model failed...\n"
<< model_path
<< std::endl;
std::cout << status_load.ToString() << "\n";
system("pause");
return;
}
tensorflow::SessionOptions options;
tensorflow::ConfigProto &config = options.config;
config.set_log_device_placement(true);
config.mutable_gpu_options()->set_allow_growth(true);
//config.mutable_gpu_options()->set_allocator_type(std::string("BFC"));
//config.mutable_gpu_options()->set_visible_device_list("");//this no error,but still can only call one gpu
//config.mutable_gpu_options()->set_visible_device_list("0");//error!
config.mutable_gpu_options()->set_visible_device_list("0,1");//error!
config.mutable_gpu_options()->set_per_process_gpu_memory_fraction(1);
Session* session;
Status status = NewSession(SessionOptions(options), &session);
Status status_create = session->Create(graphdef);
以上两种方法均失败，错误提示相同：
2018-08-08 09:25:55.953495: I D:\MyProject\tensorflow-1.10.0-rc1\tensorflow\core\platform\cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX AVX2
2018-08-08 09:25:56.541237: I D:\MyProject\tensorflow-1.10.0-rc1\tensorflow\core\common_runtime\gpu\gpu_device.cc:1404] Found device 0 with properties:
name: GeForce GTX 1080 Ti major: 6 minor: 1 memoryClockRate(GHz): 1.582
pciBusID: 0000:06:00.0
totalMemory: 11.00GiB freeMemory: 9.02GiB
2018-08-08 09:25:56.708385: I D:\MyProject\tensorflow-1.10.0-rc1\tensorflow\core\common_runtime\gpu\gpu_device.cc:1404] Found device 1 with properties:
name: GeForce GTX 1080 Ti major: 6 minor: 1 memoryClockRate(GHz): 1.582
pciBusID: 0000:0b:00.0
totalMemory: 11.00GiB freeMemory: 9.02GiB
2018-08-08 09:25:56.731390: I D:\MyProject\tensorflow-1.10.0-rc1\tensorflow\core\common_runtime\gpu\gpu_device.cc:1483] Adding visible gpu devices: 0, 1
2018-08-08 09:26:04.117910: I D:\MyProject\tensorflow-1.10.0-rc1\tensorflow\core\common_runtime\gpu\gpu_device.cc:964] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-08-08 09:26:04.131670: I D:\MyProject\tensorflow-1.10.0-rc1\tensorflow\core\common_runtime\gpu\gpu_device.cc:970] 0 1
2018-08-08 09:26:04.142367: I D:\MyProject\tensorflow-1.10.0-rc1\tensorflow\core\common_runtime\gpu\gpu_device.cc:983] 0: N N
2018-08-08 09:26:04.152745: I D:\MyProject\tensorflow-1.10.0-rc1\tensorflow\core\common_runtime\gpu\gpu_device.cc:983] 1: N N
2018-08-08 09:26:04.173833: E D:\MyProject\tensorflow-1.10.0-rc1\tensorflow\core\common_runtime\gpu\gpu_process_state.cc:105] Invalid allocator type: 0,1
2018-08-08 09:26:04.189278: E D:\MyProject\tensorflow-1.10.0-rc1\tensorflow\core\common_runtime\direct_session.cc:158] Internal: Failed to get memory allocator for TF GPU 0 with 11811160064 bytes of memory.
ERROR: Creating Session failed...
Internal: Failed to create session.
Press any key to continue......
根据提示，我把gpu的id换成了“/gpu/:0”和“/device:GPU:0”。但是提示解析失败，如下：
2018-08-08 09:31:07.052736: I D:\MyProject\tensorflow-1.10.0-rc1\tensorflow\core\platform\cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX AVX2
2018-08-08 09:31:07.643228: E D:\MyProject\tensorflow-1.10.0-rc1\tensorflow\core\common_runtime\direct_session.cc:158] Invalid argument: Could not parse entry in 'visible_device_list': '/device:GPU:0'. visible_device_list = /device:GPU:0
ERROR: Creating Session failed...
Internal: Failed to create session.
或
2018-08-08 09:32:28.753232: I D:\MyProject\tensorflow-1.10.0-rc1\tensorflow\core\platform\cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX AVX2
2018-08-08 09:32:29.082282: E D:\MyProject\tensorflow-1.10.0-rc1\tensorflow\core\common_runtime\direct_session.cc:158] Invalid argument: Could not parse entry in 'visible_device_list': '/gpu:0'. visible_device_list = /gpu:0
ERROR: Creating Session failed...
Internal: Failed to create session.
然后我在/github/tensorflow的issues中发现了同样的错误。我按照他们的方法尝试了以下方法：
遵循这些计划#5379
1. {tf_root}\tensorflow\tf_version_script.lds
修改这个文件，添加“protobuf;”
失败！
2.添加相应的库。
tf_core_gpu_kernelss.lib
training_ops_gen_cc.lib
transform_graph.lib
tf_protos_cc.lib
user_ops_gen_cc.lib
失败！
但是如果我用下面的方法：
config.mutable_gpu_options()->set_visible_device_list("")
或
tensorflow::graph::SetDefaultDevice("", &graphdef)
这可以通过，而且运行，但仍然只调用了一个gpu！
我在这个问题中发现了同样的错误#18861，但是我没有在下面找到C++解决方案，所以我怀疑是我的tensorflow问题，我重新编译了1.9.0和最新的1.10.0 -rc1。但是得到同样的错误
有人可以帮我解决这个问题吗？ └(^o^)┘
我真的很感激！
谢谢你重播我！
我可能找到了解决方案，但是今天测试没有达到我的要求。
tensorflow::SessionOptions options;
tensorflow::ConfigProto &config = options.config;
auto* device_count = config.mutable_device_count();
/*device_count->insert({ "CPU", 1 });*/
//device_count->insert({ "GPU", 1 });//1 represents one gpu, not the "/gpu:0"
device_count->insert({ "GPU", 2 });//2 represents two gpu, it is "/gpu:0" and "/gpu:1"
Session* session;
Status status = NewSession(options, &session);//creat new Session
std::vector<DeviceAttributes> response;
session->ListDevices(&response);
//print the device list
for (int temIndex = 0; temIndex < response.size(); ++temIndex) {
auto temValue= response[temIndex];
std::cout << "ListDevices(): " << temIndex << " " << response[temIndex].name() << std::endl;
}
使用此方法与以下方法相同：
options.config.mutable_gpu_options()->set_visible_device_list("");
仍然无法明确定义使用的gpu，仍然所有的计算都放在一个gpu上，我想这可能是我的方法还是有问题。
但我觉得我要找到解决办法....
使用CUDA_VISIBLE_DEVICES为不同的进程设置特定的设备，这是我的解决方案
先说明一下我的运行环境：
win10x64
cuda9.1 and cudnn7
gtx1080Ti x2
i7-6850k
我用tensorflow的c++版本写了一个读取pb文件然后输入图像进行预测的程序。我的目标是在一个线程或一个线程一个gpu中使用tensorflow时可以调用所有gpu。
首先使用windows下的python调用tensorflow slim训练，然后使用freeze_graph.py.
将保存的模型文件转为freeze文件但是我发现在使用session->运行()函数时只调用了一个gpu。无论是创建多线程还是单线程，我都是用下面的方法调用多个gpu：
tensorflow::graph::SetDefaultDevice("0", &graphdef);
或
GraphDef graphdef; //Graph Definition for current model
Status status_load = ReadBinaryProto(Env::Default(), model_path, &graphdef); //read graph from pb_file
if (!status_load.ok()) {
std::cout << " ERROR: Loading model failed...\n"
<< model_path
<< std::endl;
std::cout << status_load.ToString() << "\n";
system("pause");
return;
}
tensorflow::SessionOptions options;
tensorflow::ConfigProto &config = options.config;
config.set_log_device_placement(true);
config.mutable_gpu_options()->set_allow_growth(true);
//config.mutable_gpu_options()->set_allocator_type(std::string("BFC"));
//config.mutable_gpu_options()->set_visible_device_list("");//this no error,but still can only call one gpu
//config.mutable_gpu_options()->set_visible_device_list("0");//error!
config.mutable_gpu_options()->set_visible_device_list("0,1");//error!
config.mutable_gpu_options()->set_per_process_gpu_memory_fraction(1);
Session* session;
Status status = NewSession(SessionOptions(options), &session);
Status status_create = session->Create(graphdef);
以上两种方法均失败，错误提示相同：
2018-08-08 09:25:55.953495: I D:\MyProject\tensorflow-1.10.0-rc1\tensorflow\core\platform\cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX AVX2
2018-08-08 09:25:56.541237: I D:\MyProject\tensorflow-1.10.0-rc1\tensorflow\core\common_runtime\gpu\gpu_device.cc:1404] Found device 0 with properties:
name: GeForce GTX 1080 Ti major: 6 minor: 1 memoryClockRate(GHz): 1.582
pciBusID: 0000:06:00.0
totalMemory: 11.00GiB freeMemory: 9.02GiB
2018-08-08 09:25:56.708385: I D:\MyProject\tensorflow-1.10.0-rc1\tensorflow\core\common_runtime\gpu\gpu_device.cc:1404] Found device 1 with properties:
name: GeForce GTX 1080 Ti major: 6 minor: 1 memoryClockRate(GHz): 1.582
pciBusID: 0000:0b:00.0
totalMemory: 11.00GiB freeMemory: 9.02GiB
2018-08-08 09:25:56.731390: I D:\MyProject\tensorflow-1.10.0-rc1\tensorflow\core\common_runtime\gpu\gpu_device.cc:1483] Adding visible gpu devices: 0, 1
2018-08-08 09:26:04.117910: I D:\MyProject\tensorflow-1.10.0-rc1\tensorflow\core\common_runtime\gpu\gpu_device.cc:964] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-08-08 09:26:04.131670: I D:\MyProject\tensorflow-1.10.0-rc1\tensorflow\core\common_runtime\gpu\gpu_device.cc:970] 0 1
2018-08-08 09:26:04.142367: I D:\MyProject\tensorflow-1.10.0-rc1\tensorflow\core\common_runtime\gpu\gpu_device.cc:983] 0: N N
2018-08-08 09:26:04.152745: I D:\MyProject\tensorflow-1.10.0-rc1\tensorflow\core\common_runtime\gpu\gpu_device.cc:983] 1: N N
2018-08-08 09:26:04.173833: E D:\MyProject\tensorflow-1.10.0-rc1\tensorflow\core\common_runtime\gpu\gpu_process_state.cc:105] Invalid allocator type: 0,1
2018-08-08 09:26:04.189278: E D:\MyProject\tensorflow-1.10.0-rc1\tensorflow\core\common_runtime\direct_session.cc:158] Internal: Failed to get memory allocator for TF GPU 0 with 11811160064 bytes of memory.
ERROR: Creating Session failed...
Internal: Failed to create session.
Press any key to continue......
根据提示，我把gpu的id换成了“/gpu/:0”和“/device:GPU:0”。但是提示解析失败，如下：
2018-08-08 09:31:07.052736: I D:\MyProject\tensorflow-1.10.0-rc1\tensorflow\core\platform\cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX AVX2
2018-08-08 09:31:07.643228: E D:\MyProject\tensorflow-1.10.0-rc1\tensorflow\core\common_runtime\direct_session.cc:158] Invalid argument: Could not parse entry in 'visible_device_list': '/device:GPU:0'. visible_device_list = /device:GPU:0
ERROR: Creating Session failed...
Internal: Failed to create session.
或
2018-08-08 09:32:28.753232: I D:\MyProject\tensorflow-1.10.0-rc1\tensorflow\core\platform\cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX AVX2
2018-08-08 09:32:29.082282: E D:\MyProject\tensorflow-1.10.0-rc1\tensorflow\core\common_runtime\direct_session.cc:158] Invalid argument: Could not parse entry in 'visible_device_list': '/gpu:0'. visible_device_list = /gpu:0
ERROR: Creating Session failed...
Internal: Failed to create session.
然后我在/github/tensorflow的issues中发现了同样的错误。我按照他们的方法尝试了以下方法：
遵循这些计划#5379
1. {tf_root}\tensorflow\tf_version_script.lds
修改这个文件，添加“protobuf;”
失败！
2.添加相应的库。
tf_core_gpu_kernelss.lib
training_ops_gen_cc.lib
transform_graph.lib
tf_protos_cc.lib
user_ops_gen_cc.lib
失败！
但是如果我用下面的方法：
config.mutable_gpu_options()->set_visible_device_list("")
或
tensorflow::graph::SetDefaultDevice("", &graphdef)
这可以通过，而且运行，但仍然只调用了一个gpu！
我在这个问题中发现了同样的错误#18861，但是我没有在下面找到C++解决方案，所以我怀疑是我的tensorflow问题，我重新编译了1.9.0和最新的1.10.0 -rc1。但是得到同样的错误
有人可以帮我解决这个问题吗？ └(^o^)┘
我真的很感激！
谢谢你重播我！
我可能找到了解决方案，但是今天测试没有达到我的要求。
tensorflow::SessionOptions options;
tensorflow::ConfigProto &config = options.config;
auto* device_count = config.mutable_device_count();
/*device_count->insert({ "CPU", 1 });*/
//device_count->insert({ "GPU", 1 });//1 represents one gpu, not the "/gpu:0"
device_count->insert({ "GPU", 2 });//2 represents two gpu, it is "/gpu:0" and "/gpu:1"
Session* session;
Status status = NewSession(options, &session);//creat new Session
std::vector<DeviceAttributes> response;
session->ListDevices(&response);
//print the device list
for (int temIndex = 0; temIndex < response.size(); ++temIndex) {
auto temValue= response[temIndex];
std::cout << "ListDevices(): " << temIndex << " " << response[temIndex].name() << std::endl;
}
使用此方法与以下方法相同：
options.config.mutable_gpu_options()->set_visible_device_list("");
仍然无法明确定义使用的gpu，仍然所有的计算都放在一个gpu上，我想这可能是我的方法还是有问题。
但我觉得我要找到解决办法....
使用CUDA_VISIBLE_DEVICES为不同的进程设置特定的设备，这是我的解决方案