OpenVINO 无法设置 FP16 精度与用户输入精度不对应的 Blob

Question

我命令--data_type FP16确认我在生成IR格式文件时可以使用FP16精度。

python mo_tf.py --saved_model_dir C:\Users\Hsien\Desktop\OCT\saved_model --input_shape [1,256,256,3] --data_type FP16

但是当我尝试以 FP16 精度执行我的推理引擎代码并且设备分配是 MYRIAD 或 GPU.

Traceback (most recent call last):
  File "C:\Users\Hsien\Desktop\OCT\test.py", line 27, in <module>
    inference_request.set_blob(blob_name=input_blob_name, blob=input_blob)
  File "ie_api.pyx", line 1438, in openvino.inference_engine.ie_api.InferRequest.set_blob
RuntimeError: [ PARAMETER_MISMATCH ] Failed to set Blob with precision not corresponding to user input precision

这是我的推理引擎代码

from openvino.inference_engine import IECore, Blob, TensorDesc
import cv2
import numpy as np

IMG_PATH = r"C:\Users\Hsien\Desktop\OCT\input_crop.jpg"
XML_PATH = r"C:\Users\Hsien\Desktop\OCT\saved_model\saved_model.xml"
BIN_PATH = r"C:\Users\Hsien\Desktop\OCT\saved_model\saved_model.bin"

ie_core_handler = IECore()
network = ie_core_handler.read_network(model=XML_PATH, weights=BIN_PATH)
executable_network = ie_core_handler.load_network(network, device_name='MYRIAD', num_requests=1)
inference_request = executable_network.requests[0]

image = cv2.imread(IMG_PATH)
re_img = cv2.resize(src=image, dsize=(256, 256))
input_data = np.expand_dims(np.transpose(re_img, (2, 0, 1)), 0).astype(np.float16)
tensor_description = TensorDesc(precision="FP16", dims=(1, 3, 256, 256), layout='NCHW')
input_blob = Blob(tensor_description, input_data)

input_blob_name = next(iter(inference_request.input_blobs))
inference_request.set_blob(blob_name=input_blob_name, blob=input_blob)
inference_request.infer()
output_blob_name = next(iter(inference_request.output_blobs))
output = inference_request.output_blobs[output_blob_name].buffer
print(output)

它可以与FP32精度一起工作，设备分配是CPU，我只改变了以下三行.

executable_network = ie_core_handler.load_network(network, device_name='CPU', num_requests=1)
input_data = np.expand_dims(np.transpose(re_img, (2, 0, 1)), 0).astype(np.float32)
tensor_description = TensorDesc(precision="FP32", dims=(1, 3, 256, 256), layout='NCHW')

我的环境是Windows11，版本openvino_2021.4.752。

我的设备是配备英特尔® UHD 显卡 750 和英特尔® 神经计算棒 2 的英特尔® 酷睿™ i7-11700K。

Answer 1

最后，我发现我不需要在推理引擎代码中将 FP32 更改为 FP16。

我的 MYRIAD 设备可以正常工作，感谢！

Answer 2

您遇到的错误：Failed to set Blob with precision not corresponding to user input precision是由于输入精度默认设置为 FP32。参考Step 2. Configure Input and Output of the Model, in Integrate Inference Engine.

请在executable_network前添加以下代码：

# Get names of the input blob
input_blob_name = next(iter(network.input_info))
# Set input precision
network.input_info[input_blob_name].precision = 'FP16'

OpenVINO 无法设置 FP16 精度与用户输入精度不对应的 Blob

OpenVINO failed to set Blob with FP16 precision not corresponding to user input precision

python

openvino