如何使用英特尔推理引擎 (OpenVINO) 优化带有 batchnorm 层的 keras 模型？

Question

使用英特尔推理引擎（OpenVINO 工具包 R.5）优化 keras 模型失败

我按照 tutorial 的建议冻结我的模型。 keras 模型经过训练和测试。我需要优化它以进行推理。但是，在自定义模型上运行模型优化器（mo.py 脚本）时出现错误。

[ ERROR ] shapes (128,9) and (0,) not aligned: 9 (dim 1) != 0 (dim 0)

我模型的最后几层（9 是类的输出数）是：

conv2d_4 (Conv2D) (None, 4, 4, 128) 204928 batch_normalization_3[0][0]
__________________________________________________________________________________________________
activation_4 (Activation) (None, 4, 4, 128) 0 conv2d_4[0][0]
__________________________________________________________________________________________________
batch_normalization_4 (BatchNor (None, 4, 4, 128) 512 activation_4[0][0]
__________________________________________________________________________________________________
average_pooling2d_2 (AveragePoo (None, 1, 1, 128) 0 batch_normalization_4[0][0]
__________________________________________________________________________________________________
dropout_2 (Dropout) (None, 1, 1, 128) 0 average_pooling2d_2[0][0]
__________________________________________________________________________________________________
flatten (Flatten) (None, 128) 0 dropout_2[0][0]
__________________________________________________________________________________________________
dense (Dense) (None, 128) 16512 flatten[0][0]
__________________________________________________________________________________________________
activation_5 (Activation) (None, 128) 0 dense[0][0]
__________________________________________________________________________________________________
batch_normalization_5 (BatchNor (None, 128) 512 activation_5[0][0]
__________________________________________________________________________________________________
dropout_3 (Dropout) (None, 128) 0 batch_normalization_5[0][0]
__________________________________________________________________________________________________
dense_1 (Dense) (None, 9) 1161 dropout_3[0][0]
__________________________________________________________________________________________________
color_prediction (Activation) (None, 9) 0 dense_1[0][0]
__________________________________________________________________________________________________

由于存在 BatchNormalization 层，模型优化器失败。当我删除它们时，它会成功运行。但是我用

冻结图表

tf.keras.backend.set_learning_phase(0)

因此必须在冻结图中删除 BatchNormalization 和 Dropout 等节点，我不明白为什么不删除它们。

非常感谢！

Answer 1

我设法运行在具有批量标准化层的 Keras 模型上使用 OpenVINO 模型优化器。该模型似乎也收敛得更快了。尽管测试分类率比没有 BN 的模型低约 5-7%（并且测试和训练数据集的分类率之间的差距更大）。我不确定 BatchNormalization 是否已从我的解决方案中的模型中正确删除（但 openVINO 模型文件不包含一个，因此已将其删除）。

移除 BN 和 Dropout 层：

#Clear any previous session.
tf.keras.backend.clear_session()
#This line must be executed before loading Keras model.
tf.keras.backend.set_learning_phase(0) 
model = tf.keras.models.load_model(weights_path)

for layer in model.layers:
    layer.training = False
    if isinstance(layer, tf.keras.layers.BatchNormalization):
        layer._per_input_updates = {}
    elif isinstance(layer, tf.keras.layers.Dropout):
        layer._per_input_updates = {}

然后冻结会话：

def freeze_session(session, keep_var_names=None, output_names=None, clear_devices=True):
"""
Freezes the state of a session into a pruned computation graph.

Creates a new computation graph where variable nodes are replaced by
constants taking their current value in the session. The new graph will be
pruned so subgraphs that are not necessary to compute the requested
outputs are removed.
@param session The TensorFlow session to be frozen.
@param keep_var_names A list of variable names that should not be frozen,
                    or None to freeze all the variables in the graph.
@param output_names Names of the relevant graph outputs.
@param clear_devices Remove the device directives from the graph for better portability.
@return The frozen graph definition.
"""
from tensorflow.python.framework.graph_util import convert_variables_to_constants
graph = session.graph
with graph.as_default():
    freeze_var_names = list(set(v.op.name for v in tf.global_variables()).difference(keep_var_names or []))
    output_names = output_names or []
    output_names += [v.op.name for v in tf.global_variables()]
    # Graph -> GraphDef ProtoBuf
    input_graph_def = graph.as_graph_def()
    if clear_devices:
        for node in input_graph_def.node:
            node.device = ""
    frozen_graph = convert_variables_to_constants(session, input_graph_def,
                                                output_names, freeze_var_names)
    return frozen_graph

如何使用英特尔推理引擎 (OpenVINO) 优化带有 batchnorm 层的 keras 模型？

How to optimize keras model with batchnorm layers with Intel inference engine (OpenVINO)?

inference-engine

keras

tensorflow

openvino