将 tensorflow 转换为统一后端 (barracuda) 的问题

Question

我正在尝试将我使用 Keras 创建的 UNet 模型转换为 .nn 以用于 unity 的神经网络后端。但是我收到了这个错误。对于我的模型导出，我导出了一个“.h5”，我将其转换为二进制“.pb”，后来我使用了 tensorflow_to_barracuda.py。有没有人有统一的工作分割程序？

Converting unet_person.bytes to unet_person.nn
IGNORED: PlaceholderWithDefault unknown layer
IGNORED: Switch unknown layer
IGNORED: Switch unknown layer
IGNORED: Shape unknown layer
IGNORED: Switch unknown layer
IGNORED: Merge unknown layer
IGNORED: Shape unknown layer
IGNORED: Shape unknown layer
---------------------------------------------------------------------------
UnboundLocalError                         Traceback (most recent call last)
<ipython-input-22-d09d8c6d2c1a> in <module>
      1 from mlagents.trainers import tensorflow_to_barracuda as tb
      2 
----> 3 tb.convert('unet_person.bytes', 'unet_person.nn')

/anaconda3/lib/python3.6/site-packages/mlagents/trainers/tensorflow_to_barracuda.py in convert(source_file, target_file, trim_unused_by_output, verbose, compress_f16)
938     o_model = barracuda.Model()
939     o_model.layers, o_input_shapes, o_model.tensors, o_model.memories = \
--> 940         process_model(i_model, args)
941 
942     # Cleanup unconnected Identities (they might linger after processing complex node patterns like LSTM)

/anaconda3/lib/python3.6/site-packages/mlagents/trainers/tensorflow_to_barracuda.py in process_model(model, args)
870                 nodes = nodes_as_array[node_index:pattern_end]
871                 name = nodes[-1].name
--> 872                 var_tensors, const_tensors = get_tensors(nodes)
873                 if args.print_patterns or args.verbose:
874                     print('PATTERN:', name, '~~', pattern_name, pattern, '<-', var_tensors, '+', [t.name for t in const_tensors])

/anaconda3/lib/python3.6/site-packages/mlagents/trainers/tensorflow_to_barracuda.py in get_tensors(pattern_nodes)
845                 tensor_nodes = [n for n in pattern_nodes if n.op == 'Const']
846                 tensors = [Struct(name = n.name, obj = n.attr["value"].tensor, shape = get_tensor_dims(n.attr["value"].tensor), data = get_tensor_data(n.attr["value"].tensor))
--> 847                     for n in tensor_nodes]
848 
849                 # TODO: unify / reuse code from process_layer

/anaconda3/lib/python3.6/site-packages/mlagents/trainers/tensorflow_to_barracuda.py in <listcomp>(.0)
845                 tensor_nodes = [n for n in pattern_nodes if n.op == 'Const']
846                 tensors = [Struct(name = n.name, obj = n.attr["value"].tensor, shape = get_tensor_dims(n.attr["value"].tensor), data = get_tensor_data(n.attr["value"].tensor))
--> 847                     for n in tensor_nodes]
848 
849                 # TODO: unify / reuse code from process_layer

/anaconda3/lib/python3.6/site-packages/mlagents/trainers/tensorflow_to_barracuda.py in get_tensor_data(tensor)
492     if tensor.bool_val:
493         data = np.array(tensor.bool_val, dtype=float)
--> 494     return np.array(data).reshape(dims)
495 
496 def flatten(items,enter=lambda x:isinstance(x, list)):

UnboundLocalError: local variable 'data' referenced before assignment

Answer 1

我发现这个框架还不够完善。对我有用的是为所有平台编译 Tensorflow Lite 源代码并使用该后端。转换为 Tensorflow Lite 仍然有点棘手，因为只支持某些层。最后，您需要将 C 二进制文件包装在 C# 中，这已在此处部分完成：https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/experimental/examples/unity

编译比较容易

Answer 2

在 Barracuda 1.0 中，有一种方法可以使用 Keras2ONNX pip 包将 Keras (.h5) 模型转换为 ONNX 模型。

你安装keras2ONNX然后运行

import keras2onnx
onnx_model = keras2onnx.convert_keras(unet, name='unet')
keras2onnx.save_model(onnx_model, "unet.onnx")

请注意，您需要以下标志：channel_first_inputs=[unet.layers[0].layers[0]]

onnx_model = keras2onnx.convert_keras(unet, name='unet')

由于 Barracuda 输入是通道优先，也就是说对于 batch_size x 宽 x 高 x rgb 图像，顺序是 rgb x 宽 x 高 x batch_size。

将 tensorflow 转换为统一后端 (barracuda) 的问题

Issue converting tensorflow into unity backend (barracuda)

unity3d

tensorflow

machine-learning

barracuda