如何处理 org.tensorflow.lite.Interpreter.runForMultipleInputsOutputs() 的结果

Question

我在 android 上使用 tflite 运行 posenet（这是一个 CNN）。该模型具有多个具有以下维度的输出数组： 1x14x14x17, 1x14x14x34, 1x14x14x32, 1x14x14x32

因此运行 java tflite 解释器

import org.tensorflow.lite.Interpreter;
Interpreter tflite;
...
tflite.runForMultipleInputsOutputs(inputs,outputs)

i 可以使用 tflite.getOutputTensor(i) 或 outputs.get(i)（i el. [0,3]）访问四个输出张量，因为 outputs 是 HashMap 填充有 java.nio.HeapByteBuffer 个对象。

如何将这些输出或 tflite 张量转换为 java 多维数组（类似于 float[][][][];）以便能够对它们执行数学计算？

Answer 1

像下面这样定义输出可以让您使用本机 java 数组，这正是我想要的：

out1 = new float[1][14][14][17];
out2 = new float[1][14][14][34];
out3 = new float[1][14][14][32];
out4 = new float[1][14][14][32];
Map<Integer, Object> outputs = new HashMap<>();
outputs.put(0, out1);
outputs.put(1, out2);
outputs.put(2, out3);
outputs.put(3, out4);

Answer 2

// The shape of *1* output's tensor
int[] OutputShape;
// The type of the *1* output's tensor
DataType OutputDataType;
// The multi-tensor ready storage
outputProbabilityBuffers = new HashMap<>();

ByteBuffer x;
// For each model's tensors (there are getOutputTensorCount() of them for this tflite model)
for (int i = 0; i < tflite.getOutputTensorCount(); i++) {
    OutputShape = tflite.getOutputTensor(i).shape();
    OutputDataType = tflite.getOutputTensor(i).dataType();
    x = TensorBuffer.createFixedSize(OutputShape, OutputDataType).getBuffer();
    outputProbabilityBuffers.put(i, x);
    LOGGER.d("Created a buffer of %d bytes for tensor %d.", x.limit(), i);
}

LOGGER.d("Created a tflite output of %d output tensors.", outputProbabilityBuffers.size());

示例输出：

Classifier: Created a buffer of 11264 bytes for tensor 0.
Classifier: Created a buffer of 11264 bytes for tensor 1.
Classifier: Created a buffer of 4 bytes for tensor 2.
Classifier: Created a buffer of 11264 bytes for tensor 3.
Classifier: Created a tflite output of 4 output tensors.

然后这样使用它：

Object[] inputs = { your_regular_input };
tflite.runForMultipleInputsOutputs(inputs, outputProbabilityBuffers);

Answer 3

输出：https://www.tensorflow.org/lite/models/object_detection/overview#output

val locations = outputs.getValue(0).asFlowArray(),
val classes = outputs.getValue(1).asFlowArray(),
val scores = outputs.getValue(2).asFlowArray(),
val detections = outputs.getValue(3).asFlowArray()

如何处理 org.tensorflow.lite.Interpreter.runForMultipleInputsOutputs() 的结果

How to deal with with result of org.tensorflow.lite.Interpreter.runForMultipleInputsOutputs()

java

arrays

android

tensorflow

tensorflow-lite