从 ONNX.js 的图像创建张量

Question

我正在尝试在浏览器中使用 ONNX.js 到运行 ONNX 对象检测模型。我知道在 Tensorflow.js 中你只需将一个图像对象传递给模型，Tensorflow 会自动创建模型所需的张量，但在 ONNX 中我们必须从图像创建张量，然后将其传递给模型。

我阅读了官方存储库 - https://github.com/Microsoft/onnxjs，但我在 Javascript.

中找不到任何关于如何从图像创建张量的资源

如果有人知道什么请帮助？

我正在使用以下代码运行使用 ONNX.js

在浏览器中进行对象检测

<html>

<head> </head>

<body>
    <div onclick="runOD()">
        Run Model
    </div>
    <img src="box.jpg" id="image">

    <!-- Load ONNX.js -->
    <script src="https://cdn.jsdelivr.net/npm/onnxjs/dist/onnx.min.js"></script>
    <!-- Code that consume ONNX.js -->
    <script>
        async function runOD() {
            // Creat the session and load the pre-trained model
            const session = new onnx.InferenceSession({
                backendHint: 'webgl'
            });

            await session.loadModel("./model.onnx");

            //Function to load image to Canvas and Convert it to Tensor
            const preprocessedData = loadImage();

            const inputTensor = new onnx.Tensor(preprocessedData, 'float32', [1, 3, 244, 244]);

            // Run model with Tensor inputs and get the result.
            const outputMap = await session.run([inputTensor]);
            const outputData = outputMap.values().next().value.data;
            console.log(`model output tensor: ${outputData.data}.`);
        }
    </script>
</body>

</html>

Answer 1

以下代码对我有用

<script>
const imageSize = 416;
var count = 0;

async function runOD() {
    // Creat the session and load the pre-trained model
    const session = new onnx.InferenceSession({
        backendHint: 'webgl'
    });

    await session.loadModel("./model.onnx");

    // Run model with Tensor inputs and get the result.

        // Load image.
        const imageLoader = new ImageLoader(imageSize, imageSize);
        const imageData = await imageLoader.getImageData('./box.jpg');

        // Preprocess the image data to match input dimension requirement
        const width = imageSize;
        const height = imageSize;
        const preprocessedData = preprocess(imageData.data, width, height);

        const inputTensor = new onnx.Tensor(preprocessedData, 'float32', [1, 3, width, height]);
        const outputMap = await session.run([inputTensor]);
        const outputData = outputMap.values().next().value.data;
        var x = outputData.toString();
        var blob = new Blob([x], { type: "text/plain;charset=utf-8" });
        saveAs(blob, "data.txt");

        console.log(outputData);
        count = count+1;
        document.getElementById('count').innerHTML = count;
}
/**
 * Preprocess raw image data to match Resnet50 requirement.
 */
function preprocess(data, width, height) {
    const dataFromImage = ndarray(new Float32Array(data), [width, height, 4]);
    const dataProcessed = ndarray(new Float32Array(width * height * 3), [1, 3, height, width]);

    // Normalize 0-255 to (-1)-1
    ndarray.ops.divseq(dataFromImage, 128.0);
    ndarray.ops.subseq(dataFromImage, 1.0);

    // Realign imageData from [224*224*4] to the correct dimension [1*3*224*224].
    ndarray.ops.assign(dataProcessed.pick(0, 0, null, null), dataFromImage.pick(null, null, 2));
    ndarray.ops.assign(dataProcessed.pick(0, 1, null, null), dataFromImage.pick(null, null, 1));
    ndarray.ops.assign(dataProcessed.pick(0, 2, null, null), dataFromImage.pick(null, null, 0));

    return dataProcessed.data;
}

从 ONNX.js 的图像创建张量

Create Tensors from Image for ONNX.js

javascript

onnx