如何在来自 FastAPI 的 React 上渲染 Streamable 图像

Question

我想在 React 上从 FastAPI 渲染 StreamingResponse 图像。图片是numpy数组的形式，是cv2类型的object

@app.post("/predict")
async def root(file: UploadFile = File(...)):
    global model
    global store_coordinates
    global store_faces
    global store_mesh

    content = await file.read()
    nparr = np.fromstring(content, np.uint8)
    bg_img = cv2.imdecode(nparr, cv2.IMREAD_COLOR)
    ......................
    for i in range(len(store_coordinates)):
            x, y, w, h = store_coordinates[i]
            bg_img [b:b + d, a:a + c] = store_mesh[i]
    
    res,im_png = cv2.imencode(".png", bg_img)
    return StreamingResponse(io.BytesIO(im_png.tobytes()), media_type="image/png")

我在这里创建了一个 API 端点，其中使用 post 请求接收上传的图像并返回 StreamableResponse(Image)。我如何在 React 前端渲染这个返回的响应

反应代码：

import React, { Component } from "react";
import axios from "axios";

class Detect extends Component {
  state = {
    title: "",
    content: "",
    image: null,
  };

  handleChange = (e) => {
    this.setState({
      [e.target.id]: e.target.value,
    });
  };

  handleImageChange = (e) => {
    this.setState({
      image: e.target.files[0],
    });
  };

  
  handleSubmit = (e) => {
    e.preventDefault();
    console.log(this.state);
    let form_data = new FormData();
    form_data.append("image", this.state.image, this.state.image.name);
    let url = "http://127.0.0.1:8000/predict";
    axios
      .post(url, form_data, {
        headers: {
          "content-type": "multipart/form-data",
        },
      })
      .then((res) => {
        console.log(res.data);
      })
      .catch((err) => console.log(err));
  };

  render() {
    return (
      <div className="App">
        <form onSubmit={this.handleSubmit}>
          <p>
            <input
              type="file"
              id="image"
              accept="image/png, image/jpeg"
              onChange={this.handleImageChange}
              required
            />
          </p>
          <input type="submit" />
        </form>
        <div id="image-render">
          <img></img>
        </div>
      </div>
    );
  }
}

export default Detect;

我想在 ID 为 rendered-image

的 div 标签中渲染返回的图像

编辑 -

Answer 1

您可以将图像数据编码为 base64 格式，然后 return 它可以用于显示 HTML 中的图像，如图 here (e.g., <img src="data:image/png;base64, ...), or send the raw bytes, as you already do, and convert them into base64 format (using btoa(), String.fromCharCode() and Uint8Array) or Blob object (and then call URL.createObjectURL() to convert the blob into a URL.) on client side. The below examples show how to achieve the last two methods, using Axios library, which you seem to be using in your project, as well as Fetch API.

使用 Axios

选项 1 - 将原始图像字节转换为 Blob

axios({
        method: 'POST',
        url: '/upload',
        data: formData,
        headers: {
            'Content-Type': 'multipart/form-data'
        },
        responseType: "blob"
    })
    .then(response => {
        var blobURL = URL.createObjectURL(response.data);
        var image = document.getElementById("myImage");
        image.onload = function(){
            URL.revokeObjectURL(this.src); // release the blob URL once the image is loaded
        }
        image.src = blobURL;
    })
    .catch(error => {
        console.error(error);
    });

选项 2 - 将原始图像字节转换为 Base64 编码字符串

axios({
        method: 'POST',
        url: '/predict',
        data: formData,
        headers: {
            'Content-Type': 'multipart/form-data'
        },
        responseType: "arraybuffer"
    })
    .then(response => {
        base64string = btoa(String.fromCharCode(...new Uint8Array(response.data)))
        contentType = response.headers['content-type']
        return base64string;
    })
    .then(base64string => {
        var image = document.getElementById("myImage");
        image.src = "data:" + contentType + ";base64," + base64string;
    })
    .catch(error => {
        console.error(error);
    });

使用 Fetch API

选项 1 - 将原始图像字节转换为 Blob

fetch('/predict', {
        method: 'POST',
        body: formData,
    })
    .then(response => response.blob())
    .then(blob => {
        var blobURL = URL.createObjectURL(blob);
        var image = document.getElementById("myImage");
        image.onload = function(){
            URL.revokeObjectURL(this.src); // release the blob URL once the image is loaded
        }
        image.src = blobURL;
    })
    .catch(error => {
        console.error(error);
    });

选项 2 - 将原始图像字节转换为 Base64 编码字符串

fetch('/predict', {
        method: 'POST',
        body: formData,
    })
    .then(response => {
        contentType = response.headers.get('content-type')
        return response.arrayBuffer();
    })
    .then(arrayBuffer => {
        base64string = btoa(String.fromCharCode(...new Uint8Array(arrayBuffer)))
        var image = document.getElementById("myImage");
        image.src = "data:" + contentType + ";base64," + base64string;
    })
    .catch(error => {
        console.error(error);
    });

记住在你的 HTML 文件中定义一个 <img> 标签，你希望在其中显示图像：

 <img id="myImage" src="">

如何在来自 FastAPI 的 React 上渲染 Streamable 图像

How to render Streamable image on React coming from the FastAPI

image

reactjs

fastapi

使用 Axios

选项 1 - 将原始图像字节转换为 Blob

选项 2 - 将原始图像字节转换为 Base64 编码字符串

使用 Fetch API

选项 1 - 将原始图像字节转换为 Blob

选项 2 - 将原始图像字节转换为 Base64 编码字符串