Error: pixels passed to tf.browser.fromPixels() must be either an HTMLVideoElement

Question

我正在使用 TensorFlow JS 使用 Coco-SSD 模型进行对象检测。但我的图像采用 base64 字符串格式，与 tf.browser.fromPixels() 不兼容。我收到此错误

Error: pixels passed to tf.browser.fromPixels() must be either an HTMLVideoElement, HTMLImageElement, HTMLCanvasElement, ImageData in browser, or OffscreenCanvas, ImageData in webworker or {data: Uint32Array, width: number, height: number}, but was Function

refreshImage(){
    this.httpClient.get('http://localhost:8080/image/get/test.jpg')
          .subscribe(
            res => {
              this.retrieveResonse = res;
              this.base64Data = this.retrieveResonse.picByte;
              console.log(this.base64Data = this.retrieveResonse.picByte)
              this.retrievedImage = 'data:image/jpeg;base64,' + this.base64Data;
            }
          );

          this.predictWithCocoModel();
  }


  public async predictWithCocoModel(){
    const model = await cocoSSD.load();
    this.detectFrame(this.retrievedImage,model);
    console.log('model loaded');
  }
  detectFrame = (retrievedImage, model) => {
    model.detect(this.refreshImage).then(predictions => {
      this.renderPredictions(predictions);
      /*requestAnimationFrame(() => {
        this.detectFrame(video, model);
      });*/
    });
  }

  renderPredictions = predictions => {
    const canvas = <HTMLCanvasElement> document.getElementById("canvas");

    const ctx = canvas.getContext("2d");

    canvas.width  = 360;
    canvas.height = 360;

    ctx.clearRect(0, 0, ctx.canvas.width, ctx.canvas.height);
    // Font options.
    const font = "16px sans-serif";
    ctx.font = font;
    ctx.textBaseline = "top";
    ctx.drawImage(this.retrievedImage,0, 0,300,300);

    predictions.forEach(prediction => {
      let i = 0
      const x = prediction.bbox[0];
      const y = prediction.bbox[1];
      const width = prediction.bbox[2];
      const height = prediction.bbox[3];
      if (prediction.class === 'person') {
        i++
      }
      console.log(i);
      // Draw the bounding box.
      ctx.strokeStyle = "#00FFFF";
      ctx.lineWidth = 2;
      ctx.strokeRect(x, y, width, height);
      // Draw the label background.
      ctx.fillStyle = "#00FFFF";
      const textWidth = ctx.measureText(prediction.class).width;
      const textHeight = parseInt(font, 10); // base 10
      ctx.fillRect(x, y, textWidth + 4, textHeight + 4);
    });

    predictions.forEach(prediction => {
      const x = prediction.bbox[0];
      const y = prediction.bbox[1];
      // Draw the text last to ensure it's on top.
      ctx.fillStyle = "#000000";
      ctx.fillText(prediction.class, x, y);
    });
  };

Answer 1

model.detect 预计错误指示图像元素。

您正在向 detect 方法传递一个导致错误

的函数 refreshImage