我可以用 flutter 相机包制作流媒体吗？

Question

我的目的是流式传输到 websocket。

首先我需要相机数据。我为此使用了 startImageStream 函数。此 func return 图像为 yuv 格式。我找到了一些关于 yuv 到图像小部件的东西，但我无法将它转换为 base 64。

问题是如何将此 yuv 图像数据转换为 base64？或者这是用 flutter 制作流的糟糕方法。

Answer 1

controller.startImageStream((image) {
  // convert image here

  // assuming it's converted
  final imageBytes = <int>[];

  final encodedImage = base64.encode(imageBytes);
});

至于 yuv->rgb 转换，你可以这样做（代码复制自 https://gist.github.com/Alby-o/fe87e35bc21d534c8220aed7df028e03）：

// imgLib -> Image package from https://pub.dartlang.org/packages/image
import 'package:image/image.dart' as imglib;
import 'package:camera/camera.dart';

Future<List<int>> convertImagetoPng(CameraImage image) async {
  try {
    imglib.Image img;
    if (image.format.group == ImageFormatGroup.yuv420) {
      img = _convertYUV420(image);
    } else if (image.format.group == ImageFormatGroup.bgra8888) {
      img = _convertBGRA8888(image);
    }

    imglib.PngEncoder pngEncoder = new imglib.PngEncoder();

    // Convert to png
    List<int> png = pngEncoder.encodeImage(img);
    return png;
  } catch (e) {
    print(">>>>>>>>>>>> ERROR:" + e.toString());
  }
  return null;
}

// CameraImage BGRA8888 -> PNG
// Color
imglib.Image _convertBGRA8888(CameraImage image) {
  return imglib.Image.fromBytes(
    image.width,
    image.height,
    image.planes[0].bytes,
    format: imglib.Format.bgra,
  );
}

// CameraImage YUV420_888 -> PNG -> Image (compresion:0, filter: none)
// Black
imglib.Image _convertYUV420(CameraImage image) {
  var img = imglib.Image(image.width, image.height); // Create Image buffer

  Plane plane = image.planes[0];
  const int shift = (0xFF << 24);

  // Fill image buffer with plane[0] from YUV420_888
  for (int x = 0; x < image.width; x++) {
    for (int planeOffset = 0;
        planeOffset < image.height * image.width;
        planeOffset += image.width) {
      final pixelColor = plane.bytes[planeOffset + x];
      // color: 0x FF  FF  FF  FF
      //           A   B   G   R
      // Calculate pixel color
      var newVal = shift | (pixelColor << 16) | (pixelColor << 8) | pixelColor;

      img.data[planeOffset + x] = newVal;
    }
  }

  return img;
}

一些背景：

https://github.com/flutter/flutter/issues/26348

在后者中，您可以找到链接到似乎更有效实施的评论，它在 YUV_2_RGB project: https://github.com/alexcohn/YUV_2_RGB/blob/master/lib/service/image_result_processor_service.dart

中

Answer 2

我解决了。这是我在互联网上找到的转换函数的答案：

imglib.Image img;


      if (Platform.isAndroid) {
        // Allocate memory for the 3 planes of the image
        Pointer<Uint8> p =
            allocate(count: _savedImage.planes[0].bytes.length);
        Pointer<Uint8> p1 =
            allocate(count: _savedImage.planes[1].bytes.length);
        Pointer<Uint8> p2 =
            allocate(count: _savedImage.planes[2].bytes.length);

        // Assign the planes data to the pointers of the image
        Uint8List pointerList =
            p.asTypedList(_savedImage.planes[0].bytes.length);
        Uint8List pointerList1 =
            p1.asTypedList(_savedImage.planes[1].bytes.length);
        Uint8List pointerList2 =
            p2.asTypedList(_savedImage.planes[2].bytes.length);
        pointerList.setRange(0, _savedImage.planes[0].bytes.length,
            _savedImage.planes[0].bytes);
        pointerList1.setRange(0, _savedImage.planes[1].bytes.length,
            _savedImage.planes[1].bytes);
        pointerList2.setRange(0, _savedImage.planes[2].bytes.length,
            _savedImage.planes[2].bytes);

        // Call the convertImage function and convert the YUV to RGB
        Pointer<Uint32> imgP = conv(
            p,
            p1,
            p2,
            _savedImage.planes[1].bytesPerRow,
            _savedImage.planes[1].bytesPerPixel,
            _savedImage.planes[0].bytesPerRow,
            _savedImage.height);

        // Get the pointer of the data returned from the function to a List
        List imgData = imgP.asTypedList(
            (_savedImage.planes[0].bytesPerRow * _savedImage.height));
        // Generate image from the converted data
        img = imglib.Image.fromBytes(_savedImage.height, _savedImage.planes[0].bytesPerRow, imgData);

        // Free the memory space allocated
        // from the planes and the converted data
        free(p);
        free(p1);
        free(p2);
        free(imgP);
      }
      else if (Platform.isIOS) {
        img = imglib.Image.fromBytes(
          _savedImage.planes[0].bytesPerRow,
          _savedImage.height,
          _savedImage.planes[0].bytes,
          format: imglib.Format.bgra,
        );
      }

在这段代码给你一个 imglib.Image 对象之后，我用那个函数把这张图片转换成 jpg :

      List<int> a = imglib.encodeJpg(img, quality: 100);
      var resp = await sendImage(base64.encode(a));
      print(resp.body);

我将其发送到 api 以查看图像并且有效..

没有编码，我无法转换为 base64，所以重点是：

List a = imglib.encodeJpg(img, quality: 100);

我可以用 flutter 相机包制作流媒体吗？

Can i make stream with flutter camera package?

android-camera

flutter