自适应阈值的 OpenCV 输出

Question

我是android的新手，都打开了简历。但是，我正在尝试从相机拍摄图像，将其转换为所需的格式，然后将其传递给 tflite 模型。

捕获图像并对其应用图像处理的代码。

    public Mat onCameraFrame(CameraBridgeViewBase.CvCameraViewFrame inputFrame) {
        float mh = mRGBA.height();
        float cw = (float) Resources.getSystem().getDisplayMetrics().widthPixels;
        float scale = mh / cw * 0.7f;

        mRGBA = inputFrame.rgba();
        frame = classifier.processMat(mRGBA);

        Mat temp = new Mat();
        Mat temp3= new Mat();

        if (!isDebug) {
            if (counter == CLASSIFY_INTERVAL) {
                Imgproc.cvtColor(frame, frame, Imgproc.COLOR_RGBA2GRAY);
                Core.rotate(frame, frame, Core.ROTATE_90_CLOCKWISE);
                Imgproc.GaussianBlur(frame, frame, new Size(5, 5), 0);

                Imgproc.adaptiveThreshold(frame, frame, 255, Imgproc.ADAPTIVE_THRESH_GAUSSIAN_C, Imgproc.THRESH_BINARY_INV , 3, 2);

                Bitmap bmsp = null;

                runInterpreter();
                counter = 0;
            } else {
                counter++;
            }
        }

        Imgproc.rectangle(mRGBA,
                new Point(mRGBA.cols() / 2f - (mRGBA.cols() * scale / 2),
                        mRGBA.rows() / 2f - (mRGBA.cols() * scale / 2)),
                new Point(mRGBA.cols() / 2f + (mRGBA.cols() * scale / 2),
                        mRGBA.rows() / 2f + (mRGBA.cols() * scale / 2)),
                new Scalar(0, 255, 0), 1);
        if (isEdge) {
            mRGBA = classifier.debugMat(mRGBA);
        }

        System.gc();
        return mRGBA;
    }

我的输出看起来像这张图片，但我希望手在传递给模型之前用白色填充。有人可以建议吗？

Answer 1

主要问题是 adaptiveThreshold 的结果在外边缘有间隙，因此您不能将其用作 findContours 的输入。

我认为使用 GaussianBlur 会使事情变得更糟，因为它模糊了手和背景之间的边缘。

您可以使用以下阶段：

将帧转换为灰度。
应用 adaptiveThreshold 大内核大小（我使用大小 51）。
使用大的内核大小，保持粗的边缘线没有间隙（除了指甲上的小间隙）。
寻找等高线。
找到面积最大的轮廓。
在零图像上绘制轮廓（填充实心值 255）。
有一个问题：由于轮廓形状怪异，手的内部没有填充。
完成填写：
找到轮廓的中心，并使用 floodFill.

这是一个 Python 代码示例：

import numpy as np
import cv2

frame = cv2.imread("hand.jpg")  # Read image from file (for testing).

gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)  # Use BGR to Gray conversion (not RGBA, because image is read from file)

# Apply adaptiveThreshold with large filter size.
thres_gray = cv2.adaptiveThreshold(gray, 255, cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY_INV, 51, 2)

# Find contours (external contours)
cnts, hier = cv2.findContours(thres_gray, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)

# Find contour with the maximum area
c = max(cnts, key=cv2.contourArea)

res = np.zeros_like(gray)  # Create new zeros images for storing the result.

# Fill the contour with white color - draw the filled contour on res image.
cv2.drawContours(res, [c], -1, 255, -1)

# Compute the center of the contour
# https://www.pyimagesearch.com/2016/02/01/opencv-center-of-contour/
M = cv2.moments(c)
cX = int(M["m10"] / M["m00"])
cY = int(M["m01"] / M["m00"])

# Use floodFill for filling the center of the contour
cv2.floodFill(res, None, (cX, cY), 255)

# Show images for testing
cv2.imshow('thres_gray', thres_gray)
cv2.imshow('res', res)
cv2.waitKey()
cv2.destroyAllWindows()

结果：

thres_gray:

res 之前 floodFill:

res 在 floodFill 之后：

JAVA 实施：

package myproject;

import org.opencv.core.Core;
import org.opencv.core.Mat;
import org.opencv.core.CvType;
import org.opencv.core.Scalar;
import org.opencv.core.Point;
import org.opencv.core.MatOfPoint;
import org.opencv.imgproc.Imgproc;
import org.opencv.imgproc.Moments;
import org.opencv.imgcodecs.Imgcodecs;
import java.util.List;
import java.util.ArrayList;

class Sample {
    

static { System.loadLibrary(Core.NATIVE_LIBRARY_NAME); }


  
public static void main(String[] args) {
    Mat frame = Imgcodecs.imread("hand.jpg");
    
    Mat gray = new Mat();
    Mat thres_gray = new Mat();
    
    Imgproc.cvtColor(frame, gray, Imgproc.COLOR_BGR2GRAY);
    
    //Apply adaptiveThreshold with large filter size.
    Imgproc.adaptiveThreshold(gray, thres_gray, 255, Imgproc.ADAPTIVE_THRESH_GAUSSIAN_C, Imgproc.THRESH_BINARY_INV, 51, 2);    
      
    List<MatOfPoint> contours = new ArrayList<>();
    Mat hierarchy = new Mat();
    
    //Find contours
    Imgproc.findContours(thres_gray, contours, hierarchy, Imgproc.RETR_EXTERNAL, Imgproc.CHAIN_APPROX_NONE);

    //Find contour with the maximum area
    //
    double maxVal = 0;
    int maxValIdx = 0;
    for (int contourIdx = 0; contourIdx < contours.size(); contourIdx++)
    {
        double contourArea = Imgproc.contourArea(contours.get(contourIdx));
        if (maxVal < contourArea)
        {
            maxVal = contourArea;
            maxValIdx = contourIdx;
        }
    }
       
    Mat res = Mat.zeros(gray.size(), CvType.CV_8UC1);    //Create new zeros images for storing the result.
    
    Imgproc.drawContours(res, contours, maxValIdx, new Scalar(255), -1);
    
    //Compute the center of the contour
    //https://www.pyimagesearch.com/2016/02/01/opencv-center-of-contour/
    Moments M = Imgproc.moments(contours.get(maxValIdx));
    int cX = (int)(M.get_m10() / M.get_m00());
    int cY = (int)(M.get_m01() / M.get_m00());    

    //Use floodFill for filling the center of the contour.    
    Mat mask = Mat.zeros(res.rows() + 2, res.cols() + 2, CvType.CV_8UC1);
    Imgproc.floodFill(res, mask, new Point(cX, cY), new Scalar(255));
       
    Imgcodecs.imwrite("res.png", res);
}

}

自适应阈值的 OpenCV 输出

OpenCV output of Adaptive Threshold

python

android

opencv

image-thresholding

adaptive-threshold