在嘈杂的二值图像中检测不同的形状
Detect different shapes in noisy binary image
我想检测这张图片中的圆和五个正方形：
这是我目前使用的代码的相关部分：
# detect shapes in black-white RGB formatted cv2 image
def detect_shapes(img, approx_poly_accuracy=APPROX_POLY_ACCURACY):
res_dict = {
"rectangles": [],
"squares": []
}
vis = img.copy()
shape = img.shape
height, width = shape[0], shape[1]
total_area = height * width
# Morphological closing: get rid of holes
# img = cv2.morphologyEx(img, cv2.MORPH_CLOSE, cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (5, 5)))
# Morphological opening: get rid of extensions at the border of the objects
# img = cv2.morphologyEx(img, cv2.MORPH_OPEN, cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (121, 121)))
img = cv2.cvtColor(img, cv2.COLOR_RGB2GRAY)
# cv2.imshow('intermediate', img)
# cv2.waitKey(0)
contours, hierarchy = cv2.findContours(img, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
logging.info("Number of found contours for shape detection: {0}".format(len(contours)))
# vis = img.copy()
# cv2.drawContours(vis, contours, -1, (0, 255, 0), 2)
cv2.imshow('vis', vis)
cv2.waitKey(0)
for contour in contours:
area = cv2.contourArea(contour)
if area < MIN_SHAPE_AREA:
logging.warning("Area too small: {0}. Skipping.".format(area))
continue
if area > MAX_SHAPE_AREA_RATIO * total_area:
logging.warning("Area ratio too big: {0}. Skipping.".format(area / total_area))
continue
approx = cv2.approxPolyDP(contour, approx_poly_accuracy * cv2.arcLength(contour, True), True)
cv2.drawContours(vis, [approx], -1, (0, 0, 255), 2)
la = len(approx)
# find the center of the shape
M = cv2.moments(contour)
if M['m00'] == 0.0:
logging.warning("Unable to compute shape center! Skipping.")
continue
x = int(M['m10'] / M['m00'])
y = int(M['m01'] / M['m00'])
if la < 3:
logging.warning("Invalid shape detected! Skipping.")
continue
if la == 3:
logging.info("Triangle detected at position {0}".format((x, y)))
elif la == 4:
logging.info("Quadrilateral detected at position {0}".format((x, y)))
if approx.shape != (4, 1, 2):
raise ValueError("Invalid shape before reshape to (4, 2): {0}".format(approx.shape))
approx = approx.reshape(4, 2)
r_check, data = check_rect_or_square(approx)
blob_data = {"position": (x, y), "approx": approx}
blob_data.update(data)
if r_check == 2:
res_dict["squares"].append(blob_data)
elif r_check == 1:
res_dict["rectangles"].append(blob_data)
elif la == 5:
logging.info("Pentagon detected at position {0}".format((x, y)))
elif la == 6:
logging.info("Hexagon detected at position {0}".format((x, y)))
else:
logging.info("Circle, ellipse or arbitrary shape detected at position {0}".format((x, y)))
cv2.drawContours(vis, [contour], -1, (0, 255, 0), 2)
cv2.imshow('vis', vis)
cv2.waitKey(0)
logging.info("res_dict: {0}".format(res_dict))
return res_dict
问题是：如果我将 approx_poly_accuracy 参数设置得太高，圆会被检测为多边形（例如六边形或八边形）。如果我将它设置得太低，则不会将正方形检测为正方形，而是检测为五边形，例如：
红线是近似轮廓，绿线是原始轮廓。文本被检测为完全错误的轮廓，永远不应该被近似到这个级别（我不太关心文本，但如果它被检测为一个小于5个顶点的多边形，它将是一个误报).
对于人来说，很明显左边的object是一个圆，右边的五个object是正方形，所以应该有办法让计算机实现也具有很高的准确性。如何修改此代码以正确检测所有 objects？
我已经尝试过的：
- 应用
MedianFilter 这样的过滤器。这让事情变得更糟，因为正方形的圆形边缘促使它们被检测为具有四个以上顶点的多边形。
- 改变
approx_poly_accuracy参数。没有符合我目的的值，考虑到其他一些图像甚至可能有更多噪音。
- 找到允许我指定精确数量的输出点的 RDP 算法的实现。这将允许我为一定数量的点（例如在 3..10 范围内）计算建议的多边形，然后计算
(A_1 + A_2) / A_common - 1 以使用面积而不是弧长作为精度，这将可能会导致更好的结果。我还没有找到一个好的实现。我现在将尝试使用数值求解器方法动态计算出 RDP 的正确 epsilon 参数。虽然这种方法并不是真正干净和高效。我会尽快post这里的结果。如果有人有更好的方法，请告诉我。
一种可能的方法涉及计算一些 blob 描述符并根据这些属性过滤 blob。例如，您可以计算 blob 的长宽比、（近似）顶点数和面积 .步骤非常简单：
- 加载图像并将其转换为灰度。
- （反转）阈值图像。让我们确保斑点是白色的。
- 获取二值图像的轮廓。
- 计算两个特征：纵横比和顶点数
- 根据这些特征过滤 blob
来看代码：
# Imports:
import cv2
import numpy as np
# Load the image:
fileName = "yh6Uz.png"
path = "D://opencvImages//"
# Reading an image in default mode:
inputImage = cv2.imread(path + fileName)
# Prepare a deep copy of the input for results:
inputImageCopy = inputImage.copy()
# Grayscale conversion:
grayscaleImage = cv2.cvtColor(inputImage, cv2.COLOR_BGR2GRAY)
# Threshold via Otsu:
_, binaryImage = cv2.threshold(grayscaleImage, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)
# Find the blobs on the binary image:
contours, hierarchy = cv2.findContours(binaryImage, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
# Store the bounding rectangles here:
circleData = []
squaresData = []
好的。到目前为止，我已经在输入图像上加载、阈值化和计算轮廓。此外，我准备了两个列表来存储正方形和圆形的边界框。让我们创建功能过滤器：
for i, c in enumerate(contours):
# Get blob perimeter:
currentPerimeter = cv2.arcLength(c, True)
# Approximate the contour to a polygon:
approx = cv2.approxPolyDP(c, 0.04 * currentPerimeter, True)
# Get polygon's number of vertices:
vertices = len(approx)
# Get the polygon's bounding rectangle:
(x, y, w, h) = cv2.boundingRect(approx)
# Compute bounding box area:
rectArea = w * h
# Compute blob aspect ratio:
aspectRatio = w / h
# Set default color for bounding box:
color = (0, 0, 255)
我遍历每个轮廓并计算当前 blob 的 perimeter 和 polygon approximation。此信息用于近似计算 blob vertices。 aspect ratio 的计算非常简单。我首先获取 blob 的边界框并获取其尺寸：左上角 (x, y)、width 和 height。宽高比就是宽度除以高度。
正方形和圆形很紧凑。这意味着它们的宽高比应该接近 1.0。然而，正方形恰好有 4 个顶点，而（近似的）圆有更多。我使用此信息来构建一个非常基本的功能过滤器。它首先检查 aspect ratio、area，然后检查 vertices 的数量。我使用理想特征和真实特征之间的差异。参数 delta 调整过滤器容差。一定要过滤掉微小的斑点，为此使用区域：
# Set minimum tolerable difference between ideal
# feature and actual feature:
delta = 0.15
# Set the minimum area:
minArea = 400
# Check features, get blobs with aspect ratio
# close to 1.0 and area > min area:
if (abs(1.0 - aspectRatio) < delta) and (rectArea > minArea):
print("Got target blob.")
# If the blob has 4 vertices, it is a square:
if vertices == 4:
print("Target is square")
# Save bounding box info:
tempTuple = (x, y, w, h)
squaresData.append(tempTuple)
# Set green color:
color = (0, 255, 0)
# If the blob has more than 6 vertices, it is a circle:
elif vertices > 6:
print("Target is circle")
# Save bounding box info:
tempTuple = (x, y, w, h)
circleData.append(tempTuple)
# Set blue color:
color = (255, 0, 0)
# Draw bounding rect:
cv2.rectangle(inputImageCopy, (int(x), int(y)), (int(x + w), int(y + h)), color, 2)
cv2.imshow("Rectangles", inputImageCopy)
cv2.waitKey(0)
这是结果。正方形用绿色矩形标识，圆形用蓝色矩形标识。此外，边界框分别存储在 squaresData 和 circleData 中：
我想检测这张图片中的圆和五个正方形：
这是我目前使用的代码的相关部分：
# detect shapes in black-white RGB formatted cv2 image
def detect_shapes(img, approx_poly_accuracy=APPROX_POLY_ACCURACY):
res_dict = {
"rectangles": [],
"squares": []
}
vis = img.copy()
shape = img.shape
height, width = shape[0], shape[1]
total_area = height * width
# Morphological closing: get rid of holes
# img = cv2.morphologyEx(img, cv2.MORPH_CLOSE, cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (5, 5)))
# Morphological opening: get rid of extensions at the border of the objects
# img = cv2.morphologyEx(img, cv2.MORPH_OPEN, cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (121, 121)))
img = cv2.cvtColor(img, cv2.COLOR_RGB2GRAY)
# cv2.imshow('intermediate', img)
# cv2.waitKey(0)
contours, hierarchy = cv2.findContours(img, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
logging.info("Number of found contours for shape detection: {0}".format(len(contours)))
# vis = img.copy()
# cv2.drawContours(vis, contours, -1, (0, 255, 0), 2)
cv2.imshow('vis', vis)
cv2.waitKey(0)
for contour in contours:
area = cv2.contourArea(contour)
if area < MIN_SHAPE_AREA:
logging.warning("Area too small: {0}. Skipping.".format(area))
continue
if area > MAX_SHAPE_AREA_RATIO * total_area:
logging.warning("Area ratio too big: {0}. Skipping.".format(area / total_area))
continue
approx = cv2.approxPolyDP(contour, approx_poly_accuracy * cv2.arcLength(contour, True), True)
cv2.drawContours(vis, [approx], -1, (0, 0, 255), 2)
la = len(approx)
# find the center of the shape
M = cv2.moments(contour)
if M['m00'] == 0.0:
logging.warning("Unable to compute shape center! Skipping.")
continue
x = int(M['m10'] / M['m00'])
y = int(M['m01'] / M['m00'])
if la < 3:
logging.warning("Invalid shape detected! Skipping.")
continue
if la == 3:
logging.info("Triangle detected at position {0}".format((x, y)))
elif la == 4:
logging.info("Quadrilateral detected at position {0}".format((x, y)))
if approx.shape != (4, 1, 2):
raise ValueError("Invalid shape before reshape to (4, 2): {0}".format(approx.shape))
approx = approx.reshape(4, 2)
r_check, data = check_rect_or_square(approx)
blob_data = {"position": (x, y), "approx": approx}
blob_data.update(data)
if r_check == 2:
res_dict["squares"].append(blob_data)
elif r_check == 1:
res_dict["rectangles"].append(blob_data)
elif la == 5:
logging.info("Pentagon detected at position {0}".format((x, y)))
elif la == 6:
logging.info("Hexagon detected at position {0}".format((x, y)))
else:
logging.info("Circle, ellipse or arbitrary shape detected at position {0}".format((x, y)))
cv2.drawContours(vis, [contour], -1, (0, 255, 0), 2)
cv2.imshow('vis', vis)
cv2.waitKey(0)
logging.info("res_dict: {0}".format(res_dict))
return res_dict
问题是：如果我将 approx_poly_accuracy 参数设置得太高，圆会被检测为多边形（例如六边形或八边形）。如果我将它设置得太低，则不会将正方形检测为正方形，而是检测为五边形，例如：
红线是近似轮廓，绿线是原始轮廓。文本被检测为完全错误的轮廓，永远不应该被近似到这个级别（我不太关心文本，但如果它被检测为一个小于5个顶点的多边形，它将是一个误报).
对于人来说，很明显左边的object是一个圆，右边的五个object是正方形，所以应该有办法让计算机实现也具有很高的准确性。如何修改此代码以正确检测所有 objects？
我已经尝试过的：
- 应用
MedianFilter这样的过滤器。这让事情变得更糟，因为正方形的圆形边缘促使它们被检测为具有四个以上顶点的多边形。 - 改变
approx_poly_accuracy参数。没有符合我目的的值，考虑到其他一些图像甚至可能有更多噪音。 - 找到允许我指定精确数量的输出点的 RDP 算法的实现。这将允许我为一定数量的点（例如在 3..10 范围内）计算建议的多边形，然后计算
(A_1 + A_2) / A_common - 1以使用面积而不是弧长作为精度，这将可能会导致更好的结果。我还没有找到一个好的实现。我现在将尝试使用数值求解器方法动态计算出 RDP 的正确 epsilon 参数。虽然这种方法并不是真正干净和高效。我会尽快post这里的结果。如果有人有更好的方法，请告诉我。
一种可能的方法涉及计算一些 blob 描述符并根据这些属性过滤 blob。例如，您可以计算 blob 的长宽比、（近似）顶点数和面积 .步骤非常简单：
- 加载图像并将其转换为灰度。
- （反转）阈值图像。让我们确保斑点是白色的。
- 获取二值图像的轮廓。
- 计算两个特征：纵横比和顶点数
- 根据这些特征过滤 blob
来看代码：
# Imports:
import cv2
import numpy as np
# Load the image:
fileName = "yh6Uz.png"
path = "D://opencvImages//"
# Reading an image in default mode:
inputImage = cv2.imread(path + fileName)
# Prepare a deep copy of the input for results:
inputImageCopy = inputImage.copy()
# Grayscale conversion:
grayscaleImage = cv2.cvtColor(inputImage, cv2.COLOR_BGR2GRAY)
# Threshold via Otsu:
_, binaryImage = cv2.threshold(grayscaleImage, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)
# Find the blobs on the binary image:
contours, hierarchy = cv2.findContours(binaryImage, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
# Store the bounding rectangles here:
circleData = []
squaresData = []
好的。到目前为止，我已经在输入图像上加载、阈值化和计算轮廓。此外，我准备了两个列表来存储正方形和圆形的边界框。让我们创建功能过滤器：
for i, c in enumerate(contours):
# Get blob perimeter:
currentPerimeter = cv2.arcLength(c, True)
# Approximate the contour to a polygon:
approx = cv2.approxPolyDP(c, 0.04 * currentPerimeter, True)
# Get polygon's number of vertices:
vertices = len(approx)
# Get the polygon's bounding rectangle:
(x, y, w, h) = cv2.boundingRect(approx)
# Compute bounding box area:
rectArea = w * h
# Compute blob aspect ratio:
aspectRatio = w / h
# Set default color for bounding box:
color = (0, 0, 255)
我遍历每个轮廓并计算当前 blob 的 perimeter 和 polygon approximation。此信息用于近似计算 blob vertices。 aspect ratio 的计算非常简单。我首先获取 blob 的边界框并获取其尺寸：左上角 (x, y)、width 和 height。宽高比就是宽度除以高度。
正方形和圆形很紧凑。这意味着它们的宽高比应该接近 1.0。然而，正方形恰好有 4 个顶点，而（近似的）圆有更多。我使用此信息来构建一个非常基本的功能过滤器。它首先检查 aspect ratio、area，然后检查 vertices 的数量。我使用理想特征和真实特征之间的差异。参数 delta 调整过滤器容差。一定要过滤掉微小的斑点，为此使用区域：
# Set minimum tolerable difference between ideal
# feature and actual feature:
delta = 0.15
# Set the minimum area:
minArea = 400
# Check features, get blobs with aspect ratio
# close to 1.0 and area > min area:
if (abs(1.0 - aspectRatio) < delta) and (rectArea > minArea):
print("Got target blob.")
# If the blob has 4 vertices, it is a square:
if vertices == 4:
print("Target is square")
# Save bounding box info:
tempTuple = (x, y, w, h)
squaresData.append(tempTuple)
# Set green color:
color = (0, 255, 0)
# If the blob has more than 6 vertices, it is a circle:
elif vertices > 6:
print("Target is circle")
# Save bounding box info:
tempTuple = (x, y, w, h)
circleData.append(tempTuple)
# Set blue color:
color = (255, 0, 0)
# Draw bounding rect:
cv2.rectangle(inputImageCopy, (int(x), int(y)), (int(x + w), int(y + h)), color, 2)
cv2.imshow("Rectangles", inputImageCopy)
cv2.waitKey(0)
这是结果。正方形用绿色矩形标识，圆形用蓝色矩形标识。此外，边界框分别存储在 squaresData 和 circleData 中：