对图像中的单个字符进行校正

Question

我正在尝试破坏程序中的反机器人功能，在该程序中，用户必须按字母数字顺序单击图像中的字母。我已经设法使用预处理从背景中提取文本，但仍然需要使用 Tesseract 对每个单独的字符进行校正以获得最佳结果。

使用Hough Lines之前的图像，只是预处理

原始图像绘制了 HoughLinesP 检测到的线条我尝试使用 Canny Edge Detector + Hough Lines 来尝试找到每个字符下方的线。但是，它认为不一致并且过于依赖线条的质量，我无法区分底线和字符本身检测到的线条。

这是我试过的代码：

# -*- coding:utf-8 -*-
import cv2, numpy as np, time
img_roi = [48, 191, 980, 656]  # x1, y1, x2, y2
src_img_dir = "images/source/9.png"
bg_img = cv2.imread("images/background.png", cv2.IMREAD_COLOR)[img_roi[1]:img_roi[3], img_roi[0]:img_roi[2]]
# The background of the area is constant. So I have used a reference background image and removed pixels which have a similar H value as the background

bg_hsv = cv2.cvtColor(bg_img, cv2.COLOR_BGR2HSV)
src_img = cv2.imread(src_img_dir, cv2.IMREAD_COLOR)[img_roi[1]:img_roi[3], img_roi[0]:img_roi[2]]
# This image is the image where letters are placed on top of the background image

src_hsv = cv2.cvtColor(src_img, cv2.COLOR_BGR2HSV)
mask = np.zeros([src_img.shape[0], src_img.shape[1], 3], dtype=np.uint8)

offset = 3
start_time = time.time()
for y in range(src_img.shape[0]):
    for x in range(src_img.shape[1]):
        sp = src_hsv[y][x]
        bp = bg_hsv[y][x]

        if bp[0]-offset <= sp[0] <= bp[0]+offset:
            if sp[1] >= 109:
                mask[y][x] = src_img[y][x]
        elif sp[1] <= 90:
            if sp[0] >= 67:
                mask[y][x] = src_img[y][x]
            elif sp[2] >= 125 and sp[1] >= 20:
                mask[y][x] = src_img[y][x]
        else:
            mask[y][x] = src_img[y][x]
        """if sp[1] >= 60 and sp[2] >= 60:
            mask[y][x] = src_img[y][x]
            #mask[y][x] = conv"""

print("duration", time.time()-start_time)
mask = cv2.cvtColor(mask, cv2.COLOR_BGR2HSV)
#mask[:,:,2] = 255
mask = cv2.cvtColor(mask, cv2.COLOR_HSV2BGR)
mask_gray = cv2.cvtColor(mask, cv2.COLOR_BGR2GRAY)
ret, thresh = cv2.threshold(mask_gray, 0, 255, cv2.THRESH_BINARY | cv2.THRESH_OTSU)
opened = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, (3,3))
opened = cv2.morphologyEx(opened, cv2.MORPH_OPEN, (3,3))
opened = cv2.erode(opened, (3,3))
opened = cv2.dilate(opened, (3,3))
opened = cv2.dilate(opened, (5, 5))
opened = cv2.morphologyEx(opened, cv2.MORPH_CLOSE, (3,3))
opened = cv2.erode(opened, (3,3))
opened = cv2.erode(opened, (3,3))
final_img = opened
#edges = cv2.Canny(final_img, 0, 255)
lines = cv2.HoughLinesP(final_img, 1, np.pi / 180, 20, minLineLength=10, maxLineGap=3)
for line in lines:
        coords = line[0]
        cv2.line(src_img, (coords[0], coords[1]), (coords[2], coords[3]), [255,255,255], 2)
#cv2.imshow("can", edges)


#cv2.drawContours(src_img, fixed_contours, -1, (0,255,0), 2)
cv2.imshow("src", src_img)
cv2.imshow("", final_img)

cv2.waitKey(0)
cv2.destroyAllWindows()

Answer 1

乍一看，偏斜不是很严重，而且字符相距很远。

我会对你的过滤图像进行多步处理（已经很不错了）

首先检测包含两个非常大的斑点（letter/numeral + 下划线）的感兴趣区域，滤除噪声像素
然后将下划线明确检测为两者中的长而平（字母 'I' 和数字 '1' 在这方面可能有问题）
使用与局部感兴趣区域相关的下划线方向（下划线 + 字符）来确定哪一侧向下
启发式确定倾斜角度：假设 x 度（在 x 的窄范围内循环），下划线上方的四边形内有多少感兴趣的局部区域信号，使得底部（下划线）和左侧之间的角度为 x。
使用图像反变形功能，将下划线映射到具有适当宽高比的矩形的底部边缘
利润

对图像中的单个字符进行校正

Deskewing indivisual characters in an image

python

opencv

image-processing

ocr

text-extraction