通过较小的第一张图片找到正确的第二张图片

Question

我们来看看这个棋盘(https://prnt.sc/r8vjth) 可以看到，有两个绿色的方块(g8和f6)，表示敌方的走法。 Square g8 是空的，很容易通过 locateOnScreen 函数找到。当我试图找到 f6 时问题就开始了，因为它 returns 又是 g8 的位置。

比如在截图的位置，找到了g8。第一部分是正确的，它来自下面代码中的 h 参数。但是第二个g8来自这个函数，这是不正确的。

def pos_loop(h): # h is the first position (Second square's position) which is struct consisting of top, left, width, height # I assume that h is correct because it always gives the right position values
    global previous_green
    global previous_white

    for p1 in pyautogui.locateAllOnScreen("C:\Users\Admin\Desktop\chesspic\small_on_white.png"):
        if p1.height != h.height and p1.width != h.width and p1 != previous_white and p1 != previous_green:
            previous_white = p1
            print("\n Returned P1")
            return p1
    for p2 in pyautogui.locateAllOnScreen("C:\Users\Admin\Desktop\chesspic\small_on_green.png"):
        if p2.height != h.height and p2.width != h.width and p2 != previous_green and p2 != previous_white:
            previous_green = p2
            print("\n Returned P2")
            return p2

    return None

previous_green和previous_white只是全局变量，在程序开始时设置为None。尽管我实施了一些检查，以确保第二个位置不等于第一个位置，但它是随机丢失的。最奇怪的是这个错误是随机的。有时它会找到正确的着法，有时则不会。

也许您知道如何解决找到正确的第二张图片 (f6) 的问题。

Answer 1

如果您不熟悉图像处理，我将向您展示一些可能会帮助您入门的简单内容。我会将您的图像作为 PIL/Pillow 图像加载，并将其转换为 Numpy 数组 - 您应该能够从 pyautogui.

中获取这样的图像或数组

from PIL import Image
import numpy as np

# Load your image and make into Numpy array for processing
im = Image.open('chess.png').convert('RGB')
na = np.array(im)

接下来，我使用颜色滴管对 g8 和 f6 上的绿色进行采样以获得它们的 RGB 值

greenA = [246,246,130]
greenB = [186,202,68]

现在我可以制作一个 True 的蒙版，无论您的图像是那种颜色：

maskA = (na[:] == greenA).all(2)
maskB = (na[:] == greenB).all(2)

如果想把蒙版可视化，可以做成图片展示出来：

Image.fromarray((maskA*255).astype(np.uint8)).show()

或将它们保存为文件：

Image.fromarray((maskA*255).astype(np.uint8)).save('a.png')

如果现在像这样对各行的像素求和：

rowTotals = np.sum(maskA,axis=1) 

array([ 0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0, 65, 65,  4,  4,  4, 59,
       59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59,
       59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59,
       59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 52, 51, 53, 53, 53,
       51, 52, 56,  4,  4,  4, 65, 65])

您可以看到所有全黑的行总和为零，只有最后几行大于零 - 因此您可以找到彩色方块的坐标。

同样，如果您像这样对列中的像素求和：

colTotals = np.sum(maskA,axis=0)

array([ 0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0, 65, 65,  4,  4,
        4, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59,
       59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59,
       59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 54, 52, 52,
       55, 55, 51, 51, 51,  4,  4,  4, 65, 65,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0])

您可以看到前几列是空的（黑色或零），然后是彩色像素，然后是更多黑色像素。

基本上，我是这样做的：

通过较小的第一张图片找到正确的第二张图片

Finding correct second image, by smaller representation of first

python

chess

image