为图像拼接创建邻接矩阵

Question

我在图像对之间建立了单应性。如何创建邻接矩阵来描述哪些图像相互重叠？

这是我的代码。我调用函数 'match' 来获得两个图像之间的单应性

a = s.left_list[0]
N = len(s.left_list)
adjacency_matrix = np.zeros((N, N))
for i in range(N):
    for j in range(i + 1, N):
        for b in s.left_list[1:]:
            H = s.matcher_obj.match(a, b, 'left')
            print("Homography is : ", H)




def match(self, i1, i2, direction=None):
    imageSet1 = self.getSURFFeatures(i1)
    imageSet2 = self.getSURFFeatures(i2)
    print("Direction : ", direction)
    matches = self.flann.knnMatch(
        imageSet2['des'],
        imageSet1['des'],
        k=2
        )
    good = []
    for i , (m, n) in enumerate(matches):
        if m.distance < 0.9*n.distance:
            good.append((m.trainIdx, m.queryIdx))

    if len(good) > 4:
        pointsCurrent = imageSet2['kp']
        pointsPrevious = imageSet1['kp']

        matchedPointsCurrent = np.float32(
            [pointsCurrent[i].pt for (__, i) in good]
        )
        matchedPointsPrev = np.float32(
            [pointsPrevious[i].pt for (i, __) in good]
            )

        H, s = cv2.findHomography(matchedPointsCurrent,matchedPointsPrev, cv2.RANSAC, 4)
       return H

    return None

def getSURFFeatures(self, im):

    gray = cv2.cvtColor(im, cv2.COLOR_BGR2GRAY)
    kp, des = self.surf.detectAndCompute(gray, None)
    return {'kp':kp, 'des':des}

Answer 1

换句话说，您有许多图像，您想要确定哪些图像重叠，以便将它们放入图像拼接器中。

最常见的方法之一是简单地遍历每一对独特的图像，然后计算这两幅图像之间的局部单应性。一旦你计算了单应性，你就计算了作为单应性内点的关键点对的总分数。如果它高于某个阈值，比如 50%，那么您会认为这两张图像之间有大量重叠，您会将它们算作有效对。

伪代码如下，假设您的图像存储在某个列表中 lst:

N = len(lst)
adjacency_matrix = np.zeros((N, N))
for i in range(N):
    for j in range(i + 1, N):
        1. Calculate homography between lst[i] and lst[j]
        2. Compute the total number of inlier keypoint pairs from the homography
        3. Take (2) and divide by the total number of keypoint pair matches
        4. If (3) is above a threshold (50% or 0.5), then:
            adjacency_matrix[i, j] = 1
            adjacency_matrix[j, i] = 1

使用您刚刚向我展示的代码，请注意 cv2.findHomography returns 不仅是一个矩阵，而且是一个掩码，用于告诉您哪些点对用作构建矩阵的内点。您可以简单地对掩码求和，然后除以该掩码中的元素总数，得到该比例。这只会更改代码的 return 语句。您已将重投影阈值指定为 4 个像素，该阈值很大，因此拼接结果可能会很差。当你得到这个运行.

时把它变小

def match(self, i1, i2, direction=None):
    imageSet1 = self.getSURFFeatures(i1)
    imageSet2 = self.getSURFFeatures(i2)
    print("Direction : ", direction)
    matches = self.flann.knnMatch(
        imageSet2['des'],
        imageSet1['des'],
        k=2
        )
    good = []
    for i , (m, n) in enumerate(matches):
        if m.distance < 0.9*n.distance:
            good.append((m.trainIdx, m.queryIdx))

    if len(good) > 4:
        pointsCurrent = imageSet2['kp']
        pointsPrevious = imageSet1['kp']

        matchedPointsCurrent = np.float32(
            [pointsCurrent[i].pt for (__, i) in good]
        )
        matchedPointsPrev = np.float32(
            [pointsPrevious[i].pt for (i, __) in good]
            )

        H, s = cv2.findHomography(matchedPointsCurrent,matchedPointsPrev, cv2.RANSAC, 4)
        return H, sum(s) / len(s)  #  Change here

    return None

最后，伪代码是在您的特定设置下实现的：

lst = s.left_list
N = len(lst)
adjacency_matrix = np.zeros((N, N)) 
for i in range(N):
    for j in range(i + 1, N):
        # 1. Calculate homography between lst[i] and lst[j]
        out = s.matcher_obj.match(lst[i], lst[j], 'left')

        # 2. Compute the total number of inlier keypoint pairs from the homography - done in (1)
        # 3. Take (2) and divide by the total number of keypoint pair matches - done in (1)

        # 4. If (3) is above a threshold (50% or 0.5), then:
        #    adjacency_matrix[i, j] = 1
        #    adjacency_matrix[j, i] = 1
        if out is not None:
            H, s = out
            if s >= 0.5:
                adjacency_matrix[i, j] = 1
                adjacency_matrix[j, i] = 1

请注意，如果没有足够数量的内点来提供对良好单应性的信心，您的匹配方法会输出 None。因此我们需要检查输出是否不是 None，然后相应地检查比例。最后，我强烈建议调整重投影阈值和相似性阈值（在上面的代码中为 0.5），直到您获得足够好的针脚来满足您的目的。这些目前已嵌入到您的代码中，但请考虑使它们可调。

为图像拼接创建邻接矩阵

Сreate an adjacency matrix for image stitching

python

opencv

image

image-processing

image-stitching