关键点检测和图像拼接

Question

]2

所以如下图所示，我在图像上检测到了关键点，但是环绕透视后的输出图像忽略了左侧的第一张图像，无法弄清楚为什么！

    import numpy as np
    import imutils
    import cv2

class Stitcher:
def __init__(self):
    # determine if we are using OpenCV v3.X
    self.isv3 = imutils.is_cv3()

def stitch(self, imageA,imageB, ratio=0.75, reprojThresh=10.0,
    showMatches=False):
    # unpack the images, then detect keypoints and extract
    # local invariant descriptors from them
    #(imageB, imageA) = images
    (kpsA, featuresA) = self.detectAndDescribe(imageA)
    (kpsB, featuresB) = self.detectAndDescribe(imageB)

    # match features between the two images
    M = self.matchKeypoints(kpsA, kpsB,
        featuresA, featuresB, ratio, reprojThresh)

    # if the match is None, then there aren't enough matched
    # keypoints to create a panorama
    if M is None:
        return None

    # otherwise, apply a perspective warp to stitch the images
    # together
    (matches, H, status) = M
    #print(M)
    #print(matches)
    #print(H)
    #print(status)
    #cv2.imwrite('intermediate.jpg',matches)
    result = cv2.warpPerspective(imageA, H,
        (imageA.shape[1] + imageB.shape[1], imageA.shape[0]))
    result[0:imageB.shape[0], 0:imageB.shape[1]] = imageB
    #cv2.imshow('intermediate',result)

    # check to see if the keypoint matches should be visualized
    if showMatches:
        vis = self.drawMatches(imageA, imageB, kpsA, kpsB, matches,
            status)

        # return a tuple of the stitched image and the
        # visualization
        return (result, vis)

    # return the stitched image
    return result

def detectAndDescribe(self, image):
    # convert the image to grayscale
    gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)

    # check to see if we are using OpenCV 3.X
    if self.isv3:
        # detect and extract features from the image
        #SIFT Algorithm
        descriptor = cv2.xfeatures2d.SIFT_create()
        #SURF Algorithm
        #descriptor = cv2.xfeatures2d.SURF_create()# 400 is hesian threshold, optimum values should be around 300-500
        #upright SURF: faster and can be used for panorama stiching i.e our case.
        #descriptor.upright = True
        print(descriptor.descriptorSize())
        (kps, features) = descriptor.detectAndCompute(image, None)
        print(len(kps),features.shape)

    # otherwise, we are using OpenCV 2.4.X
    else:
        # detect keypoints in the image
        detector = cv2.FeatureDetector_create("SIFT")
        kps = detector.detect(gray)

        # extract features from the image
        extractor = cv2.DescriptorExtractor_create("SIFT")
        (kps, features) = extractor.compute(gray, kps)

    # convert the keypoints from KeyPoint objects to NumPy
    # arrays
    kps = np.float32([kp.pt for kp in kps])

    # return a tuple of keypoints and features
    #print("features",features)
    return (kps, features)

def matchKeypoints(self, kpsA, kpsB, featuresA, featuresB,
    ratio, reprojThresh):
    # compute the raw matches and initialize the list of actual
    # matches
    matcher = cv2.DescriptorMatcher_create("BruteForce")
    rawMatches = matcher.knnMatch(featuresA, featuresB, 2)
    matches = []

    # loop over the raw matches
    for m in rawMatches:
        # ensure the distance is within a certain ratio of each
        # other (i.e. Lowe's ratio test)
        if len(m) == 2 and m[0].distance < m[1].distance * ratio:
            matches.append((m[0].trainIdx, m[0].queryIdx))
    print(len(matches))

    # computing a homography requires at least 4 matches
    if len(matches) > 4:
        # construct the two sets of points
        ptsA = np.float32([kpsA[i] for (_, i) in matches])
        ptsB = np.float32([kpsB[i] for (i, _) in matches])

        # compute the homography between the two sets of points
        (H, status) = cv2.findHomography(ptsA, ptsB, cv2.RANSAC,
            reprojThresh)

        # return the matches along with the homograpy matrix
        # and status of each matched point
        return (matches, H, status)

    # otherwise, no homograpy could be computed
    return None

def drawMatches(self, imageA, imageB, kpsA, kpsB, matches, status):
    # initialize the output visualization image
    (hA, wA) = imageA.shape[:2]
    (hB, wB) = imageB.shape[:2]
    vis = np.zeros((max(hA, hB), wA + wB, 3), dtype="uint8")
    vis[0:hA, 0:wA] = imageA
    vis[0:hB, wA:] = imageB

    # loop over the matches
    for ((trainIdx, queryIdx), s) in zip(matches, status):
        # only process the match if the keypoint was successfully
        # matched
        if s == 1:
            # draw the match
            ptA = (int(kpsA[queryIdx][0]), int(kpsA[queryIdx][1]))
            ptB = (int(kpsB[trainIdx][0]) + wA, int(kpsB[trainIdx][1]))
            cv2.line(vis, ptA, ptB, (0, 255, 0), 1)

    # return the visualization
    return vis

以上是关键点检测和拼接使用的代码，

还有一个问题，除了旋转图像和执行水平拼接之外，是否有人可以帮助我进行垂直图像拼接。

非常感谢！

我更改了我的代码并使用@Alexander 的padtransf.warpPerspectivePadded 函数来执行包装和混合！你能帮我得到输出图像的照明统一吗？

Answer 1

我自己也遇到过这个问题。如果我没记错的话，您正在使用 this 博客作为参考。

问题是 warpPerspective 关于行：

result = cv2.warpPerspective(imageA, H,
        (imageA.shape[1] + imageB.shape[1], imageA.shape[0]))
    result[0:imageB.shape[0], 0:imageB.shape[1]] = imageB

这个方法是全方位的。我的意思是，您只是通过根据 .shape[0] 和 .shape[1] 表示的宽度和高度替换像素值，将 imageA 拼接在 imageB 上。我用 C++ 解决了这个问题，因此没有 python 代码可以显示，但可以给你一个运行必须完成的事情。

获取您正在使用的每个图像的四个角。
获取在步骤 1 中找到的每个图像的最小角和最大角。
创建一个垫子“HTR”，用于映射图像一，使其与已经变形的图像二一致。 HTR.at(0,2) 表示 mats 3x3 矩阵中的一个位置。 Numpy 可能是您在这里需要使用的。

Mat Htr = Mat::eye(3,3,CV_64F);
    if (min_x < 0){
        max_x = image2.size().width - min_x;
        Htr.at<double>(0,2)= -min_x;
    }
    if (min_y < 0){
        max_y = image2.size().height - min_y;
        Htr.at<double>(1,2)= -min_y;
    }

对每个图像的四个角执行透视变换，以查看它们在 space 中的最终位置。

perspectiveTransform(vector<Point2f> fourPointImage1, vector<Point2f> image1dst, Htr*homography);
perspectiveTransform(vector<Point2f> fourPointImage2, vector<Point2f> image2dst, Htr);

从 image1dst 个四个角和 iamge2dst 个四个角获取最小值和最大值。
获取 image1dst 和 iamge2dst 的最小值和最大值，并用于创建一个正确大小的新 blank image 来保存最终拼接的图像。
这次重复第 3 步的过程，以确定调整每个图像的四个角所需的 translation，以确保将它们移动到 blank image[ 的虚拟 space 中=48=]
最后把你拥有的所有单应图像都扔进去 found/made。

warpPerspective(image1, blankImage, (translation*homography),result.size(), INTER_LINEAR,BORDER_CONSTANT,(0));
warpPerspective(image2, image2Updated, translation, result.size(), INTER_LINEAR, BORDER_CONSTANT,   (0));

最终目标和结果是确定图像将被扭曲到哪里，这样您就可以制作一个空白图像来容纳整个拼接图像，这样就不会裁剪掉任何内容。只有完成所有预处理后，您才真正将图像拼接在一起。我希望这对您有所帮助，如果您有任何疑问，请大声疾呼。

关键点检测和图像拼接

Key-point Detection and Image Stitching

image

opencv

keypoint