如何在 OpenCV 4.1.1 Aruco 标记检测中实现更高的稳定性？

Question

我正在视频中试验 Aruco 检测。

我正在尝试检测视频帧中的 Aruco 标记，但检测器无法检测到所有标记。这是一张稳定的图像，因此视频中没有任何移动。

我无法使用默认检测器参数使其正常工作，因此我进行了一些试验以获得最佳结果。

在标准输出和显示器上，我可以看到检测器无法获取每一帧上的每个标记，但我不知道如何实现更好的检测率。

这是我的 python 代码：


import cv2 as cv

print(cv.__version__)

import argparse
import sys
import os.path
import numpy as np
import json
winName = "Augmented Reality using Aruco markers in OpenCV"

parser = argparse.ArgumentParser(description='Augmented Reality using Aruco markers in OpenCV')
parser.add_argument('--video', default='ar_in.avi', help='Path to the input video file.')
args = parser.parse_args()

if not os.path.isfile(args.video):
    print("Input video file ", args.video, " doesn't exist")
    parser.print_help()
    sys.exit(1)
cap = cv.VideoCapture(args.video)

# Initialize the detector parameters - picked a working combination from millions of random examples
parameters =  cv.aruco.DetectorParameters_create()
parameters.minDistanceToBorder =  7
parameters.cornerRefinementMaxIterations = 149
parameters.minOtsuStdDev= 4.0
parameters.adaptiveThreshWinSizeMin= 7
parameters.adaptiveThreshWinSizeStep= 49
parameters.minMarkerDistanceRate= 0.014971725679291437
parameters.maxMarkerPerimeterRate= 10.075976700411534 
parameters.minMarkerPerimeterRate= 0.2524866841549599 
parameters.polygonalApproxAccuracyRate= 0.05562707541937206
parameters.cornerRefinementWinSize= 9
parameters.adaptiveThreshConstant= 9.0
parameters.adaptiveThreshWinSizeMax= 369
parameters.minCornerDistanceRate= 0.09167132584946237

#Load the dictionary that was used to generate the markers.
dictionary = cv.aruco.Dictionary_get(cv.aruco.DICT_5X5_50)

detected_markers={}
frame_counter = 0
while cv.waitKey(1) < 0:
    frame_counter += 1
    try:
        # get frame from the video
        hasFrame, frame = cap.read()
        # Stop the program if reached end of video
        if not hasFrame:
            print("Done processing !!!")
            cv.waitKey(3000)
            break
                
        # Detect the markers in the image
        markerCorners, markerIds, rejectedCandidates = cv.aruco.detectMarkers(frame, dictionary, parameters=parameters)
        print('frame: {} ids: {}'.format(frame_counter, markerIds.tolist()))
        im_out = cv.aruco.drawDetectedMarkers(frame, markerCorners, markerIds)

        # Showing the original image with the markers drawn on it
        cv.imshow(winName, im_out.astype(np.uint8))

    except Exception as inst:
        print(inst)
    
cv.destroyAllWindows()

标准输出如下所示：

$ ar_with_aruco.py
4.1.1
frame: 1 ids: [[21], [22]]
frame: 2 ids: [[23], [21], [22]]
frame: 3 ids: [[21], [22]]
frame: 4 ids: [[23], [22]]
frame: 5 ids: [[22]]
frame: 6 ids: [[23], [21], [22]]
frame: 7 ids: [[23], [21], [22]]
frame: 8 ids: [[22]]
frame: 9 ids: [[22]]
frame: 10 ids: [[23], [22]]
frame: 11 ids: [[23], [22]]
frame: 12 ids: [[23], [22]]
frame: 13 ids: [[22]]
frame: 14 ids: [[23], [22]]
frame: 15 ids: [[23], [22]]
frame: 16 ids: [[22]]
frame: 17 ids: [[23], [22]]
frame: 18 ids: [[22]]
frame: 19 ids: [[23], [22]]
frame: 20 ids: [[23], [21], [22]]
frame: 21 ids: [[23], [22]]
frame: 22 ids: [[23], [21], [22]]
frame: 23 ids: [[21], [22]]
frame: 24 ids: [[22]]
frame: 25 ids: [[22]]
frame: 26 ids: [[23], [22]]
frame: 27 ids: [[23], [21], [22]]
frame: 28 ids: [[22]]
frame: 29 ids: [[23], [22]]
frame: 30 ids: [[23], [21], [22]]
frame: 31 ids: [[23], [22]]
frame: 32 ids: [[23], [22]]
frame: 33 ids: [[23], [22]]
frame: 34 ids: [[23], [22]]
frame: 35 ids: [[23], [22]]
frame: 36 ids: [[23], [22]]
frame: 37 ids: [[23], [21], [22]]
frame: 38 ids: [[21], [22]]
frame: 39 ids: [[21], [22]]
frame: 40 ids: [[22]]
frame: 41 ids: [[23], [22]]
frame: 42 ids: [[23], [22]]
frame: 43 ids: [[22]]
frame: 44 ids: [[23], [21], [22]]
frame: 45 ids: [[22]]
frame: 46 ids: [[23], [22]]
frame: 47 ids: [[23], [22]]
frame: 48 ids: [[22]]
frame: 49 ids: [[22]]
frame: 50 ids: [[21], [22]]
frame: 51 ids: [[23], [22]]
frame: 52 ids: [[22]]
frame: 53 ids: [[23], [22]]
frame: 54 ids: [[23], [21], [22]]
frame: 55 ids: [[21], [22]]
frame: 56 ids: [[23], [21], [22]]
frame: 57 ids: [[23], [21], [22]]
frame: 58 ids: [[22]]
frame: 59 ids: [[23], [22]]
frame: 60 ids: [[22]]
frame: 61 ids: [[22]]
frame: 62 ids: [[21], [22]]
frame: 63 ids: [[23], [21], [22]]
frame: 64 ids: [[21], [22]]
frame: 65 ids: [[23], [21], [22]]
frame: 66 ids: [[23], [22]]
frame: 67 ids: [[23], [22]]
frame: 68 ids: [[23], [22]]
frame: 69 ids: [[23], [22]]
frame: 70 ids: [[23], [22]]
frame: 71 ids: [[22]]
frame: 72 ids: [[23], [21], [22]]
frame: 73 ids: [[23], [21], [22]]
frame: 74 ids: [[21], [22]]
frame: 75 ids: [[23], [22]]
frame: 76 ids: [[22]]
frame: 77 ids: [[22]]
frame: 78 ids: [[22]]
frame: 79 ids: [[23], [22]]
frame: 80 ids: [[21], [22]]
frame: 81 ids: [[21], [22]]
frame: 82 ids: [[23], [22]]
frame: 83 ids: [[21], [22]]
frame: 84 ids: [[21], [22]]
frame: 85 ids: [[23], [22]]
frame: 86 ids: [[23], [22]]
frame: 87 ids: [[23], [21], [22]]
frame: 88 ids: [[21], [22]]
frame: 89 ids: [[23], [21], [22]]
frame: 90 ids: [[21], [22]]
frame: 91 ids: [[21], [22]]
frame: 92 ids: [[23], [21], [22]]
frame: 93 ids: [[22]]
frame: 94 ids: [[22]]
frame: 95 ids: [[21], [22]]
frame: 96 ids: [[23], [21], [22]]
frame: 97 ids: [[21], [22]]
frame: 98 ids: [[22]]
frame: 99 ids: [[22]]
frame: 100 ids: [[23], [21], [22]]
frame: 101 ids: [[22]]
frame: 102 ids: [[22]]
frame: 103 ids: [[21], [22]]
frame: 104 ids: [[23], [22]]
frame: 105 ids: [[23], [22]]
frame: 106 ids: [[21], [22]]
frame: 107 ids: [[21], [22]]
frame: 108 ids: [[23], [21], [22]]
frame: 109 ids: [[23], [22]]
frame: 110 ids: [[22]]
frame: 111 ids: [[21], [22]]
frame: 112 ids: [[23], [21], [22]]
frame: 113 ids: [[23], [21], [22]]
frame: 114 ids: [[22]]
frame: 115 ids: [[22]]
frame: 116 ids: [[23], [22]]
frame: 117 ids: [[21], [22]]
frame: 118 ids: [[22]]
frame: 119 ids: [[23], [21], [22]]
frame: 120 ids: [[23], [21], [22]]
frame: 121 ids: [[22]]
frame: 122 ids: [[22]]
frame: 123 ids: [[22]]
frame: 124 ids: [[22]]
frame: 125 ids: [[23], [22]]
frame: 126 ids: [[22]]
frame: 127 ids: [[23], [22]]
frame: 128 ids: [[21], [22]]
frame: 129 ids: [[22]]
frame: 130 ids: [[21], [22]]
frame: 131 ids: [[22]]
frame: 132 ids: [[23], [22]]
frame: 133 ids: [[22]]
frame: 134 ids: [[23], [22]]
frame: 135 ids: [[22]]
frame: 136 ids: [[23], [22]]
frame: 137 ids: [[22]]
frame: 138 ids: [[23], [21], [22]]
frame: 139 ids: [[23], [21], [22]]
frame: 140 ids: [[23], [22]]
frame: 141 ids: [[22]]
frame: 142 ids: [[23], [21], [22]]
frame: 143 ids: [[23], [21], [22]]
frame: 144 ids: [[21], [22]]
frame: 145 ids: [[22]]
frame: 146 ids: [[23], [21], [22]]
frame: 147 ids: [[23], [22]]
frame: 148 ids: [[23], [21], [22]]
frame: 149 ids: [[23], [22]]
frame: 150 ids: [[23], [22]]
frame: 151 ids: [[23], [22]]
frame: 152 ids: [[22]]
frame: 153 ids: [[22]]
frame: 154 ids: [[23], [21], [22]]
frame: 155 ids: [[22]]
frame: 156 ids: [[22]]
frame: 157 ids: [[22]]
frame: 158 ids: [[23], [22]]
frame: 159 ids: [[22]]
frame: 160 ids: [[23], [21], [22]]
frame: 161 ids: [[23], [21], [22]]
frame: 162 ids: [[21], [22]]
frame: 163 ids: [[21], [22]]
frame: 164 ids: [[23], [21], [22]]
frame: 165 ids: [[23], [21], [22]]
frame: 166 ids: [[23], [22]]
frame: 167 ids: [[23], [22]]
frame: 168 ids: [[23], [22]]
frame: 169 ids: [[23], [21], [22]]
frame: 170 ids: [[23], [21], [22]]
frame: 171 ids: [[23], [22]]
frame: 172 ids: [[23], [22]]
frame: 173 ids: [[23], [22]]
frame: 174 ids: [[23], [22]]
frame: 175 ids: [[23], [21], [22]]
frame: 176 ids: [[23], [22]]
frame: 177 ids: [[23], [22]]
frame: 178 ids: [[23], [22]]
frame: 179 ids: [[23], [22]]
frame: 180 ids: [[23], [21], [22]]
frame: 181 ids: [[23], [22]]
frame: 182 ids: [[22]]
frame: 183 ids: [[22]]
frame: 184 ids: [[23], [22]]
frame: 185 ids: [[21], [22]]
frame: 186 ids: [[23], [22]]
frame: 187 ids: [[22]]
frame: 188 ids: [[21], [22]]
frame: 189 ids: [[23], [22]]
frame: 190 ids: [[23], [22]]
frame: 191 ids: [[23], [22]]
frame: 192 ids: [[23], [21], [22]]
frame: 193 ids: [[23], [22]]
frame: 194 ids: [[23], [21], [22]]
frame: 195 ids: [[21], [22]]
frame: 196 ids: [[23], [22]]
frame: 197 ids: [[23], [21], [22]]
frame: 198 ids: [[23], [22]]
frame: 199 ids: [[21], [22]]
frame: 200 ids: [[21], [22]]
frame: 201 ids: [[22]]
frame: 202 ids: [[22]]
frame: 203 ids: [[22]]
frame: 204 ids: [[23], [21], [22]]
frame: 205 ids: [[23], [22]]
frame: 206 ids: [[23], [22]]
frame: 207 ids: [[23], [22]]
frame: 208 ids: [[23], [22]]
frame: 209 ids: [[23], [21], [22]]
frame: 210 ids: [[23], [22]]
frame: 211 ids: [[23], [22]]
frame: 212 ids: [[23], [22]]
frame: 213 ids: [[23], [21], [22]]
frame: 214 ids: [[22]]
frame: 215 ids: [[23], [22]]
frame: 216 ids: [[22]]
frame: 217 ids: [[23], [22]]
frame: 218 ids: [[22]]
frame: 219 ids: [[23], [22]]
frame: 220 ids: [[22]]
frame: 221 ids: [[23], [22]]
frame: 222 ids: [[22]]
frame: 223 ids: [[23], [21], [22]]
frame: 224 ids: [[23], [21], [22]]
frame: 225 ids: [[23], [22]]
frame: 226 ids: [[23], [22]]
frame: 227 ids: [[23], [22]]
frame: 228 ids: [[23], [21], [22]]
frame: 229 ids: [[23], [22]]
Done processing !!!

Answer 1

马克笔太粗了，相机拍到了侧面。该算法可能无法识别黑色方形边缘。我使标记更薄，因此 aruco 检测现在更加稳健。也许一点白边可以使标记可识别。我把我的问题留在这里，这样任何人都可以从中学习。我已将新视频上传到我的github。

如何在 OpenCV 4.1.1 Aruco 标记检测中实现更高的稳定性？

How to achieve more stability in OpenCV 4.1.1 Aruco marker detection?

python

opencv

stability

marker

aruco