如何在视频上测试自定义 Faster RCNN 模型（使用 Detectron 2 和 pytorch）？

Question

我已经在用于对象检测的自定义数据集上训练了 Faster RCNN 模型，并想在视频上对其进行测试。我可以在图像上测试结果，但我对如何为视频执行此操作感到困惑。

下面是图像推理代码：

cfg.MODEL.WEIGHTS = os.path.join(cfg.OUTPUT_DIR, "model_final.pth")
cfg.DATASETS.TEST = ("my_dataset_test", )
cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = 0.7   # set the testing threshold for this model
predictor = DefaultPredictor(cfg)
test_metadata = MetadataCatalog.get("my_dataset_test")

from detectron2.utils.visualizer import ColorMode
import glob

for imageName in glob.glob('/content/test/*jpg'):
  im = cv2.imread(imageName)
  outputs = predictor(im)
  v = Visualizer(im[:, :, ::-1],
                metadata=test_metadata, 
                scale=0.8
                 )
  out = v.draw_instance_predictions(outputs["instances"].to("cpu"))
  cv2_imshow(out.get_image()[:, :, ::-1])

有人告诉我如何调整此代码以检测视频吗？

使用的平台：Google Colab

技术Stack:Detectron2、Pytorch

Answer 1

检查这个循环：

from detectron2.utils.visualizer import ColorMode
import glob
import cv2
from google.colab.patches import cv2_imshow

cap = cv2.VideoCapture('/path/to/video')
while cap.isOpened():
    ret, frame = cap.read()
    # if frame is read correctly ret is True
    if not ret:
        break

    outputs = predictor(im)
    v = Visualizer(im[:, :, ::-1],
    metadata=test_metadata, 
    scale=0.8
    )
    out = v.draw_instance_predictions(outputs["instances"].to("cpu"))
    cv2_imshow(out.get_image()[:, :, ::-1])
    
    if cv2.waitKey(1) == ord('q'):
        break

cap.release()
cv2.destroyAllWindows()

如何在视频上测试自定义 Faster RCNN 模型（使用 Detectron 2 和 pytorch）？

How to test custom Faster RCNN model(using Detectron 2 and pytorch) on video?

object-detection

pytorch

opencv-python

faster-rcnn

detectron