如何使用dlib facelandmarks提取自定义ROI？

Question

我不知道如何提取绿线包围的不规则区域。即一张脸的左脸颊和右脸颊。

from collections import OrderedDict
import numpy as np
import cv2
import dlib
import imutils

CHEEK_IDXS = OrderedDict([("left_cheek", (1, 2, 3, 4, 5, 48, 31)),
                          ("right_cheek", (11, 12, 13, 14, 15, 35, 54))
                          ])

detector = dlib.get_frontal_face_detector()
predictor = dlib.shape_predictor("shape_predictor_68_face_landmarks.dat")

img = cv2.imread('Tom_Cruise.jpg')
img = imutils.resize(img, width=600)

overlay = img.copy()
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

detections = detector(gray, 0)
for k, d in enumerate(detections):
    shape = predictor(gray, d)
    for (_, name) in enumerate(CHEEK_IDXS.keys()):
        pts = np.zeros((len(CHEEK_IDXS[name]), 2), np.int32)
        for i, j in enumerate(CHEEK_IDXS[name]):
            pts[i] = [shape.part(j).x, shape.part(j).y]

        pts = pts.reshape((-1, 1, 2))
        cv2.polylines(overlay, [pts], True, (0, 255, 0), thickness=2)

    cv2.imshow("Image", overlay)

    cv2.waitKey(0)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break
cv2.destroyAllWindows()

我知道如果只是简单地从脸上提取一个矩形区域作为脸颊，代码可以这样

ROI1 = img[shape[29][1]:shape[33][1], shape[54][0]:shape[12][0]] #right cheeks
ROI1 = img[shape[29][1]:shape[33][1], shape[4][0]:shape[48][0]] #left cheek

但我想提取不规则区域进行后续处理，我该怎么做？

Answer 1

您可以通过两个简单的步骤完成此操作：

使用您拥有的点坐标创建遮罩
执行bitwise_and操作（裁剪）

代码：

cv2.drawContours(mask, [pts], -1, (255, 255, 255), -1, cv2.LINE_AA)
output = cv2.bitwise_and(img, img, mask=mask)

输出：

此外，如果您想专注于裁剪的多边形，您可以为多边形创建一个边界矩形，然后从输出帧裁剪，如 tihs:

# Create a bounding rects list at global level
bounding_rects = []

# Calculate Bounding Rects for each pts array inside the for loop
bounding_rects.append(cv2.boundingRect(pts))

# Assign geometrical values to variables to crop (Use a range(len(bounding_boxes)) for loop here)
enter code here
x1,y1,w1,h1 = bounding_rects[0]
x2,y2,w2,h2, = bounding_rects[1]

# At the end of the program, crop the bounding boxes from output
cropped1= output[y1:y1+h1, x1:x1+w1]
cropped2= output[y2:y2+h2, x2:x2+w2]

输出：

如何使用dlib facelandmarks提取自定义ROI？

how to extract self-defined ROI with dlib facelandmarks?

python

opencv

image-processing

face-detection

dlib