如何更快地进行 mediapipe 姿势估计 (python)

Question

我正在为我的游戏制作姿势估计脚本。但是，它以 20-30 fps 的速度工作，即使没有 fps 限制也不会使用整个 CPU。它也没有使用整个 GPU。有人可以帮助我吗？

这是播放舞蹈视频时的资源使用情况：https://imgur.com/a/6yI2TWg

这是我的代码：

import cv2
import mediapipe as mp
import time

inFile = '/dev/video0'

capture = cv2.VideoCapture(inFile)
FramesVideo = int(capture.get(cv2.CAP_PROP_FRAME_COUNT)) # Number of frames inside video
FrameCount = 0 # Currently playing frame
prevTime = 0

# some objects for mediapipe
mpPose = mp.solutions.pose
mpDraw = mp.solutions.drawing_utils
pose = mpPose.Pose()

while True:
    FrameCount += 1
    #read image and convert to rgb
    success, img = capture.read()
    imgRGB = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
    
    #process image
    results = pose.process(imgRGB)

    if results.pose_landmarks:
        mpDraw.draw_landmarks(img, results.pose_landmarks, mpPose.POSE_CONNECTIONS)
        #get landmark positions
        landmarks = []
        for id, lm in enumerate(results.pose_landmarks.landmark):
            h, w, c = img.shape 
            cx, cy = int(lm.x * w), int(lm.y * h) 
            cv2.putText(img, str(id), (cx,cy), cv2.FONT_HERSHEY_PLAIN, 1, (255,0,0), 1)
            landmarks.append((cx,cy))
 
    # calculate and print fps
    frameTime = time.time()
    fps = 1/(frameTime-prevTime)
    prevTime = frameTime
    cv2.putText(img, str(int(fps)), (30,50), cv2.FONT_HERSHEY_PLAIN, 3, (255,0,0), 3)

    #show image
    cv2.imshow('Video', img)
    cv2.waitKey(1)
    if FrameCount == FramesVideo-1:
        capture.release()
        cv2.destroyAllWindows()
        break

Answer 1

将mp.Pose的model_complexity设为0。

As the documentation states:

MODEL_COMPLEXITY Complexity of the pose landmark model: 0, 1 or 2. Landmark accuracy as well as inference latency generally go up with the model complexity. Default to 1.

这是我找到的最好的解决方案，也用这个。

如何更快地进行 mediapipe 姿势估计 (python)

How to make mediapipe pose estimation faster (python)

python

artificial-intelligence

python-3.x

pose-estimation

mediapipe