如何在ROS中使用OpenCV-Python

Question

在 ROS 环境中使用 yolo 和 opencv-python 我想在ROS中用yolo配合Opencv-python来控制一个摄像头，实现物体检测。现在我已经知道如何在Windows中运行yolo，但我不知道如何在ROS中运行它。如何将我的代码移植到 ROS 中？

Answer 1

ROS 是一个用于轻松组合不同库的框架，它提供的接口未定义为头文件，而是定义为 'nodes'，由 'launch files'（xml 脚本）调用。这意味着您希望在 video/camera 提要上使用运行 Yolo，但您希望它与其他库或代码交互。如果你不这样做，那么你就不需要 ROS。

ROS(v1)，几乎现在，运行在 Ubuntu 上最好。它既可以在本地运行，也可以在 virtualbox 中运行。 ROS2 支持 windows，但如果您对此有疑问，那是另一个问题。

要为您的 python 代码创建一个 ROS 节点，首先将其全部放入一个单独的 class/module； ROS 节点在 "real code" 和通信之间应该只有 接口样板 。假设您正在使用 usb_cam_node to get the camera feed, the data will be published on the 'topic' <camera_name>/image [sensor_msgs/Image]，其中 <camera_name> 是一个 usb_cam_node 参数。主题就像所有 ROS 节点之间的全局变量，由 Subscriber（带回调）读取并由 Publisher.

发布

然后你必须决定从中发布什么。因为它是 yolo，也许你想要每次检测的边界框。有一堆预定义的 ROS 消息（这些是 'topics' 的（静态和强类型）"types"）。一种是 geometry_msgs/PolygonStamped，它允许您指定框的角并标记它。

这是一些示例代码，摘自 wiki

# yolo_boxes_node.py
import rospy
from std_msgs.msg import Header
from geometry_msgs.msg import PolygonStamped, Point32
from sensor_msgs.msg import Image
# This is your custom yolo code
import my_yolo as yolo # Assuming a method like as follows:
# yolo.evaluate(img_frame) -> 
#   boxes ([x,y,width,hight] list), confidences (list), classids (list)

# Subscribers
#     img_sub (sensor_msgs/Image): "webcam/image" #Comment: we document a name for the sub, the type, and the default topic for it

# Publishers
#     boxes_pub (geometry_msgs/PolygonStamped): "webcam/yolo/boxes"

# Publishers
boxes_pub = None

# Parameters
frequency = 100.0 # Hz

# Global Variables
img_frame = None
header = None

def img_callback(data): # data of type Image
    global img_frame
    global header
    img_frame = data.data
    header = data.header

def timer_callback(event): # This is to process data at a fixed rate, perhaps different from camera framerate
    # Convert img_frame somehow if needed
    if img_frame is None or boxes_pub is None:
        return
    boxes, confidences, classids = yolo.evaluate(img_frame)
    for b in boxes:
        msg = PolygonStamped()
        msg.header = header # You could use the header differently
        msg.polygon.points.append(Point32(x=b[0],y=b[1]))
        msg.polygon.points.append(Point32(x=b[0]+b[2],y=b[1]))
        msg.polygon.points.append(Point32(x=b[0],y=b[1]+b[3]))
        msg.polygon.points.append(Point32(x=b[0]+b[2],y=b[1]+b[3]))
        boxes_pub.publish(msg)

# In your main function, you subscribe to topics
def yolo_boxes_node():
    # Init ROS
    rospy.init_node('yolo_boxes_node', anonymous=True)

    # Parameters
    if rospy.has_param('~frequency'):
        frequency = rospy.get_param('~frequency')

    # Subscribers
    # Each subscriber has the topic, topic type, AND the callback!
    rospy.Subscriber('webcam/image', Image, img_callback)
    # Rarely/never need to hold onto the object with a variable: 
    #     img_sub = rospy.Subscriber(...)
    rospy.Timer(rospy.Duration(1.0/frequency), timer_callback)

    # Publishers
    boxes_pub = rospy.Publisher('webcam/yolo/boxes', PolygonStamped, queue_size = 100)
    # queue_size increases as buffer for msgs; if you have 1000s of boxes, might need bigger

    # spin() simply keeps python from exiting until this node is stopped
    # This is an infinite loop, the only code that gets ran are callbacks
    rospy.spin()
    # NO CODE GOES AFTER THIS, NONE! USE TIMER CALLBACKS!
    # unless you need to clean up resource allocation, close(), etc when program dies

if __name__ == '__main__':
    yolo_boxes_node()

因此，举个例子 xml 启动文件可能是：

<?xml version="1.0"?>
<!-- my_main_program.launch -->
<launch>
  <!--
    Pub: <camera_name>/image [sensor_msgs/Image]
  -->
  <node name="usb_cam_node" type="usb_cam_node" pkg="usb_cam" output="screen" restart="true">
    <param name="camera_name" value="webcam"/>
    <param name="video_device" value="/dev/video0"/>
  </node>

  <!--
    img_sub: webcam/image [sensor_msgs/Image]
    boxes_pub: webcam/yolo/boxes [geometry_msgs/PolygonStamped]
  -->
  <node name="yolo_boxes_node" type="yolo_boxes_node" pkg="my_pkg" output="screen">
    <param name="frequency" value="30.0"/>
  </node>
</launch>

如何在ROS中使用OpenCV-Python

how to use OpenCV-Python in ROS

ros

opencv-python

yolo