OpenCV Cython 桥泄漏内存

Question

我已经编写了与 Basler 相机一起使用的 VideoCapture class 的实现。它是这样使用的：

import cv2
import PyBaslerCamera

video = PyBaslerCamera.PyBaslerCamera()
video.open(0)
while True:
    ret, image = video.read()
    cv2.imshow("Test", image)
    cv2.waitKey(1)

我的 Cython 文件如下所示：

# distutils: language = c++
# distutils: sources = BaslerCamera.cpp

from cython.operator cimport dereference as deref
from cpython.ref cimport PyObject
from libcpp cimport bool

cdef extern from "opencv2/core/core.hpp" namespace "cv":    
    cdef cppclass Mat:
        bool empty() const
        void release() const

    cdef cppclass _OutputArray:
        Mat getMat(int idx=-1) const



cdef extern from "cv2.cpp":
    void import_array()
    PyObject* pyopencv_from(const Mat&)
    int pyopencv_to(PyObject*, Mat&)

cdef Mat np2mat(object array):
    cdef Mat mat
    cdef PyObject* pyobject = <PyObject*> array
    pyopencv_to(pyobject, mat)
    return <Mat>mat

cdef object mat2np(const Mat &mat):
    return <object> pyopencv_from(mat)

cdef extern from "BaslerCamera.h" namespace "cv":
    cdef cppclass BaslerCamera:
        BaslerCamera()
        bool open(int index)
        bool isOpened()
        void release()
        bool grab()
        Mat retrieve()
        bool read(_OutputArray image)
        Mat read()
        bool set(int propId, double value)
        double get(int propId)
        BaslerCamera &operator>>(Mat &image)

cdef class PyBaslerCamera:
    cdef BaslerCamera *thisptr
    cdef Mat mat

    def __cinit__(self):
        print("PyBaslerCamera init")
        import_array()
        self.thisptr = new BaslerCamera()

    def __dealloc__(self):
        del self.thisptr

    def open(self, int index = 0):
        self.thisptr.open(index)

    def read(self):
        mat = self.thisptr.read()

        if mat.empty():
            return (False, None)
        else:
            out = mat2np(mat)
            return (True, out)

而且我使用了来自 OpenCV 的 cv2.cpp 文件： https://github.com/Itseez/opencv/blob/master/modules/python/src2/cv2.cpp

现在，一切正常，我正在从摄像头获取视频流，但问题是它泄漏很多（几秒钟后它会填满我的内存，这让我相信它只是泄漏所有帧）。 Valgrind 似乎证实了

==21435== 1,050,624,000 bytes in 152 blocks are possibly lost in loss record 5,939 of 5,939
==21435==    at 0x4C2BBA0: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==21435==    by 0x20D7F3AB: ??? (in /usr/lib/python3/dist-packages/numpy/core/multiarray.cpython-34m-x86_64-linux-gnu.so)
==21435==    by 0x20D1BD89: ??? (in /usr/lib/python3/dist-packages/numpy/core/multiarray.cpython-34m-x86_64-linux-gnu.so)
==21435==    by 0x251D55E1: NumpyAllocator::allocate(int, int const*, int, void*, unsigned long*, int, cv::UMatUsageFlags) const (cv2.cpp:156)
==21435==    by 0xB983720: cv::Mat::create(int, int const*, int) (in /usr/local/lib/libopencv_core.so.3.0.0)
==21435==    by 0xB9B54C7: cv::_OutputArray::create(int, int, int, int, bool, int) const (in /usr/local/lib/libopencv_core.so.3.0.0)
==21435==    by 0xB810A7C: cv::Mat::copyTo(cv::_OutputArray const&) const (in /usr/local/lib/libopencv_core.so.3.0.0)
==21435==    by 0x251D44F9: pyopencv_from<cv::Mat> (cv2.cpp:211)
==21435==    by 0x251D44F9: __pyx_f_14PyBaslerCamera_mat2np (PyBaslerCamera.cpp:662)
==21435==    by 0x251D44F9: __pyx_pf_14PyBaslerCamera_14PyBaslerCamera_6read(__pyx_obj_14PyBaslerCamera_PyBaslerCamera*) [clone .isra.9] (PyBaslerCamera.cpp:973)
==21435==    by 0x503F5C: PyEval_EvalFrameEx (in /usr/bin/python3.4)
==21435==    by 0x5A9CB4: PyEval_EvalCodeEx (in /usr/bin/python3.4)
==21435==    by 0x5E7104: ??? (in /usr/bin/python3.4)
==21435==    by 0x5E71C8: PyRun_FileExFlags (in /usr/bin/python3.4)
==21435== 
==21435== LEAK SUMMARY:
==21435==    definitely lost: 165,107 bytes in 262 blocks
==21435==    indirectly lost: 179,724,840 bytes in 205 blocks
==21435==      possibly lost: 1,057,720,529 bytes in 646 blocks
==21435==    still reachable: 9,399,307 bytes in 10,288 blocks
==21435==         suppressed: 0 bytes in 0 blocks
==21435== Reachable blocks (those to which a pointer was found) are not shown.
==21435== To see them, rerun with: --leak-check=full --show-leak-kinds=all

看起来 Numpy 分配器创建的 ndarrays 没有被释放，但我不知道如何解决这个问题。谁能告诉我如何正确释放这段内存？

或者如果有人对如何处理整个 cv::Mat 到 np array 业务有更好的建议，我愿意接受想法。

Answer 1

问题是您需要将 pyopencv_from 的定义从 PyObject* pyopencv_from(const Mat&) 更改为 object pyopencv_from(const Mat&):

# just illustrated in place
cdef extern from "cv2.cpp":
    void import_array()
    object pyopencv_from(const Mat&)
    # etc

# and a function that appears a bit later...
cdef object mat2np(const Mat &mat):
    # return <object> pyopencv_from(mat) # Problem line!
    # can now become:
    return pyopencv_from(mat)

这是基于 a newsgroup post，它引用了我认为不再存在的文档。此处引用：

whenver the Py_ function returns a new reference to a PyObject*, the return type is "object". When the function returns a borrowed reference, the return type is PyObject*. When Cython sees "object" as a return type it doesn't increment the reference count. When it sees PyObject* in order to use the result you must explicitly cast to <object>, and when you do that Cython increments the reference count wether you want it to or not, forcing you to an explicit DECREF (or leak memory). To avoid this we make the above convention.

With borrowed references if you do an explicit typecast to <object>, [Cython] generates an INCREF and DECREF so you have to be careful.

所以要点是：

从 pyopencv_from 返回的对象的引用计数为 1。
如果你告诉 Cython 函数 returns object 它会适当地引用它（生成的代码可能会显示 GOTREF 这是一个空操作，除了调试目的后跟DECREF稍后释放内存）。
如果您告诉 Cython 函数 returns PyObject* 它什么都不做，因为它只是将其视为任意指针类型（好的 - 但您必须进行引用计数)
当您对 <object> 进行显式处理时（请参阅上面清单中的 "problem line"），它会将引用递增 1（因此现在是 2）以声明所有权，但只会递减一次。引用计数永远保持在 1，对象永远不会被释放。

OpenCV Cython 桥泄漏内存

OpenCV Cython bridge leaking memory

c++

python

opencv

cython