将 ctypes 代码转换为 cython

Question

我想将一些 ctypes 代码转换为使用 cython，但我正在努力。本质上，ctypes代码：

将两个列表的内容（浮点数）复制到 C 兼容结构中
通过 FFI 将结构发送到我的二进制文件
接收返回的结构（长度永远不会被修改）
复制内容到两个新列表
将结构发送回 FFI 边界，以便释放它们的内存

到目前为止，我的 ctypes 代码如下所示：

rlib.h

#ifndef RLIB_H
#define RLIB_H

typedef struct _FFIArray {
  void* data;
  size_t len;
} _FFIArray;

typedef struct _Result_Tuple {
    _FFIArray e;
    _FFIArray n;
} _Result_Tuple;

_Result_Tuple convert_to_bng_threaded(_FFIArray x, _FFIArray y);
void drop_float_array(_FFIArray x, _FFIArray y)

#endif

mylib.pxd

cdef extern from "rlib.h":
    struct _FFIArray:
        void* data
        size_t len

    struct _Result_Tuple:
        _FFIArray e
        _FFIArray n

    cdef _Result_Tuple convert_to_bng_threaded(_FFIArray x, _FFIArray y)
    cdef void drop_float_array(_FFIArray x, _FFIArray y)

util_cython.pyx

import cython
from mylib cimport _FFIArray, convert_to_bng_threaded, drop_float_array

def call_convert_bng(_FFIArray x, _FFIArray y):
    return convert_to_bng_threaded(x, y)

def call_drop_float_array(_FFIArray x, _FFIArray y):
    return drop_float_array(x, y)

setup.py

from setuptools import setup, Extension, find_packages

from Cython.Build import cythonize
from Cython.Distutils import build_ext

ext = Extension('util_cython', 
          sources=['util_cython.pyx'],
          libraries=['latlon_bng',],
          library_dirs=['.',],
          include_dirs=['.']
)

extensions = [ext,]

setup(
    name = "util_cython",
    ext_modules = cythonize(extensions),
    cmdclass={'build_ext': build_ext},
)

我有几个关于如何进行的问题：

首先，编译步骤当前失败：

python setup.py build_ext --inplace
Compiling util_cython.pyx because it changed.
[1/1] Cythonizing util_cython.pyx

Error compiling Cython file:
------------------------------------------------------------
...
import cython
from mylib cimport _FFIArray, convert_to_bng_threaded, drop_float_array

def call_convert_bng(_FFIArray x, _FFIArray y):
    return convert_to_bng_threaded(x, y)
                                 ^
------------------------------------------------------------

util_cython.pyx:5:34: Cannot convert '_Result_Tuple' to Python object
Traceback (most recent call last):
  File "setup.py", line 17, in <module>
    ext_modules = cythonize(extensions),
  File "/Users/sth/dev/cythonize_test/venv/lib/python2.7/site-packages/Cython/Build/Dependencies.py", line 912, in cythonize
    cythonize_one(*args)
  File "/Users/sth/dev/cythonize_test/venv/lib/python2.7/site-packages/Cython/Build/Dependencies.py", line 1034, in cythonize_one
    raise CompileError(None, pyx_file)
Cython.Compiler.Errors.CompileError: util_cython.pyx

为什么 Cython 无法转换 _Result_tuple？

其次，我如何定义一个 cython 函数来接受列表（或数组；任何支持 __iter__ 的东西），并将它们的内容复制到 _FFIArray 结构中，这样我就可以调用 call_convert_bng?

Answer 1

以下内容未经测试，因为我没有库代码，不得不对其工作原理进行一些猜测。但它应该让您了解如何去做。

我首先使用 Python 数组类型来存储您的 input/output。 standard library array type 或 numpy 数组。这些将数组连续存储在内存中（就像 C 一样），因此只要 _FFIArray data 属性可以指向此内存以供输入。

def convert_bng(double[::1] x, double[::1] y):
  # I'm assuming that the data is double, not float, but it's easy changed
  # here the [::1] promises it's continuous in memory
  cdef _FFIArray x_ffi, y_ffi
  # get a pointer to the data, and cast it to void*
  x_ffi.data = <void*>&x[0]
  x_ffi.len = x.shape[0] # possibly *sizeof(double) - depends on the C api

  # repeat for y
  y_ffi.data = <void*>&y[0]
  y_ffi.len = y.shape[0]

  cdef _Result_Tuple result = convert_to_bng_threaded(x_ffi, y_ffi)

  # get data pointers for the two result arrays
  cdef double* e_ptr = <double*>(result.e.data)
  cdef double* n_ptr = <double*>(result.n.data)
  # now view your output arrays using memoryviews
  # you need to tell it the length (this is how many doubles the contain)
  cdef double[::1] e = <double[:result.e.len:1]>e_ptr
  cdef double[::1] n = <double[:result.n.len:1]>n_ptr

  # create a numpy copy of the two arrays
  import numpy as np
  e_numpy = np.copy(e)
  n_numpy = np.copy(n)

  # you can now free your two returned arrays
  # I assume this is done with drop_float_array
  drop_float_array(result.e,result.n)

  # return as tuple containing two arrays to python
  return e_numpy, n_numpy

将 ctypes 代码转换为 cython

Convert ctypes code to cython

python

struct

cython