使用 PyTCC 库从 C 函数返回 PyObject 时如何修复 "Access Violation" 错误

Question

我正在使用一个 Python 库，它包装了名为 PyTCC 的 LibTCC。

我正在试验 Python 中 JIT 编译代码的方法。问题是，当调用一个函数时，我可以正确地 return 正常的 C 数据类型，但是当 returning 任何 PyObject *.

时我得到一个 "Access Violation" 错误

我已确保代码可以从 PyTCC 执行，如我的代码示例所示。这也意味着代码示例编译成功。

import ctypes, pytcc

program = b"""
#include "Python.h"

/* Cannot return 3 due to access violation */
PyObject * pop(PyObject * self, PyObject * args, PyObject * kwargs) {
    // Cannot return *any* Python object
    return PyLong_FromLong(3);
}

int foobar() { return 3; }  // Returns 3 just fine

// Needed to appease TCC:
int main() { }
"""

jit_code = pytcc.TCCState()
jit_code.add_include_path('C:/Python37/include')
jit_code.add_library_path('C:/Python37')
jit_code.add_library('python37')
jit_code.compile_string(program)
jit_code.relocate()

foobar_proto = ctypes.CFUNCTYPE(ctypes.c_int)
foobar = foobar_proto(jit_code.get_symbol('foobar'))

print(f'It works: {foobar()}')

pop_proto = ctypes.CFUNCTYPE(ctypes.c_voidp)
pop = pop_proto(jit_code.get_symbol('pop'))

print('But this does not for some reason:')
print(pop())
print('Never gets here due to access violation :(')

程序的输出应该是：

It works: 3
But this does not for some reason:
3
Never gets here due to access violation :(

但是，我得到了这个确切的错误：

It works: 3
But this does not for some reason:
Traceback (most recent call last):
  File "fails.py", line 40, in <module>
    print(pop())
OSError: exception: access violation writing 0x00000000FFC000E9

Answer 1

很有可能是因为你在创建对象的时候没有GIL。 return 类型也有问题。 ctypes.c_voidp 告诉 python 将其视为 int 而不是 PyObject，因此如果不是因为访问冲突，您所看到的只是值指针本身而不是它指向的内容。

尝试：

    PyObject * pop() {
    PyGILState_STATE gstate;
    gstate = PyGILState_Ensure();
    PyObject* obj = PyLong_FromLong(10);
    PyGILState_Release(gstate);
    return obj;
}

并切换
pop_proto = ctypes.CFUNCTYPE(ctypes.c_voidp)
至
pop_proto = ctypes.CFUNCTYPE(ctypes.py_object)

我的运行的输出（将 pyobject 中的值从 3 更改为 10 只是为了表明它成功了）

It works: 3
But this does not for some reason:
10
Never gets here due to access violation :(

Answer 2

无法使用 PyTCC，但代码中有问题。

根据[Python 3]: class ctypes.PyDLL(name, mode=DEFAULT_MODE, handle=None)（重点是我的）：

Instances of this class behave like CDLL instances, except that the Python GIL is not released during the function call, and after the function execution the Python error flag is checked. If the error flag is set, a Python exception is raised.

Thus, this is only useful to call Python C api functions directly.

注意：CFUNCTYPE用于CDLL，与相同PYFUNCTYPE 适用于 PyDLL.

因此，在 pop_proto 中，您应该将 ctypes.CFUNCTYPE 替换为 ctypes.PyFUNCTYPE（请注意，您在 c_voidp).

接下来，同一页指出对于 PyObject* (C)，py_object[=应该使用 53=] (Python)。所以：

pop_proto = ctypes.PyFUNCTYPE(ctypes.py_object)

如果你想严谨一点，你必须在原型中包含参数，这会使代码看起来有点复杂，但对于这种特殊情况（它们被忽略），这不是强制性的：

pop_proto = ctypes.PyFUNCTYPE(ctypes.py_object, ctypes.py_object, ctypes.py_object, ctypes.py_object)

下面是 PyObject *PyBytes_Repr(PyObject *obj, int smartquotes) 的示例（以 "old fashioned" 方式调用 C 函数）：

[cfati@CFATI-5510-0:C:\WINDOWS\system32]> "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\Scripts\python.exe" Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 22:22:05) [MSC v.1916 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> >>> import sys >>> import os >>> import ctypes >>> >>> python_dll_name = os.path.join(os.path.dirname(sys.executable), "python" + str(sys.version_info.major) + str(sys.version_info.minor) + ".dll") >>> python_dll_name 'e:\Work\Dev\VEnvs\py_064_03.07.03_test0\Scripts\python37.dll' >>> >>> python_dll = ctypes.PyDLL(python_dll_name) >>> >>> pybytes_repr_proto = ctypes.PYFUNCTYPE(ctypes.py_object, ctypes.py_object, ctypes.c_int) >>> pybytes_repr = pybytes_repr_proto(("PyBytes_Repr", python_dll)) >>> >>> b = b"abcd" >>> >>> reprb = pybytes_repr(b, 0) >>> reprb "b'abcd'"

您也可以查看。

使用 PyTCC 库从 C 函数返回 PyObject 时如何修复 "Access Violation" 错误

How to fix "Access Violation" error when returning PyObject from C function using PyTCC library

python

jit

ctypes

tcc