当嵌入式 Python 解释器第二次尝试导入外部模块时，C++ 应用程序崩溃

Question

如果我在不同的 pybind11::scoped_interpreter 会话中两次导入外部模块，应用程序会在 eval.h 函数 eval 中的以下行崩溃：

PyObject *result = PyRun_String(buffer.c_str(), start, global.ptr(), local.ptr());

和

Exception thrown at 0x00007FFD710C4E0C (multiarray.cp36-win_amd64.pyd) in pybind-test.exe: 0xC0000005: Access violation writing location 0x000000000000000A.

可重现的示例代码

namespace py = pybind11;
void test() {
    try {
        py::scoped_interpreter guard{};
        py::object mainScope = py::module::import("__main__").attr("__dict__");
        py::exec(
            "import numpy\n",
            mainScope);
    }
    catch (py::error_already_set const &pythonErr) {  std::cout << pythonErr.what(); }
}
int main() {
    test();   // Runs fine
    test();   // Crashes at py::exec
}

我觉得这与 pybind11 中的评论有关 embed.h:

The interpreter can be restarted by calling initialize_interpreter again. Modules created using pybind11 can be safely re-initialized. However, Python itself cannot completely unload binary extension modules and there are several caveats with regard to interpreter restarting. All the details can be found in the CPython documentation. In short, not all interpreter memory may be freed, either due to reference cycles or user-created global data.

所以没有办法调用Python解释器两次？我有一个 python 文件，其中包含我需要在 C++ 算法执行的不同点调用的辅助 numpy 函数。这是否意味着我不能那样做？

Answer 1

转述自 pybind11 github repo.

的讨论

不要使用 py::scoped_interpreter，而是使用 py::initialize_interpreter 和 py::finalize_interpreter。任意多次调用解释器。

警告：，“Python解释器不是完全线程安全的为了支持多线程Python程序，有一个全局锁，称为全局解释器锁或GIL“.

用法示例：

namespace py = pybind11;
void test() {
    try {
        py::object mainScope = py::module::import("__main__").attr("__dict__");
        py::exec(
            "import numpy\n",
            mainScope);
    }
    catch (py::error_already_set const &pythonErr) {  std::cout << pythonErr.what(); }
}
int main() {
   py::initialize_interpreter();
    test();  
    test();   
    py::finalize_interpreter();
}

Answer 2

根据这篇文章 https://www.boost.org/doc/libs/1_47_0/libs/python/todo.html#pyfinalize-safety。

，目前从不直接或通过 scoped_interpreter 调用 PyFinalize 可能更好]

在调用 finalize 然后重新初始化后重新加载模块时可能会出错。当遵循当前批准的答案 .

时，我运行进入了那个

https://www.boost.org/doc/libs/1_47_0/libs/python/todo.html#pyfinalize-safety

PyFinalize Safety: Currently Boost.Python has several global (or function-static) objects whose existence keeps reference counts from dropping to zero until the Boost.Python shared object is unloaded. This can cause a crash because when the reference counts do go to zero, there's no interpreter. In order to make it safe to call PyFinalize() we must register an atexit routine which destroys these objects and releases all Python reference counts so that Python can clean them up while there's still an interpreter. Dirk Gerrits has promised to do this job.

当嵌入式 Python 解释器第二次尝试导入外部模块时，C++ 应用程序崩溃

C++ application crashes when embedded Python interpreter tries to import external module a second time

c++

python

pybind11

可重现的示例代码