Python 模块中 atexit 调用的对象清理和函数序列

Question

我正在使用 boost-python 将遗留 C++ 库与 Python 集成。遗留库有一些全局初始化，然后其中的类使用应用程序范围的数据。我需要确保在销毁所有包装对象后调用遗留库的关闭函数，并认为这可以通过使用 atexit 注册关闭函数来实现。但是，我发现在atexit调用关闭函数后，包装对象正在被清理，导致遗留库中出现多个段错误！

我可以通过在退出前对包装对象调用 del 来实现所需的行为，但希望将删除留给 Python。我已经查看了documentation of object.__del__中的红色警告框，我想知道我的理想世界是否遥不可及。

在 python 模块中包装遗留代码时，对于确保在清除所有对象后调用关闭方法有何建议？

一些重要的平台详细信息：

Python 2.7.2
Visual Studio 2013
64 位版本

最小代码：

#include <iostream>
#include <boost/python.hpp>

using namespace std;

namespace legacy
{
    void initialize() { cout << "legacy::initialize" << endl; }
    void shutdown() { cout << "legacy::shutdown" << endl; }

    class Test
    {
    public:
        Test();
        virtual ~Test();
    };

    Test::Test() { }
    Test::~Test() { cout << "legacy::Test::~Test" << endl; }
}

BOOST_PYTHON_MODULE(legacy)
{
    using namespace boost::python;
    legacy::initialize();
    class_<legacy::Test>("Test");
    def("_finalize", &legacy::shutdown);
    object atexit = object(handle<>(PyImport_ImportModule("atexit")));
    object finalize = scope().attr("_finalize");
    atexit.attr("register")(finalize);
}

编译后，可以运行使用 python 并显示以下输入和输出：

>>> import legacy
legacy::initialize
>>> test = legacy.Test()
>>> ^Z
legacy::shutdown
legacy::Test::~Test

Answer 1

简而言之，创建一个守卫类型，它将在其构造函数和析构函数中初始化和关闭遗留库，然后通过每个公开对象中的智能指针管理守卫。

有一些微妙的细节可能会使正确的销毁过程变得困难：

Py_Finalize() 中对象和模块中对象的销毁顺序是随机的。
模块没有完成。特别是，动态加载的扩展模块不会被卸载。
遗留 API 只有在使用它的所有对象都被销毁后才能关闭。然而，对象本身可能并不知道彼此。

为此，Boost.Python 对象需要协调何时初始化和关闭遗留 API。这些对象还需要拥有使用遗留 API 的遗留对象的所有权。使用 single responsibility principle，可以将职责划分为几个类.

可以使用 resource acquisition is initialization (RAII) 习惯用法来初始化和关闭旧版 AP。例如下面的legacy_api_guard，当构造legacy_api_guard对象时，会初始化遗留API。当 legacy_api_guard 对象被破坏时，它会关闭遗留的 API.

/// @brief Guard that will initialize or shutdown the legacy API.
struct legacy_api_guard
{
  legacy_api_guard()  { legacy::initialize(); }
  ~legacy_api_guard() { legacy::shutdown();   }
};

由于多个对象需要共享管理何时初始化和关闭遗留 API，可以使用智能指针，例如 std::shared_ptr，负责管理守卫。以下示例延迟初始化并关闭旧版 API:

/// @brief Global shared guard for the legacy API.
std::weak_ptr<legacy_api_guard> legacy_api_guard_;

/// @brief Get (or create) guard for legacy API.
std::shared_ptr<legacy_api_guard> get_api_guard()
{
  auto shared = legacy_api_guard_.lock();
  if (!shared)
  {
    shared = std::make_shared<legacy_api_guard>();
    legacy_api_guard_ = shared;
  }
  return shared;
}

最后，将嵌入到 Boost.Python 对象中的实际类型需要在创建遗留对象的实例之前获得遗留 API 守卫的句柄。此外，在销毁时，遗留 API 守卫应在遗留对象被销毁后释放。实现此目的的一种非侵入式方法是在将遗留类型公开给 Boost.Python 时使用自定义 HeldType。公开类型时，需要抑制默认 Boost.Python 生成的初始值设定项，因为将使用自定义工厂函数来控制对象创建：

/// @brief legacy_object_holder is a smart pointer that will hold
///        legacy types and help guarantee the legacy API is initialized
///        while these objects are alive.  This smart pointer will remain
///        transparent to the legacy library and the user-facing Python.
template <typename T>
class legacy_object_holder
{
public:

  typedef T element_type;

  template <typename... Args>
  legacy_object_holder(Args&&... args)
    : legacy_guard_(::get_api_guard()),
      ptr_(std::make_shared<T>(std::forward<Args>(args)...))
  {}

  legacy_object_holder(legacy_object_holder& rhs) = default;

  element_type* get() const { return ptr_.get(); }

private:

  // Order of declaration is critical here.  The guard should be
  // allocated first, then the element.  This allows for the
  // element to be destroyed first, followed by the guard.
  std::shared_ptr<legacy_api_guard> legacy_guard_;
  std::shared_ptr<element_type> ptr_;
};

/// @brief Helper function used to extract the pointed to object from
///        an object_holder.  Boost.Python will use this through ADL.
template <typename T>
T* get_pointer(const legacy_object_holder<T>& holder)
{
  return holder.get();
}

/// Auxiliary function to make exposing legacy objects easier.
template <typename T, typename ...Args>
legacy_object_holder<T>* make_legacy_object(Args&&... args)
{
  return new legacy_object_holder<T>(std::forward<Args>(args)...);
}

BOOST_PYTHON_MODULE(example)
{
  namespace python = boost::python;
  python::class_<
      legacy::Test, legacy_object_holder<legacy::Test>, 
      boost::noncopyable>("Test", python::no_init)
    .def("__init__", python::make_constructor(
      &make_legacy_object<legacy::Test>))
    ;
}

这是一个完整的示例 demonstrating 使用自定义 HeldType 以非侵入式惰性方式保护具有共享管理的资源：

#include <iostream> // std::cout, std::endl
#include <memory> // std::shared_ptr, std::weak_ptr
#include <boost/python.hpp>

/// @brief legacy namespace that cannot be changed.
namespace legacy {

void initialize() { std::cout << "legacy::initialize()" << std::endl; }
void shutdown()   { std::cout << "legacy::shutdown()" << std::endl;   }

class Test
{
public:
  Test()          { std::cout << "legacy::Test::Test()" << std::endl;  }
  virtual ~Test() { std::cout << "legacy::Test::~Test()" << std::endl; }
};

void use_test(Test&) {}

} // namespace legacy

namespace {

/// @brief Guard that will initialize or shutdown the legacy API.
struct legacy_api_guard
{
  legacy_api_guard()  { legacy::initialize(); }
  ~legacy_api_guard() { legacy::shutdown();   }
};

/// @brief Global shared guard for the legacy API.
std::weak_ptr<legacy_api_guard> legacy_api_guard_;

/// @brief Get (or create) guard for legacy API.
std::shared_ptr<legacy_api_guard> get_api_guard()
{
  auto shared = legacy_api_guard_.lock();
  if (!shared)
  {
    shared = std::make_shared<legacy_api_guard>();
    legacy_api_guard_ = shared;
  }
  return shared;
}

} // namespace 

/// @brief legacy_object_holder is a smart pointer that will hold
///        legacy types and help guarantee the legacy API is initialized
///        while these objects are alive.  This smart pointer will remain
///        transparent to the legacy library and the user-facing Python.
template <typename T>
class legacy_object_holder
{
public:

  typedef T element_type;

  template <typename... Args>
  legacy_object_holder(Args&&... args)
    : legacy_guard_(::get_api_guard()),
      ptr_(std::make_shared<T>(std::forward<Args>(args)...))
  {}

  legacy_object_holder(legacy_object_holder& rhs) = default;

  element_type* get() const { return ptr_.get(); }

private:

  // Order of declaration is critical here.  The guard should be
  // allocated first, then the element.  This allows for the
  // element to be destroyed first, followed by the guard.
  std::shared_ptr<legacy_api_guard> legacy_guard_;
  std::shared_ptr<element_type> ptr_;
};

/// @brief Helper function used to extract the pointed to object from
///        an object_holder.  Boost.Python will use this through ADL.
template <typename T>
T* get_pointer(const legacy_object_holder<T>& holder)
{
  return holder.get();
}

/// Auxiliary function to make exposing legacy objects easier.
template <typename T, typename ...Args>
legacy_object_holder<T>* make_legacy_object(Args&&... args)
{
  return new legacy_object_holder<T>(std::forward<Args>(args)...);
}

// Wrap the legacy::use_test function, passing the managed object.
void legacy_use_test_wrap(legacy_object_holder<legacy::Test>& holder)
{
  return legacy::use_test(*holder.get());
}

BOOST_PYTHON_MODULE(example)
{
  namespace python = boost::python;
  python::class_<
      legacy::Test, legacy_object_holder<legacy::Test>, 
      boost::noncopyable>("Test", python::no_init)
    .def("__init__", python::make_constructor(
      &make_legacy_object<legacy::Test>))
    ;

  python::def("use_test", &legacy_use_test_wrap);
}

交互使用：

>>> import example
>>> test1 = example.Test()
legacy::initialize()
legacy::Test::Test()
>>> test2 = example.Test()
legacy::Test::Test()
>>> test1 = None
legacy::Test::~Test()
>>> example.use_test(test2)
>>> exit()
legacy::Test::~Test()
legacy::shutdown()

请注意，基本的整体方法也适用于非惰性解决方案，其中遗留 API 在导入模块时被初始化。需要使用 shared_ptr 而不是 weak_ptr，并使用 atexit.register():

注册一个清理函数

/// @brief Global shared guard for the legacy API.
std::shared_ptr<legacy_api_guard> legacy_api_guard_;

/// @brief Get (or create) guard for legacy API.
std::shared_ptr<legacy_api_guard> get_api_guard()
{
  if (!legacy_api_guard_)
  {
    legacy_api_guard_ = std::make_shared<legacy_api_guard>();
  }
  return legacy_api_guard_;
}

void release_guard()
{
  legacy_api_guard_.reset();
}

...

BOOST_PYTHON_MODULE(example)
{
  // Boost.Python may throw an exception, so try/catch around
  // it to initialize and shutdown legacy API on failure.
  namespace python = boost::python;
  try
  {
    ::get_api_guard(); // Initialize.  

    ...

    // Register a cleanup function to run at exit.
    python::import("atexit").attr("register")(
      python::make_function(&::release_guard)
    );
  }
  // If an exception is thrown, perform cleanup and re-throw.
  catch (const python::error_already_set&)
  {
    ::release_guard();  
    throw;
  }
}

请参阅 here 进行演示。

Python 模块中 atexit 调用的对象清理和函数序列

Sequence of object cleanup and functions called by atexit in Python module

c++

python

atexit

boost-python