TypeError: can't pickle _thread.RLock objects in python 3

Question

我有一个非常大的 Web 项目 API 使用 Flask 和 Python。它用于自动测试一些电子硬件。该程序使用一些线程来运行网络 UI 而服务器运行一些服务（SSH、串行、VISA）等等。

该程序最初是在 python 2.7 中编写的，在此版本中运行良好。现在，出于显而易见的原因，我正在尝试将其更新为 python 3.8。

在更新项目时，copy 库出现问题。它应该序列化一个 _thread.RLock 对象并将其发送到另一个线程，但它一直给我一个错误。这是我得到的回溯：

Traceback (most recent call last):
  File "c:\git_files\[...]\nute\route_config\flask_api_testbench.py", line 208, in _hook_run
    super(FlaskAPITestbench, self).hook_run()
  File "c:\git_files\[...]\nute\core\testbench\base.py", line 291, in hook_run
    while self.state_machine():
  File "c:\git_files\[...]\nute\core\testbench\base.py", line 304, in state_machine
    on_input=self.state_testrun
  File "c:\git_files\[...]\nute\core\testbench\base.py", line 380, in wait_for_input_or_testrun
    self.hook_load_testrun(config_with_input)
  File "c:\git_files\[...]\nute\core\testbench\base.py", line 428, in hook_load_testrun
    self.interface.load_testrun(self.load_testrun(config))
  File "c:\git_files\[...]\nute\core\testbench\base.py", line 461, in load_testrun
    testrun = self.test_loader.load_testrun(config, context_type=self.TestRunContext)
  File "c:\git_files\[...]\nute\core\testrun\loader.py", line 89, in load_testrun
    testrun_template = process_all_loaders(self.batchers, _process_batcher)
  File "c:\git_files\[...]\nute\core\config\loader.py", line 127, in process_all_loaders
    return fn(loader)
  File "c:\git_files\[...]\nute\core\testrun\loader.py", line 85, in _process_batcher
    batcher.batch_testrun(testrun_template, config, context)
  File "c:\git_files\[...]\nute\batcher\python_module_batcher.py", line 21, in batch_testrun
    batch_module.main(testrun, context)
  File "C:\GIT_Files\[...]\pyscripts\script\patest\_batch.py", line 168, in main
    test.suite(ImpedanceTest)
  File "c:\git_files\[...]\nute\core\testrun\base.py", line 213, in suite
    testsuite = testsuite_instance_or_class()
  File "c:\git_files\[...]\nute\core\functions\helpers.py", line 233, in __new__
    cls._attach_nodes_to(template)
  File "c:\git_files\[...]\nute\core\functions\helpers.py", line 271, in _attach_nodes_to
    node = root.import_testcase(testcase)
  File "c:\git_files\[...]\nute\core\functions\specific.py", line 307, in import_testcase
    test_node = testcase.copy(cls=self.__class__)
  File "c:\git_files\[...]\nute\core\functions\base.py", line 645, in copy
    value = copy(value)
  File "c:\users\[...]\.conda\envs\py37\lib\copy.py", line 96, in copy
    rv = reductor(4)
TypeError: can't pickle _thread.RLock objects

它在 Python 2.7 中运行良好，但在 Python 3.x 中运行不佳。我在 3.7.10、3.8.9 和 3.9.6 上试过，结果相同。

下面是我的 copy 的 wrap 方法的实现：

from copy import copy

...

def copy(self, cls=None): # class method
    if cls is None:  
        cls = self.__class__
    new_self = cls()
    for key, value in self.__dict__.items():
        # if key == "multithread_lock":
        #     continue

        if self.should_copy_attribute(key, value):
            # Handle recursion by pointing to the new object instead of copying.
            if value is self:
                value = new_self
            else:
                value = copy(value)  # This is where it fails
            new_self.__dict__[key] = value
    return new_self

正如您在评论部分看到的那样，跳过任何 _thread.RLock 对象的 pickling 可以使程序运行，但我需要手动刷新网络 UI 才能看到它运行ning 因为线程不工作。

知道为什么它适用于 python 2.7 但不适用于较新的版本吗？提前致谢。

Answer 1

所以我发现一个_thread.RLock()对象是不能复制的。我只是添加了一个条件来跳过要复制的对象，它工作正常。 web端UI不爽，我换了低版本的Flask-CoketIO，效果还不错

TypeError: can't pickle _thread.RLock objects in python 3

TypeError: can't pickle _thread.RLock objects in python 3

python

python-3.x

python-2.7

pickle