TypeError: can't pickle Struct objects

Question

我是 python 包 Construct 的维护者，我寻求帮助使这个库变得可挑选。有人来找我，要求它能够进行 cloudpickle。不幸的是，我拥有的类不能泡菜，也不能用 cloudpickle，也不能用 dill。请帮忙

相关票是：https://github.com/construct/construct/issues/894

$ python3
Python 3.6.9 (default, Oct  8 2020, 12:12:24) 
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import cloudpickle
>>> import construct
>>> cloudpickle.dumps(construct.Byte)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.6/dist-packages/cloudpickle/cloudpickle_fast.py", line 102, in dumps
    cp.dump(obj)
  File "/usr/local/lib/python3.6/dist-packages/cloudpickle/cloudpickle_fast.py", line 563, in dump
    return Pickler.dump(self, obj)
  File "/usr/lib/python3.6/pickle.py", line 409, in dump
    self.save(obj)
  File "/usr/lib/python3.6/pickle.py", line 521, in save
    self.save_reduce(obj=obj, *rv)
  File "/usr/lib/python3.6/pickle.py", line 634, in save_reduce
    save(state)
  File "/usr/lib/python3.6/pickle.py", line 476, in save
    f(self, obj) # Call unbound method with explicit self
  File "/usr/lib/python3.6/pickle.py", line 821, in save_dict
    self._batch_setitems(obj.items())
  File "/usr/lib/python3.6/pickle.py", line 847, in _batch_setitems
    save(v)
  File "/usr/lib/python3.6/pickle.py", line 496, in save
    rv = reduce(self.proto)
TypeError: can't pickle Struct objects

莳萝也有同样的错误。 Pickle 模块产生单行错误。

Answer 1

已解决：错误给了我一条线索。我试图 pickle 的 Byte class 对象是一个 FormatField，与 Struct class 无关。经过几个小时的思考，我才想到 Struct 指的是 struct.Struct 而不是 construct.Struct。摆脱它后，它可以正确序列化。

空 construct.Struct class 对象序列化没有问题。

违规代码：

class FormatField(Construct):
    def __init__(self, endianity, format):
        if endianity not in list("=<>"):
            raise FormatFieldError("endianity must be like: = < >", endianity)
        if format not in list("fdBHLQbhlqe?"):
            raise FormatFieldError("format must be like: f d B H L Q b h l q e ?", format)

        super().__init__()
        self.fmtstr = endianity+format
        self.length = struct.calcsize(endianity+format)
        self.packer = struct.Struct(endianity+format) # <---- culprit

TypeError: can't pickle Struct objects

TypeError: can't pickle Struct objects

python-3.x

struct

pickle

dill

cloudpickle