钻井错误。它在内置函数中找不到

Question

我正在尝试提取一个存储在字典中的 class 对象，但由于某些原因我不能这样做。

    local_env = {}
    global_env = {}
    exec(x, global_env, local_env)
    dill.dumps(local_env['Human'])

澄清一下，x 是 normal/simple class 定义的字符串，"Human" 是 class 的名称。当我运行上面的代码时出现以下错误。

------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
/usr/local/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/pickle.py in _getattribute(obj, name)
    267             parent = obj
--> 268             obj = getattr(obj, subpath)
    269         except AttributeError:

AttributeError: module 'builtins' has no attribute 'Human'

During handling of the above exception, another exception occurred:

AttributeError                            Traceback (most recent call last)
/usr/local/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/pickle.py in save_global(self, obj, name)
    906             module = sys.modules[module_name]
--> 907             obj2, parent = _getattribute(module, name)
    908         except (ImportError, KeyError, AttributeError):

/usr/local/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/pickle.py in _getattribute(obj, name)
    270             raise AttributeError("Can't get attribute {!r} on {!r}"
--> 271                                  .format(name, obj))
    272     return obj, parent

AttributeError: Can't get attribute 'Human' on <module 'builtins' (built-in)>

During handling of the above exception, another exception occurred:

PicklingError                             Traceback (most recent call last)
<ipython-input-27-6ce5592364a6> in <module>()
----> 1 dill.dumps(local_env['Human'])

/usr/local/lib/python3.5/site-packages/dill/dill.py in dumps(obj, protocol, byref, fmode, recurse)
    241     """pickle an object to a string"""
    242     file = StringIO()
--> 243     dump(obj, file, protocol, byref, fmode, recurse)#, strictio)
    244     return file.getvalue()
    245 

/usr/local/lib/python3.5/site-packages/dill/dill.py in dump(obj, file, protocol, byref, fmode, recurse)
    234             return
    235     # end hack
--> 236     pik.dump(obj)
    237     stack.clear()  # clear record of 'recursion-sensitive' pickled objects
    238     return

/usr/local/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/pickle.py in dump(self, obj)
    406         if self.proto >= 4:
    407             self.framer.start_framing()
--> 408         self.save(obj)
    409         self.write(STOP)
    410         self.framer.end_framing()

/usr/local/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/pickle.py in save(self, obj, save_persistent_id)
    473         f = self.dispatch.get(t)
    474         if f is not None:
--> 475             f(self, obj) # Call unbound method with explicit self
    476             return
    477 

/usr/local/lib/python3.5/site-packages/dill/dill.py in save_type(pickler, obj)
   1229        #print ("%s\n%s" % (type(obj), obj.__name__))
   1230        #print ("%s\n%s" % (obj.__bases__, obj.__dict__))
-> 1231         StockPickler.save_global(pickler, obj)
   1232         log.info("# T4")
   1233     return

/usr/local/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/pickle.py in save_global(self, obj, name)
    909             raise PicklingError(
    910                 "Can't pickle %r: it's not found as %s.%s" %
--> 911                 (obj, module_name, name))
    912         else:
    913             if obj2 is not obj:

PicklingError: Can't pickle <class 'Human'>: it's not found as builtins.Human

Answer 1

我是 dill 的作者。简短的回答是它不起作用，因为 dill 不知道如何。

>>> import dill
>>> local_env = {}
>>> global_env = {}
>>> x = "class Human(object):\n  pass"
>>> exec(x, global_env, local_env)
>>> local_env
{'Human': <class 'Human'>}
>>> dill.dumps(local_env['Human'])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/mmckerns/lib/python2.7/site-packages/dill-0.2.5.dev0-py2.7.egg/dill/dill.py", line 243, in dumps
    dump(obj, file, protocol, byref, fmode, recurse)#, strictio)
  File "/Users/mmckerns/lib/python2.7/site-packages/dill-0.2.5.dev0-py2.7.egg/dill/dill.py", line 236, in dump
    pik.dump(obj)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 224, in dump
    self.save(obj)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/Users/mmckerns/lib/python2.7/site-packages/dill-0.2.5.dev0-py2.7.egg/dill/dill.py", line 1231, in save_type
    StockPickler.save_global(pickler, obj)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 754, in save_global
    (obj, module, name))
pickle.PicklingError: Can't pickle <class 'Human'>: it's not found as __builtin__.Human

较长的答案是 pickle 只能序列化在文件内部定义的 classes，但不能在解释器中，或者更一般地在 __main__ 中。但是，dill 通常可以 pickle 这类动态构建的 classes，因为它将 __main__ 视为文件，并解析 python 的内置缓冲区以获取class 定义（除其他外）。将 class 定义隐藏在一个字符串中，然后使用 exec 生成 class ......好吧，这是 dill 还不能处理的情况。出于同样的原因，dill 也不能 pickle 定义在 exec 中的函数。然而，这两种情况都是有效的。

已经有函数的工单(https://github.com/uqfoundation/dill/issues/145)...所以我将在工单上的这个问题上添加一个link。

Answer 2

import dill
import builtins
x = """
def test(x, y):
    return x + y
"""
env = {}
env['__builtins__'] = builtins
local_env = Environment()
exec(x, env, local_env)
exec("__builtins__.{0} = {0}".format('test'), env, local_env)
test = dill.loads(dill.dumps(local_env['test']))

使用 Mike 给出的答案作为灵感，我想出了一个解决方法。

钻井错误。它在内置函数中找不到

Dilling Error. It's not found in builtins

python

pickle

dill