pyarrow PlasmaStoreFull 的导入问题

Question

我尝试运行使用来自 python3.7 的 pandaralell 进行一些工作时得到 'cannot import name PlasmaStoreFull from pyarrow.lib'。我试过 un/reinstalling pyarrow，它本身似乎导入 ok 。我没有在此处或一般 google 搜索中找到对这种特定情况的参考。

(p37) jeremyr@bolt:$ pip3 uninstall pyarrow 
Uninstalling pyarrow-0.15.0:
  Would remove:
    /home/jeremyr/p37/bin/plasma_store
    /home/jeremyr/p37/lib/python3.7/site-packages/pyarrow-0.15.0.dist-info/*
    /home/jeremyr/p37/lib/python3.7/site-packages/pyarrow/*
Proceed (y/n)? y
  Successfully uninstalled pyarrow-0.15.0
(p37) jeremyr@bolt:$ pip3 install pyarrow
Collecting pyarrow
  Using cached https://files.pythonhosted.org/packages/02/61/62a74c9ca255dbe8cdff62ea1e9c8f0007f7ce809fccc02d7bf95dd19313/pyarrow-0.15.0-cp37-cp37m-manylinux2010_x86_64.whl
Requirement already satisfied: six>=1.0.0 in /home/jeremyr/p37/lib/python3.7/site-packages (from pyarrow) (1.12.0)
Requirement already satisfied: numpy>=1.14 in /home/jeremyr/p37/lib/python3.7/site-packages (from pyarrow) (1.17.2)
Installing collected packages: pyarrow
Successfully installed pyarrow-0.15.0
(p37) jeremyr@bolt:$ ipython 
Python 3.7.0 (default, Sep 23 2019, 10:05:28) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.8.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import pyarrow                                                                                                                          

In [2]: pyarrow.__version__                                                                                                                     
Out[2]: '0.15.0'

In [3]: from pyarrow.lib import PlasmaStoreFull                                                                                                 
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-3-c640508e3f5e> in <module>
----> 1 from pyarrow.lib import PlasmaStoreFull

ImportError: cannot import name 'PlasmaStoreFull' from 'pyarrow.lib' (/home/jeremyr/p37/lib/python3.7/site-packages/pyarrow/lib.cpython-37m-x86_64-linux-gnu.so)

Answer 1

看起来 PlasmaStoreFull 从 v0.14 中的 pyarrow.lib 移到了 v0.15 中的 pyarrow.plasma - 以下内容在正确的时间做了正确的事情

try:
    # Pyarrow version > 0.14
    from pyarrow.plasma import PlasmaStoreFull as _PlasmaStoreFull
except ImportError:
    # Pyarrow version <= 0.14
    from pyarrow.lib import PlasmaStoreFull as _PlasmaStoreFull

见this bugfix

pyarrow PlasmaStoreFull 的导入问题

import trouble with pyarrow PlasmaStoreFull

python-import

python-3.x

pyarrow