需要（但不使用）cv2 时无法从 joblib 调用 Nonetype

Question

这个错误很奇怪，每当我在单元测试中使用 n_jobs > 1 的 sklearn 的 Kmeans 时，同时使用 setuptools 要求 cv2 导致 None 被 joblib 调用。

一个最小的失败示例：

setup.py:

from setuptools import setup

setup(
    name = "libbla",
# Removing "cv2" resolves the issue :S
    install_requires = ["numpy", "scikit-learn", "cv2"],
    test_suite = 'tests'
)

tests/some_test.py:

from sklearn.cluster import KMeans

# also fails without importing sklearn and cv2, just want them for the version numbers.
import unittest, numpy, sklearn, cv2

print("cv2", cv2.__version__)
print("np", numpy.__version__)
print("skl", sklearn.__version__)

class TestFeatureCreator(unittest.TestCase):
    def test_kmeans_2_features(self):
        KMeans(n_clusters = 2, n_jobs = 4).fit_predict(numpy.random.randn(360000, 3))

tests/__init__.py 为空。

然后每当我运行 python2.7 setup.py test，我得到以下输出：

$ python2.7 setup.py test
running test
Searching for cv2
Best match: cv2 1.0
Processing cv2-1.0-py2.7.egg

Using /home/herbert/Spyder/bla/.eggs/cv2-1.0-py2.7.egg
running egg_info
writing requirements to libbla.egg-info/requires.txt
writing libbla.egg-info/PKG-INFO
writing top-level names to libbla.egg-info/top_level.txt
writing dependency_links to libbla.egg-info/dependency_links.txt
reading manifest file 'libbla.egg-info/SOURCES.txt'
writing manifest file 'libbla.egg-info/SOURCES.txt'
running build_ext
('cv2', '3.0.0-dev')
('np', '1.10.1')
('skl', '0.16.1')
test_kmeans_2_features (tests.some_test.TestFeatureCreator) ... ok

----------------------------------------------------------------------
Ran 1 test in 1.156s

OK
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/usr/lib/python2.7/atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "/usr/lib/python2.7/dist-packages/joblib/pool.py", line 535, in <lambda>
    atexit.register(lambda: delete_folder(pool_folder))
TypeError: 'NoneType' object is not callable
Error in sys.exitfunc:
Traceback (most recent call last):
  File "/usr/lib/python2.7/atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "/usr/lib/python2.7/dist-packages/joblib/pool.py", line 535, in <lambda>
    atexit.register(lambda: delete_folder(pool_folder))
TypeError: 'NoneType' object is not callable
$

我不确定这是 opencv2、sklearn 还是 numpy 的错误，这就是我来这里的原因。有人知道这里发生了什么吗？

一些特点：

从 install_requires 中删除 "cv2" 会删除两个错误
同上不运行宁 Kmeans
Kmeans 同上，未提供 n_jobs。
在错误之前打印 atexit.register、delete_folder 和 pool_folder 时，其中 none 个是 None。

无法重现错误时也请评论:)

Answer 1

此问题的修复已合并到 joblib master 中（将包含在下一个版本 0.10 中）：

https://github.com/joblib/joblib/pull/329

需要（但不使用）cv2 时无法从 joblib 调用 Nonetype

Nonetype not callable from joblib when requiring (but not using) cv2

python-2.7

numpy

scikit-learn

opencv3.0

joblib