mognoengine 和 bson 包不能一起工作
mognoengine and bson package not work together
我有一个项目依赖于 mongoengine and I am using one library which was using bson 库。
结构如下。
bsNotify/
├── setup.cfg
├── setup.py
└── src
└── bsnotify
├── __init__.py
└── resources.py
setup.py
$ cat bsNotify/setup.py
"""Base module setup."""
from setuptools import setup
setup(
setup_requires=['pbr'],
pbr=True
)
setup.cfg
$ cat bsNotify/setup.cfg
[metadata]
name = bsNotify
classifiers =
License :: N/A :: N/A
Programming Language :: Python :: 3.7
[options]
zip_safe = False
include_package_data = True
python_requires = >= 3.7
install_requires =
mongoengine
bson
package_dir=
=src
packages=find:
[options.packages.find]
where=src
[tool:wheel]
universal = 1
[flake8]
exclude =
venv,
.tox,
.git,
__pycache__,
*.pyc,
*.egg-info,
.cache,
.eggs,
max-line-length = 80
[tox]
envlist = py37,unittest,lint
[testenv]
basepython=python3.7
deps =
ipython
pylint
pytest
pytest-cov
pytest-xdist
flake8
flake8-docstrings
[testenv:unittest]
commands=
pytest -v -s -n auto -l --cov=bsnotify --cov-report term-missing --cov-report xml --no-cov-on-fail tests/unit
[testenv:lint]
commands=
flake8 src/bsnotify
pylint src/bsnotify
src/bsnotify/__init__.py
$ cat bsNotify/src/bsnotify/__init__.py
src/bsnotify/resources.py
$ cat bsNotify/src/bsnotify/resources.py
src 中的文件为空。
当我 运行 tox 命令时，它会创建一个用于测试的 virtualenv。
$ tox -v -e unittest --notest
using tox.ini: /Users/myuser/bsNotify/setup.cfg (pid 30954)
using tox-3.14.6 from /Users/myuser/.pyenv/versions/3.7.4/lib/python3.7/site-packages/tox/__init__.py (pid 30954)
GLOB sdist-make: /Users/myuser/bsNotify/setup.py
[30956] /Users/myuser/bsNotify$ /Users/myuser/.pyenv/versions/3.7.4/bin/python3.7 setup.py sdist --formats=zip --dist-dir /Users/myuser/bsNotify/.tox/dist >.tox/log/GLOB-0.log
package .tmp/package/1/bsNotify-0.1.2.dev2.zip links to dist/bsNotify-0.1.2.dev2.zip (/Users/myuser/bsNotify/.tox)
unittest cannot reuse: no previous config /Users/myuser/bsNotify/.tox/unittest/.tox-config1
unittest create: /Users/myuser/bsNotify/.tox/unittest
[30992] /Users/myuser/bsNotify/.tox$ /Users/myuser/.pyenv/versions/3.7.4/bin/python3.7 -m virtualenv --no-download --python /Users/myuser/.pyenv/versions/3.7.4/bin/python3.7 unittest >unittest/log/unittest-0.log
unittest installdeps: ipython, pylint, pytest, pytest-cov, pytest-xdist, flake8, flake8-docstrings
[30993] /Users/myuser/bsNotify$ /Users/myuser/bsNotify/.tox/unittest/bin/python -m pip install ipython pylint pytest pytest-cov pytest-xdist flake8 flake8-docstrings >.tox/unittest/log/unittest-1.log
unittest inst: /Users/myuser/bsNotify/.tox/.tmp/package/1/bsNotify-0.1.2.dev2.zip
write config to /Users/myuser/bsNotify/.tox/unittest/.tox-config1 as '7186e9f46c94b6d9f7dde810ce83f8fe46740d9d42f2863cfd063c4c6f4e4a88 /Users/myuser/.pyenv/versions/3.7.4/bin/python3.7\n3.14.6 0 0 0\n00000000000000000000000000000000 ipython\n00000000000000000000000000000000 pylint\n00000000000000000000000000000000 pytest\n00000000000000000000000000000000 pytest-cov\n00000000000000000000000000000000 pytest-xdist\n00000000000000000000000000000000 flake8\n00000000000000000000000000000000 flake8-docstrings'
[31000] /Users/myuser/bsNotify$ /Users/myuser/bsNotify/.tox/unittest/bin/python -m pip install --exists-action w .tox/.tmp/package/1/bsNotify-0.1.2.dev2.zip >.tox/unittest/log/unittest-2.log
[31030] /Users/myuser/bsNotify$ /Users/myuser/bsNotify/.tox/unittest/bin/python -m pip freeze >.tox/unittest/log/unittest-3.log
unittest installed: apipkg==1.5,appnope==0.1.0,astroid==2.4.2,attrs==19.3.0,backcall==0.1.0,bsNotify==0.1.2.dev2,bson==0.5.10,coverage==5.1,decorator==4.4.2,execnet==1.7.1,flake8==3.8.3,flake8-docstrings==1.5.0,importlib-metadata==1.6.1,ipython==7.15.0,ipython-genutils==0.2.0,isort==4.3.21,jedi==0.17.0,lazy-object-proxy==1.4.3,mccabe==0.6.1,mongoengine==0.20.0,more-itertools==8.3.0,packaging==20.4,parso==0.7.0,pexpect==4.8.0,pickleshare==0.7.5,pluggy==0.13.1,prompt-toolkit==3.0.5,ptyprocess==0.6.0,py==1.8.1,pycodestyle==2.6.0,pydocstyle==5.0.2,pyflakes==2.2.0,Pygments==2.6.1,pylint==2.5.3,pymongo==3.10.1,pyparsing==2.4.7,pytest==5.4.3,pytest-cov==2.9.0,pytest-forked==1.1.3,pytest-xdist==1.32.0,python-dateutil==2.8.1,six==1.15.0,snowballstemmer==2.0.0,toml==0.10.1,traitlets==4.3.3,typed-ast==1.4.1,wcwidth==0.2.4,wrapt==1.12.1,zipp==3.1.0
_______________________________________________________ summary _______________________________________________________
unittest: skipped tests
congratulations :)
然后我尝试在新创建的virtualenv中访问mongoengine，它报错。
$ .tox/unittest/bin/python -c "from mongoengine import connection"
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/Users/myuser/bsNotify/.tox/unittest/lib/python3.7/site-packages/mongoengine/__init__.py", line 2, in <module>
from mongoengine import connection
File "/Users/myuser/bsNotify/.tox/unittest/lib/python3.7/site-packages/mongoengine/connection.py", line 1, in <module>
from pymongo import MongoClient, ReadPreference, uri_parser
File "/Users/myuser/bsNotify/.tox/unittest/lib/python3.7/site-packages/pymongo/__init__.py", line 77, in <module>
from pymongo.collection import ReturnDocument
File "/Users/myuser/bsNotify/.tox/unittest/lib/python3.7/site-packages/pymongo/collection.py", line 20, in <module>
from bson.code import Code
File "/Users/myuser/bsNotify/.tox/unittest/lib/python3.7/site-packages/bson/code.py", line 18, in <module>
from bson.py3compat import abc, string_type, PY3, text_type
ImportError: cannot import name 'abc' from 'bson.py3compat' (/Users/myuser/bsNotify/.tox/unittest/lib/python3.7/site-packages/bson/py3compat.py)
当我单独使用 mongoengine 和 bson 时，它工作正常。
$ python -m venv .venv
$ .venv/bin/pip install mongoengine
Collecting mongoengine
Using cached https://files.pythonhosted.org/packages/7d/bd/9a7239b0032157f948c69febdf71dd82cb54fcd2499077300496a3f076c9/mongoengine-0.20.0-py3-none-any.whl
Collecting pymongo<4.0,>=3.4 (from mongoengine)
Using cached https://files.pythonhosted.org/packages/23/cd/27fbc08f0bd835b4735504a758756e979b42c5bc9ebaac5ed3c2cbffd83f/pymongo-3.10.1-cp37-cp37m-macosx_10_9_x86_64.whl
Installing collected packages: pymongo, mongoengine
Successfully installed mongoengine-0.20.0 pymongo-3.10.1
$ .venv/bin/python -c "from mongoengine import connection"
$ .venv/bin/pip install bson
Collecting bson
Using cached https://files.pythonhosted.org/packages/4d/53/7c534a38850f2252275d7f949aed2219095e90df1e2d180a9c8ed139e499/bson-0.5.10.tar.gz
Collecting python-dateutil>=2.4.0 (from bson)
Using cached https://files.pythonhosted.org/packages/d4/70/d60450c3dd48ef87586924207ae8907090de0b306af2bce5d134d78615cb/python_dateutil-2.8.1-py2.py3-none-any.whl
Collecting six>=1.9.0 (from bson)
Using cached https://files.pythonhosted.org/packages/ee/ff/48bde5c0f013094d729fe4b0316ba2a24774b3ff1c52d924a8a4cb04078a/six-1.15.0-py2.py3-none-any.whl
Installing collected packages: six, python-dateutil, bson
Running setup.py install for bson ... done
Successfully installed bson-0.5.10 python-dateutil-2.8.1 six-1.15.0
You are using pip version 19.0.3, however version 20.2b1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
$ .venv/bin/python -c "from mongoengine import connection"
它检查两个 virtualenv 之间的 pip freeze。我发现的唯一区别是在 .tox env 中安装了 bsNotify==0.1.2.dev2 而在 .venv env.
中不存在
什么使 bson 更改文件 py3compat.py？
改变这个
install_requires =
mongoengine
bson
至此
install_requires =
mongoengine
pymongo
或完全删除（mongo引擎已经需要 pymongo
install_requires =
mongoengine
requirements=bson 与
相同
pip install bson
安装此第 3 方包，其中不包含 MongoDB 包中的所有好东西
https://pypi.org/project/bson/
pymongo（官方MongoDB驱动）包含一个bson包。我会将 pymongo 文件夹和 bson 文件夹放入您的站点包中。
bson（第三方 bson 实现）也想将 bson 文件夹放入您的站点包中。
安装 pymongo 和安装 bson 会在您的站点包中产生文件夹冲突。 PyMongo 只知道如何使用它自己的实现，这就是为什么您看到 pymongo 正在寻找 py3compat，它不是第 3 方 bson 包的一部分。
根据您的安装顺序，它会安装 mongo 引擎（和 pymongo 作为依赖项）然后安装 bson（覆盖 MongoDB 与 py 打包的 bson 实现mongo)。 Pymongo 调用了 py3compat 期望它在那里，但是 3td 方没有实现它（以及其他一些东西）。
pip install pymongo
https://pypi.org/project/pymongo/
如 pymongo pypi 页面所述
Do not install the “bson” package from pypi. PyMongo comes with its
own bson package; doing “easy_install bson” installs a third-party
package that is incompatible with PyMongo.
我有一个项目依赖于 mongoengine and I am using one library which was using bson 库。
结构如下。
bsNotify/
├── setup.cfg
├── setup.py
└── src
└── bsnotify
├── __init__.py
└── resources.py
setup.py
$ cat bsNotify/setup.py
"""Base module setup."""
from setuptools import setup
setup(
setup_requires=['pbr'],
pbr=True
)
setup.cfg
$ cat bsNotify/setup.cfg
[metadata]
name = bsNotify
classifiers =
License :: N/A :: N/A
Programming Language :: Python :: 3.7
[options]
zip_safe = False
include_package_data = True
python_requires = >= 3.7
install_requires =
mongoengine
bson
package_dir=
=src
packages=find:
[options.packages.find]
where=src
[tool:wheel]
universal = 1
[flake8]
exclude =
venv,
.tox,
.git,
__pycache__,
*.pyc,
*.egg-info,
.cache,
.eggs,
max-line-length = 80
[tox]
envlist = py37,unittest,lint
[testenv]
basepython=python3.7
deps =
ipython
pylint
pytest
pytest-cov
pytest-xdist
flake8
flake8-docstrings
[testenv:unittest]
commands=
pytest -v -s -n auto -l --cov=bsnotify --cov-report term-missing --cov-report xml --no-cov-on-fail tests/unit
[testenv:lint]
commands=
flake8 src/bsnotify
pylint src/bsnotify
src/bsnotify/__init__.py
$ cat bsNotify/src/bsnotify/__init__.py
src/bsnotify/resources.py
$ cat bsNotify/src/bsnotify/resources.py
src 中的文件为空。
当我 运行 tox 命令时，它会创建一个用于测试的 virtualenv。
$ tox -v -e unittest --notest
using tox.ini: /Users/myuser/bsNotify/setup.cfg (pid 30954)
using tox-3.14.6 from /Users/myuser/.pyenv/versions/3.7.4/lib/python3.7/site-packages/tox/__init__.py (pid 30954)
GLOB sdist-make: /Users/myuser/bsNotify/setup.py
[30956] /Users/myuser/bsNotify$ /Users/myuser/.pyenv/versions/3.7.4/bin/python3.7 setup.py sdist --formats=zip --dist-dir /Users/myuser/bsNotify/.tox/dist >.tox/log/GLOB-0.log
package .tmp/package/1/bsNotify-0.1.2.dev2.zip links to dist/bsNotify-0.1.2.dev2.zip (/Users/myuser/bsNotify/.tox)
unittest cannot reuse: no previous config /Users/myuser/bsNotify/.tox/unittest/.tox-config1
unittest create: /Users/myuser/bsNotify/.tox/unittest
[30992] /Users/myuser/bsNotify/.tox$ /Users/myuser/.pyenv/versions/3.7.4/bin/python3.7 -m virtualenv --no-download --python /Users/myuser/.pyenv/versions/3.7.4/bin/python3.7 unittest >unittest/log/unittest-0.log
unittest installdeps: ipython, pylint, pytest, pytest-cov, pytest-xdist, flake8, flake8-docstrings
[30993] /Users/myuser/bsNotify$ /Users/myuser/bsNotify/.tox/unittest/bin/python -m pip install ipython pylint pytest pytest-cov pytest-xdist flake8 flake8-docstrings >.tox/unittest/log/unittest-1.log
unittest inst: /Users/myuser/bsNotify/.tox/.tmp/package/1/bsNotify-0.1.2.dev2.zip
write config to /Users/myuser/bsNotify/.tox/unittest/.tox-config1 as '7186e9f46c94b6d9f7dde810ce83f8fe46740d9d42f2863cfd063c4c6f4e4a88 /Users/myuser/.pyenv/versions/3.7.4/bin/python3.7\n3.14.6 0 0 0\n00000000000000000000000000000000 ipython\n00000000000000000000000000000000 pylint\n00000000000000000000000000000000 pytest\n00000000000000000000000000000000 pytest-cov\n00000000000000000000000000000000 pytest-xdist\n00000000000000000000000000000000 flake8\n00000000000000000000000000000000 flake8-docstrings'
[31000] /Users/myuser/bsNotify$ /Users/myuser/bsNotify/.tox/unittest/bin/python -m pip install --exists-action w .tox/.tmp/package/1/bsNotify-0.1.2.dev2.zip >.tox/unittest/log/unittest-2.log
[31030] /Users/myuser/bsNotify$ /Users/myuser/bsNotify/.tox/unittest/bin/python -m pip freeze >.tox/unittest/log/unittest-3.log
unittest installed: apipkg==1.5,appnope==0.1.0,astroid==2.4.2,attrs==19.3.0,backcall==0.1.0,bsNotify==0.1.2.dev2,bson==0.5.10,coverage==5.1,decorator==4.4.2,execnet==1.7.1,flake8==3.8.3,flake8-docstrings==1.5.0,importlib-metadata==1.6.1,ipython==7.15.0,ipython-genutils==0.2.0,isort==4.3.21,jedi==0.17.0,lazy-object-proxy==1.4.3,mccabe==0.6.1,mongoengine==0.20.0,more-itertools==8.3.0,packaging==20.4,parso==0.7.0,pexpect==4.8.0,pickleshare==0.7.5,pluggy==0.13.1,prompt-toolkit==3.0.5,ptyprocess==0.6.0,py==1.8.1,pycodestyle==2.6.0,pydocstyle==5.0.2,pyflakes==2.2.0,Pygments==2.6.1,pylint==2.5.3,pymongo==3.10.1,pyparsing==2.4.7,pytest==5.4.3,pytest-cov==2.9.0,pytest-forked==1.1.3,pytest-xdist==1.32.0,python-dateutil==2.8.1,six==1.15.0,snowballstemmer==2.0.0,toml==0.10.1,traitlets==4.3.3,typed-ast==1.4.1,wcwidth==0.2.4,wrapt==1.12.1,zipp==3.1.0
_______________________________________________________ summary _______________________________________________________
unittest: skipped tests
congratulations :)
然后我尝试在新创建的virtualenv中访问mongoengine，它报错。
$ .tox/unittest/bin/python -c "from mongoengine import connection"
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/Users/myuser/bsNotify/.tox/unittest/lib/python3.7/site-packages/mongoengine/__init__.py", line 2, in <module>
from mongoengine import connection
File "/Users/myuser/bsNotify/.tox/unittest/lib/python3.7/site-packages/mongoengine/connection.py", line 1, in <module>
from pymongo import MongoClient, ReadPreference, uri_parser
File "/Users/myuser/bsNotify/.tox/unittest/lib/python3.7/site-packages/pymongo/__init__.py", line 77, in <module>
from pymongo.collection import ReturnDocument
File "/Users/myuser/bsNotify/.tox/unittest/lib/python3.7/site-packages/pymongo/collection.py", line 20, in <module>
from bson.code import Code
File "/Users/myuser/bsNotify/.tox/unittest/lib/python3.7/site-packages/bson/code.py", line 18, in <module>
from bson.py3compat import abc, string_type, PY3, text_type
ImportError: cannot import name 'abc' from 'bson.py3compat' (/Users/myuser/bsNotify/.tox/unittest/lib/python3.7/site-packages/bson/py3compat.py)
当我单独使用 mongoengine 和 bson 时，它工作正常。
$ python -m venv .venv
$ .venv/bin/pip install mongoengine
Collecting mongoengine
Using cached https://files.pythonhosted.org/packages/7d/bd/9a7239b0032157f948c69febdf71dd82cb54fcd2499077300496a3f076c9/mongoengine-0.20.0-py3-none-any.whl
Collecting pymongo<4.0,>=3.4 (from mongoengine)
Using cached https://files.pythonhosted.org/packages/23/cd/27fbc08f0bd835b4735504a758756e979b42c5bc9ebaac5ed3c2cbffd83f/pymongo-3.10.1-cp37-cp37m-macosx_10_9_x86_64.whl
Installing collected packages: pymongo, mongoengine
Successfully installed mongoengine-0.20.0 pymongo-3.10.1
$ .venv/bin/python -c "from mongoengine import connection"
$ .venv/bin/pip install bson
Collecting bson
Using cached https://files.pythonhosted.org/packages/4d/53/7c534a38850f2252275d7f949aed2219095e90df1e2d180a9c8ed139e499/bson-0.5.10.tar.gz
Collecting python-dateutil>=2.4.0 (from bson)
Using cached https://files.pythonhosted.org/packages/d4/70/d60450c3dd48ef87586924207ae8907090de0b306af2bce5d134d78615cb/python_dateutil-2.8.1-py2.py3-none-any.whl
Collecting six>=1.9.0 (from bson)
Using cached https://files.pythonhosted.org/packages/ee/ff/48bde5c0f013094d729fe4b0316ba2a24774b3ff1c52d924a8a4cb04078a/six-1.15.0-py2.py3-none-any.whl
Installing collected packages: six, python-dateutil, bson
Running setup.py install for bson ... done
Successfully installed bson-0.5.10 python-dateutil-2.8.1 six-1.15.0
You are using pip version 19.0.3, however version 20.2b1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
$ .venv/bin/python -c "from mongoengine import connection"
它检查两个 virtualenv 之间的 pip freeze。我发现的唯一区别是在 .tox env 中安装了 bsNotify==0.1.2.dev2 而在 .venv env.
什么使 bson 更改文件 py3compat.py？
改变这个
install_requires =
mongoengine
bson
至此
install_requires =
mongoengine
pymongo
或完全删除（mongo引擎已经需要 pymongo
install_requires =
mongoengine
requirements=bson 与
pip install bson
安装此第 3 方包，其中不包含 MongoDB 包中的所有好东西
https://pypi.org/project/bson/
pymongo（官方MongoDB驱动）包含一个bson包。我会将 pymongo 文件夹和 bson 文件夹放入您的站点包中。
bson（第三方 bson 实现）也想将 bson 文件夹放入您的站点包中。
安装 pymongo 和安装 bson 会在您的站点包中产生文件夹冲突。 PyMongo 只知道如何使用它自己的实现，这就是为什么您看到 pymongo 正在寻找 py3compat，它不是第 3 方 bson 包的一部分。
根据您的安装顺序，它会安装 mongo 引擎（和 pymongo 作为依赖项）然后安装 bson（覆盖 MongoDB 与 py 打包的 bson 实现mongo)。 Pymongo 调用了 py3compat 期望它在那里，但是 3td 方没有实现它（以及其他一些东西）。
pip install pymongo
https://pypi.org/project/pymongo/
如 pymongo pypi 页面所述
Do not install the “bson” package from pypi. PyMongo comes with its own bson package; doing “easy_install bson” installs a third-party package that is incompatible with PyMongo.