Solve python ValueError: max_workers must be <= 61 when running pre-commit?

Question

我正在使用 Django 开发 ERP，我想对我的项目使用预提交。

我已经安装了pre-commit, black, flake8, flake8-black。这是我的

.pre-commit-config.yaml

文件配置内容

repos:
-   repo: https://github.com/pre-commit/pre-commit-hooks
rev: v2.3.0
hooks:
-   id: check-yaml
-   id: end-of-file-fixer
-   id: trailing-whitespace
-   repo: https://github.com/psf/black
rev: 19.3b0
hooks:
-   id: black
    language_version: python3.7

当我提交我的代码时它给我这个错误

Check 
Yaml...............................................................Passed
Fix End of Files.........................................................Failed
- hook id: end-of-file-fixer
- exit code: 1
- files were modified by this hook

Fixing vms/movement/test/test_model.py
Fixing vms/payment/test/test_forms.py

Trim Trailing Whitespace.................................................Passed
black....................................................................Failed
- hook id: black
- exit code: 1

Traceback (most recent call last):
  File "C:\Program Files\Python37\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\Program Files\Python37\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\Diaa\.cache\pre-commit\repo4t4vqwkg\py_env-python3\Scripts\black.EXE\__main__.py", line 7, in <module>
  File "c:\users\diaa\.cache\pre-commit\repo4t4vqwkg\py_env-python3\lib\site-packages\black.py", line 3754, in patched_main
    main()
  File "c:\users\diaa\.cache\pre-commit\repo4t4vqwkg\py_env-python3\lib\site-packages\click\core.py", line 1137, in __call__
    return self.main(*args, **kwargs)
  File "c:\users\diaa\.cache\pre-commit\repo4t4vqwkg\py_env-python3\lib\site-packages\click\core.py", line 1062, in main
    rv = self.invoke(ctx)
  File "c:\users\diaa\.cache\pre-commit\repo4t4vqwkg\py_env-python3\lib\site-packages\click\core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "c:\users\diaa\.cache\pre-commit\repo4t4vqwkg\py_env-python3\lib\site-packages\click\core.py", line 763, in invoke
    return __callback(*args, **kwargs)
  File "c:\users\diaa\.cache\pre-commit\repo4t4vqwkg\py_env-python3\lib\site-packages\click\decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "c:\users\diaa\.cache\pre-commit\repo4t4vqwkg\py_env-python3\lib\site-packages\black.py", line 435, in main
    executor = ProcessPoolExecutor(max_workers=os.cpu_count())
  File "C:\Program Files\Python37\lib\concurrent\futures\process.py", line 523, in __init__
    f"max_workers must be <= {_MAX_WINDOWS_WORKERS}")
ValueError: max_workers must be <= 61

这是我的点数列表

-------------------------------- --------
appdirs                          1.4.4
asgiref                          3.3.4
aspy.refactor-imports            2.2.0
atomicwrites                     1.4.0
attrs                            21.2.0
Babel                            2.9.1
black                            21.6b0
cached-property                  1.5.2
cfgv                             3.3.0
click                            8.0.1
colorama                         0.4.4
coverage                         5.5
distlib                          0.3.2
Django                           3.2
django-autocomplete-light        3.8.2
django-autoslug                  1.9.8
django-colorfield                0.4.1
django-countries                 7.2.1
django-crispy-forms              1.11.0
django-datatables-view           1.19.1
django-excel                     0.0.10
django-guardian                  2.4.0
django-ipware                    3.0.2
django-js-asset                  1.2.2
django-login-required-middleware 0.5.0
django-mptt                      0.12.0
django-nested-inline             0.4.3
django-querysetsequence          0.14
django-rest-authtoken            2.1.3
django-timezone-field            4.1.1
djangorestframework              3.12.2
docopt                           0.6.2
enum34                           1.1.10
Faker                            8.8.0
filelock                         3.0.12
flake8                           3.9.2
flake8-black                     0.2.1
fpdf2                            2.4.1
identify                         2.2.10
importlib-metadata               4.5.0
iniconfig                        1.1.1
IP2Location                      8.5.1
isort                            5.9.1
lml                              0.0.9
mccabe                           0.6.1
mixer                            7.1.2
mypy-extensions                  0.4.3
nodeenv                          1.6.0
num2words                        0.5.10
packaging                        20.9
pathspec                         0.8.1
Pillow                           8.1.0
pip                              21.1.1
pluggy                           0.13.1
pre-commit                       2.13.0
psycopg2                         2.8.6
py                               1.10.0
pycodestyle                      2.7.0
pycountry                        20.7.3
pydocstyle                       6.1.1
pyexcel                          0.6.2
pyexcel-io                       0.6.4
pyexcel-webio                    0.1.2
pyflakes                         2.3.1
pynput                           1.7.3
pyparsing                        2.4.7
pytest                           6.2.4
pytest-django                    4.4.0
python-dateutil                  2.8.1
pytz                             2021.1
PyYAML                           5.4.1
regex                            2021.4.4
reportbro-fpdf                   1.7.10
reportbro-lib                    1.6.0
reportlab                        3.5.67
seed-isort-config                2.2.0
setuptools                       57.0.0
setuptools-scm                   3.4.3
simpleeval                       0.9.10
six                              1.15.0
snowballstemmer                  2.1.0
sqlparse                         0.4.1
text-unidecode                   1.3
texttable                        0.8.2
toml                             0.10.2
tox                              3.23.1
typed-ast                        1.4.3
typing                           3.7.4.3
typing-extensions                3.10.0.0
virtualenv                       20.4.7
XlsxWriter                       1.4.3
xlwt                             1.3.0
zipp                             3.4.1

当我运行 os.cpu_count() 它给了我 64 个处理器。我使用 windows 服务器 2016 数据中心。

请帮我解决这个问题。

注意:我用的是python3.7

Answer 1

这是 windows 上 cpython 的一个已知问题。当 black 尝试运行 >60 核心机器上的多个 worker 时会发生错误，因为 os.cpu_count() 给出的默认 process worker 数量打破了其他一些 windows 限制（等待进程的数量？我'我不太确定）。 Black >=19.10b0 已修复此问题，因此如果可以，请尝试在预提交配置中更新 black 的版本？

Python 错误报告：https://bugs.python.org/issue26903
此 PR 应用的黑色修复程序：https://github.com/psf/black/pull/838

Answer 2

您可以尝试修补此行site-packages\black.py, line 435

    executor = ProcessPoolExecutor(max_workers=os.cpu_count())

至

   executor = ProcessPoolExecutor(max_workers=min(32, os.cpu_count() + 4))

我相信 Python 3.8 及更高版本中的 ProcessPoolExecutor 基本上是这样的。

Solve python ValueError: max_workers must be <= 61 when running pre-commit?

Solve python ValueError: max_workers must be <= 61 when running pre-commit?

python

django

pre-commit-hook

python-black

pre-commit.com