ImportError: The curl client requires the pycurl library Docker + Django

Question

我正在 docker 化 Django 2.2 应用程序并使用 Pipenv 进行环境管理。我想将 SQS 用作 Django 芹菜的代理。

我已经使用 Pipenv 安装了 pycurl 库

[packages]
...
pycurl = "*"

当我在本地运行芹菜时

pipenv run celery -A qcg worker -l info

它有效，但是当我运行使用 docker 图像时

docker run app:latest celery -A qcg worker -l info

报错

ImportError: The curl client requires the pycurl library.

Docerkfile中的命令用于安装依赖[=19=]

RUN set -ex \
    && BUILD_DEPS=" \
    build-essential \
    libpcre3-dev \
    libpq-dev \
    libcurl4-openssl-dev libssl-dev \
    " \
    && apt-get update && apt-get install -y --no-install-recommends $BUILD_DEPS \
    && export PYCURL_SSL_LIBRARY=nss \
    && pipenv install --deploy --system \
    \
    && apt-get purge -y --auto-remove -o APT::AutoRemove::RecommendsImportant=false $BUILD_DEPS 
    \
    && rm -rf /var/lib/apt/lists/*

完整的错误日志

[2020-05-25 17:16:22,216: CRITICAL/MainProcess] Unrecoverable error: ImportError('The curl client requires the pycurl library.')
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/kombu/asynchronous/http/__init__.py", line 20, in get_client
    return hub._current_http_client
AttributeError: 'Hub' object has no attribute '_current_http_client'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/celery/worker/worker.py", line 205, in start
    self.blueprint.start(self)
  File "/usr/local/lib/python3.7/site-packages/celery/bootsteps.py", line 119, in start
    step.start(parent)
  File "/usr/local/lib/python3.7/site-packages/celery/bootsteps.py", line 369, in start
    return self.obj.start()
  File "/usr/local/lib/python3.7/site-packages/celery/worker/consumer/consumer.py", line 318, in start
    blueprint.start(self)
  File "/usr/local/lib/python3.7/site-packages/celery/bootsteps.py", line 119, in start
    step.start(parent)
  File "/usr/local/lib/python3.7/site-packages/celery/worker/consumer/consumer.py", line 599, in start
    c.loop(*c.loop_args())
  File "/usr/local/lib/python3.7/site-packages/celery/worker/loops.py", line 83, in asynloop
    next(loop)
  File "/usr/local/lib/python3.7/site-packages/kombu/asynchronous/hub.py", line 301, in create_loop
    item()
  File "/usr/local/lib/python3.7/site-packages/vine/promises.py", line 170, in __call__
    return self.throw()
  File "/usr/local/lib/python3.7/site-packages/vine/promises.py", line 167, in __call__
    retval = fun(*final_args, **final_kwargs)
  File "/usr/local/lib/python3.7/site-packages/kombu/transport/SQS.py", line 390, in _schedule_queue
    queue, callback=promise(self._loop1, (queue,)),
  File "/usr/local/lib/python3.7/site-packages/kombu/transport/SQS.py", line 406, in _get_bulk_async
    return self._get_async(queue, maxcount, callback=callback)
  File "/usr/local/lib/python3.7/site-packages/kombu/transport/SQS.py", line 416, in _get_async
    qname, count=count, connection=self.asynsqs(queue=qname),
  File "/usr/local/lib/python3.7/site-packages/kombu/transport/SQS.py", line 566, in asynsqs
    region=self.region
  File "/usr/local/lib/python3.7/site-packages/kombu/asynchronous/aws/sqs/connection.py", line 27, in __init__
    **kwargs
  File "/usr/local/lib/python3.7/site-packages/kombu/asynchronous/aws/connection.py", line 194, in __init__
    **http_client_params)
  File "/usr/local/lib/python3.7/site-packages/kombu/asynchronous/aws/connection.py", line 151, in __init__
    self._httpclient = http_client or get_client()
  File "/usr/local/lib/python3.7/site-packages/kombu/asynchronous/http/__init__.py", line 22, in get_client
    client = hub._current_http_client = Client(hub, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/kombu/asynchronous/http/__init__.py", line 13, in Client
    return CurlClient(hub, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/kombu/asynchronous/http/curl.py", line 43, in __init__
    raise ImportError('The curl client requires the pycurl library.')
ImportError: The curl client requires the pycurl library.

Answer 1

此问题与 libcurl4-nss-dev libssl-dev 安装有关。

在下面的安装脚本中

RUN set -ex \
    && BUILD_DEPS=" \
    build-essential \
    libpcre3-dev \
    libpq-dev \
    libcurl4-openssl-dev libssl-dev \
    " \
    && apt-get update && apt-get install -y --no-install-recommends $BUILD_DEPS \
    && export PYCURL_SSL_LIBRARY=nss \
    && pipenv install --deploy --system \
    \
    && apt-get purge -y --auto-remove -o APT::AutoRemove::RecommendsImportant=false $BUILD_DEPS 
    \
    && rm -rf /var/lib/apt/lists/* ```

libcurl4-nss-dev libssl-dev 安装在 Pipenv install 之后删除。所以找不到包库。

分离安装后 libcurl4-nss-dev libssl-dev 适合我。

RUN apt-get install -y --no-install-recommends libcurl4-nss-dev libssl-dev

Answer 2

我遇到了同样的错误，这对我有用：pip install celery[sqs]

ImportError: The curl client requires the pycurl library Docker + Django

ImportError: The curl client requires the pycurl library Docker + Django

django

pycurl

amazon-sqs

django-celery

docker