Celery Task 无法在 Flask docker 容器中使用 Redis

Question

我正在尝试运行烧瓶 docker 容器中的 celery 任务，但在执行 celery 任务时出现如下错误

web_1     |     sock.connect(socket_address)
web_1     | OSError: [Errno 99] Cannot assign requested address
web_1     | 
web_1     | During handling of the above exception, another exception occurred: **[shown below]**

web_1     |   File "/opt/venv/lib/python3.8/site-packages/redis/connection.py", line 571, in connect
web_1     |     raise ConnectionError(self._error_message(e))
web_1     | redis.exceptions.ConnectionError: Error 99 connecting to localhost:6379. Cannot assign requested address.

没有 celery 任务，应用程序运行良好

docker-compose.yml

version: '3'
services:
  web:
    build: ./
    volumes:
      - ./app:/app
    ports:
      - "80:80"
    environment:
      - FLASK_APP=app/main.py
      - FLASK_DEBUG=1
      - 'RUN=flask run --host=0.0.0.0 --port=80'
    depends_on:
      - redis

  redis:
    container_name: redis
    image: redis:6.2.6
    ports:
      - "6379:6379"
    expose: 
      - "6379"

  worker:
    build:
      context: ./
    hostname: worker
    command: "cd /app/routes && celery -A celery_tasks.celery worker --loglevel=info"
    volumes:
      - ./app:/app
    links:
      - redis
    depends_on:
      - redis

main.py

from flask import Flask
from instance import config, exts
from decouple import config as con

def create_app(config_class=config.Config):
    app = Flask(__name__)
    app.config.from_object(config.Config)
    app.secret_key = con('flask_secret_key')
    
    exts.mail.init_app(app)

    from routes.test_route import test_api
    app.register_blueprint(test_api)
    return app

app = create_app()

if __name__ == "__main__":
    app.run(host="0.0.0.0", debug=True, port=80)

我正在使用 Flask 蓝图拆分 api 路由

test_route.py

from flask import Flask, render_template, Blueprint
from instance.exts import celery

test_api = Blueprint('test_api', __name__)


@test_api.route('/test/<string:name>')
def testfnn(name):
    task = celery.send_task('CeleryTask.reverse',args=[name])
    return task.id

Celery 任务也写在单独的文件中

celery_tasks.py

from celery import Celery
from celery.utils.log import get_task_logger
from decouple import config
import time

celery= Celery('tasks', 
                broker = config('CELERY_BROKER_URL'), 
                backend = config('CELERY_RESULT_BACKEND'))

class CeleryTask:
    @celery.task(name='CeleryTask.reverse')
    def reverse(string):
        time.sleep(25)
        return string[::-1]

.env

CELERY_BROKER_URL = 'redis://localhost:6379/0'
CELERY_RESULT_BACKEND = 'redis://localhost:6379/0'

Dockerfile

FROM tiangolo/uwsgi-nginx:python3.8
RUN apt-get update
WORKDIR /app
ENV PYTHONUNBUFFERED 1
ENV VIRTUAL_ENV=/opt/venv
RUN python3 -m venv $VIRTUAL_ENV
ENV PATH="$VIRTUAL_ENV/bin:$PATH"
RUN python -m pip install --upgrade pip
COPY ./requirements.txt /app/requirements.txt
RUN pip install --no-cache-dir --upgrade -r /app/requirements.txt
COPY ./app /app
CMD ["python", "app/main.py"]

requirements.txt

Flask==2.0.3
celery==5.2.3
python-decouple==3.5
Flask-Mail==0.9.1
redis==4.0.2
SQLAlchemy==1.4.32

文件夹结构

提前致谢

Answer 1

在您的 docker-compose.yml 末尾，您可以添加：

networks:
  your_net_name:
    name: your_net_name

并且在每个容器中：

networks:
      - your_net_name

这两个步骤会将所有容器放在同一个网络中。默认情况下 docker 创建一个，但由于我在让它们成为 auto-renamed 时遇到了问题，我认为这种方法可以让您更好地控制。

最后我还要更改您的 env 变量以使用容器地址：

CELERY_BROKER_URL=redis://redis_addr/0
CELERY_RESULT_BACKEND=redis://redis_addr/0

所以您还要将此部分添加到您的 redis 容器中：

hostname: redis_addr

这样 env var 将获得 docker 分配给容器的任何地址。

Celery Task 无法在 Flask docker 容器中使用 Redis

Celery Task not working with redis in flask docker container

python

redis

celery

flask

docker