带有 Supervisord 工人的芹菜不工作隔离

Question

.conf 文件

[program:task1]
    directory=/home/ubuntu/proj1
    command=/usr/bin/python3 /usr/local/bin/celery -A proj1 worker -l info --concurrency=10 -n proj1_worker@%%h
    user=ubuntu
    numprocs=1
    stdout_logfile=/var/log/proj1_celeryd.log
    stderr_logfile=/var/log/proj1_celeryd.log
    autostart=true
    autorestart=true
    startsecs=10
    stopwaitsecs=600
    priority=998

[program:task2]
    directory=/home/ubuntu/proj2/
    command=/usr/bin/python3 /usr/local/bin/celery -A proj2 worker -l info --concurrency=10 -n proj2_worker@%%h
    user=ubuntu
    numprocs=1
    stdout_logfile=/var/log/proj2_celeryd.log
    stderr_logfile=/var/log/proj2_celeryd.log
    autostart=true
    autorestart=true
    startsecs=10
    stopwaitsecs=600
    priority=998

[group:celeryworkers]
programs=task1,task2

proj1_worker 和 proj2_worker 没有被孤立。起初，即使我调用 proj2_worker

，也总是调用 proj1_worker

我不知道我哪里错了。请协助。

提前致谢

Answer 1

首先我真的推荐你对每个项目使用virtualenv。创建 2 个独立的 virtualenvs（您可以指定自己的位置），参见 https://docs.python.org/3/library/venv.html.

python3 -m venv /home/ubuntu/virtualenvs/proj1
python3 -m venv /home/ubuntu/virtualenvs/proj2

激活virtualenv并安装celery：

source /home/ubuntu/virtualenvs/proj1/bin/activate
pip install --upgrade celery

source /home/ubuntu/virtualenvs/proj2/bin/activate
pip install --upgrade celery

您的主管配置应如下所示：

[program:task1]
    directory=/home/ubuntu/proj1
    command=/home/ubuntu/virtualenvs/proj1/bin/celery worker -A proj1 -l info --concurrency=10 -n proj1_worker@%%h
    # ...

[program:task2]
    directory=/home/ubuntu/proj2
    command=/home/ubuntu/virtualenvs/proj1/bin/celery worker -A proj2 -l info --concurrency=10 -n proj2_worker@%%h
    # ...

接下来，为您的项目创建 2 个独立的虚拟主机：

rabbitmqctl add_user proj_1 <PASSWORD>
rabbitmqctl add_vhost proj_1_vhost
rabbitmqctl set_permissions -p proj_1_vhost proj_1 ".*" ".*" ".*"

rabbitmqctl add_user proj_2 <PASSWORD>
rabbitmqctl add_vhost proj_2_vhost
rabbitmqctl set_permissions -p proj_2_vhost proj_2 ".*" ".*" ".*"

最后修改 celery 配置以使用新创建的虚拟主机：

app = Celery('proj1_celery_app')
app.conf.update(
    # ...
    broker_url='amqp://proj1:<PASSWORD>@localhost:5672/proj_1_vhost'
    # ...
)

app = Celery('proj2_celery_app')
app.conf.update(
    # ...
    broker_url='amqp://proj2:<PASSWORD>@localhost:5672/proj_2_vhost'
    # ...
)

有关兔子虚拟主机的更多信息，请参阅此 SO post：Running multiple instances of celery on the same server。

带有 Supervisord 工人的芹菜不工作隔离

Celery with Supervisord workers are not working Isolated

django

rabbitmq

celery

python-3.x

supervisord