Celery 任务在 Django 应用程序中执行多次

Question

我有一个 Django 应用程序，我在 task.py 下定义了几个 @task 函数以在给定的周期性任务中执行。我 100% 确定问题不是由 task.py 或任何相关代码引起的，而是由于某些配置可能在 settings.py 或我的芹菜工人中。

任务确实在周期性任务中执行但多次执行。

这里是 celery worker 日志：

celery -A cimexmonitor worker --loglevel=info -B -c 4

[2019-09-19 21:22:16,360: INFO/ForkPoolWorker-5] Project Monitor Started : APPProject1
[2019-09-19 21:22:16,361: INFO/ForkPoolWorker-4] Project Monitor Started : APPProject1
[2019-09-19 21:25:22,108: INFO/ForkPoolWorker-4] Project Monitor DONE : APPProject1
[2019-09-19 21:25:45,255: INFO/ForkPoolWorker-5] Project Monitor DONE : APPProject1
[2019-09-20 00:22:16,395: INFO/ForkPoolWorker-4] Project Monitor Started : APPProject2
[2019-09-20 00:22:16,398: INFO/ForkPoolWorker-5] Project Monitor Started : APPProject2
[2019-09-20 01:22:11,554: INFO/ForkPoolWorker-5] Project Monitor DONE : APPProject2
[2019-09-20 01:22:12,047: INFO/ForkPoolWorker-4] Project Monitor DONE : APPProject2

如果你检查上面的时间间隔，tasks.py 执行一个任务，但 2 个 celery 工人接管任务并以相同的时间间隔执行相同的任务。我不确定为什么 2 个工人负责一项任务？
settings.py

..
..
# Internationalization
# https://docs.djangoproject.com/en/2.1/topics/i18n/

LANGUAGE_CODE = 'en-us'

TIME_ZONE = 'Asia/Kolkata'

USE_I18N = True

USE_L10N = True

USE_TZ = True
..
..
..
######## CELERY : CONFIG
CELERY_BROKER_URL = 'redis://localhost:6379'
CELERY_RESULT_BACKEND = 'redis://localhost:6379'
CELERY_ACCEPT_CONTENT = ['application/json']
CELERY_TASK_SERIALIZER = 'json'
CELERY_RESULT_SERIALIZER = 'json'
CELERY_ENABLE_UTC = True
CELERYBEAT_SCHEDULER = 'django_celery_beat.schedulers:DatabaseScheduler'

celery.py

from __future__ import absolute_import, unicode_literals
from celery import Celery 
import os
from django.conf import settings

os.environ.setdefault('DJANGO_SETTINGS_MODULE','cimexmonitor.settings')
## set the default Django settings module for the 'celery' program.

# Using a string here means the worker don't have to serialize
# the configuration object to child processes.
# - namespace='CELERY' means all celery-related configuration keys
#   should have a `CELERY_` prefix.

app = Celery('cimexmonitor')
#app.config_from_object('django.conf:settings', namespace='CELERY') 
app.config_from_object('django.conf:settings')
# Load task modules from all registered Django app configs.
app.autodiscover_tasks(settings.INSTALLED_APPS)

@app.task(bind=True)
def debug_task(self):
    print('Request: {0!r}'.format(self.request))

其他信息：

→ celery --version
4.3.0 (rhubarb)

→ redis-server --version
Redis server v=3.0.6 sha=00000000:0 malloc=jemalloc-3.6.0 bits=64 build=7785291a3d2152db

django-admin-interface==0.9.2
django-celery-beat==1.5.0

请帮我解决问题的方法：

谢谢

Answer 1

来自官方文档：Ensuring a task is only executed one at a time.

此外，我希望您不会运行以相同的方式 (celery -A cimexmonitor worker --loglevel=info -B -c 4) 安排多个工作人员，因为这意味着您有多个 celery beats 调度任务运行...简而言之 - 确保你只有一个 Celery beat 运行ning!

Answer 2

worker 和 beat 服务需要同时运行才能按照 https://github.com/celery/django-celery-beat

定期执行任务

工人：

 $ celery -A [project-name] worker --loglevel=info -B -c 5

Django 调度程序：

celery -A [project-name] beat -l info --scheduler django_celery_beat.schedulers:DatabaseScheduler

我同时运行既是工作人员又是数据库调度程序，根据文档说这是导致问题同时执行的原因，我真的不确定如何 celery worker 同时开始作为数据库调度程序工作。
刚刚运行 celery worker 解决了我的问题。

Celery 任务在 Django 应用程序中执行多次

Celery Tasks are executing multiple times in Django Application

python

django

redis

celery

django-celery