如何删除花中的芹菜任务日志？

Question

我希望能够在花successful/failed个任务中删除任务...

所以我的想法是有一个预定的芹菜节拍来删除早于 X 小时的任务。

任何人都知道如何实现这一点，任务存储在哪里...等等？

目标：设置一个表示小时数的配置变量，假设保留日志 48 小时，然后自动删除

主要服务于符合 GDPR 的欧洲人，同时也保护客户的隐私

Answer 1

用芹菜打花...

app.config_from_object("django.conf:settings", namespace="CELERY")
app.conf.beat_scheduler = "django_celery_beat.schedulers.DatabaseScheduler"
# app.conf.task_default_queue = 'default'

app.conf.tasks_queues = (
    Queue("default", exchange="default", routing_key="default"),
    Queue("data_queue", exchange="data_queue", routing_key="data"),
    # Queue("cleanup_queue", exchange="cleanup_queue", routing_key="cleanup"), #you don't necessarily need this queue
)

# Load task modules from all registered Django apps.
app.autodiscover_tasks()

app.conf.beat_schedule = {
    "data": {
        "task": "app.name.execute_data_tasks",
        "schedule": crontab(minute="*/3"),
    },
    "cleanup": {
        "task": "app.name.execute_cleanup_tasks",
        "schedule": crontab(minute="*/60"), #you can play around with the config here
    },
}

Answer 2

我实现了用于删除任务的新 API

Events:
  None delete_tasks_by_time(int to_timestamp)

示例：删除超过 48 小时的任务

import celery
import os
import timedelta
import datetime

broker = os.environ.get('CELERY_BROKER_URL')
app = celery.Celery('tasks', broker=broker)
flower = Flower(capp=app, options=flower_options)

time_delta = timedelta(hours=48)
now = datetime.datetime.now()
delete_before_time = now-time_delta

flower.events.delete_tasks_by_time(delete_before_time.timestamp())

然后我们可以使用一个每小时运行的 celery 节拍调度程序，并删除任务

@app.task(queue='cleanup_tasks')
def clean_up_tasks():
    from flower.app import Flower

    time_delta = timedelta(hours=48)
    now = datetime.datetime.now()
    delete_before_time = now-time_delta
    flower_options = object()
    flower_options.db = 'flower'
    flower_options.persistent = True
    flower_options.purge_offline_workers = 1
    # note: use env vars better
    flower = Flower(capp=app, options=flower_options)
    flower.events.delete_tasks_by_time(delete_before_time.timestamp())

@app.on_after_configure.connect
def add_periodic(**kwargs):
    app.add_periodic_task(crontab(hour="*", minute=0), clean_up_tasks.s(), name='cleanup-tasks')

这对想要保持良好的 GDPR 合规性的人特别有帮助，因为您只保留短时间的数据以进行基本调试，一切都可配置以品尝

参考资料： https://github.com/mher/flower/issues/1189 https://github.com/mher/flower/pull/1188

希望 PR 尽快合并！

享受 <3

如何删除花中的芹菜任务日志？

How to delete celery task logs in flower?

python

celery

flower