在芹菜中，当任务排队时，将上下文元数据从发送进程传递给工作人员的适当方法是什么？

Question

当任何 celery 任务入队时，我想添加工作人员将能够使用的上下文元数据。

以下代码示例有效，但我想要一个合适的芹菜式解决方案。

from celery.signals import before_task_publish, task_prerun

@before_task_publish.connect
def receiver_before_task_publish(sender=None, headers=None, body=None, **kwargs):
    task_kwags = body[1]
    metadata = {"foo": "bar"}
    task_kwags['__metadata__'] = metadata

@task_prerun.connect
def receiver_task_pre_run(task_id, task, *args, **kwargs):
    metadata = kwargs['kwargs'].pop('__metadata__', {})
    # metadata == {"foo": "bar"}

Answer 1

当任务在 worker 中启动时，before_task_publish 的 header 的内容在 push_request 的 **kwargs 中。

celery/app/tasks.py:1000

    def push_request(self, *args, **kwargs):
        self.request_stack.push(Context(*args, **kwargs))

Context 的构造函数中做了一些不错的事情。 self.__dict__.update() 意味着我们可以访问 Context(metadata={'foo': 'bar'}).metadata

的值

celery/app/tasks.py:99

class Context(object)
# ...
    def __init__(self, *args, **kwargs):
        self.update(*args, **kwargs)

    def update(self, *args, **kwargs):
        return self.__dict__.update(*args, **kwargs)

可从 Task 的 request 属性.

访问任务上下文

celery/app/tasks.py:1019

class Task(object):
# ...
    def _get_request(self):
        """Get current request object."""
        req = self.request_stack.top
        if req is None:
            # task was not called, but some may still expect a request
            # to be there, perhaps that should be deprecated.
            if self._default_request is None:
                self._default_request = Context()
            return self._default_request
        return req
    request = property(_get_request)

这意味着最终的解决方案很简单：

from celery.signals import before_task_publish, task_prerun

@before_task_publish.connect
def receiver_before_task_publish(sender=None, headers=None, body=None, **kwargs):
    metadata = {"foo": "bar"}
    headers['__metadata__'] = metadata

@task_prerun.connect
def receiver_task_pre_run(task_id, task, *args, **kwargs):
    metadata = getattr(task.request, '__metadata__', {}) 
    # metadata == {"foo": "bar"}

注意：task.request.__metadata__ 也可以工作，但如果在集成信号之前任务已入队，则它会失败。这样更安全。

在芹菜中，当任务排队时，将上下文元数据从发送进程传递给工作人员的适当方法是什么？

In celery, what is the appropriate way to pass contextual metadata from sender process to worker when a task is enqueued?

celery

celery-task

django-celery