Django/Celery：max_retries 适用于重试，但打印值始终是 Celery 默认值 (3)

Question

    @app.task(bind=True, autoretry_for=(Exception,), retry_kwargs={'max_retries': 10, 'countdown': 5})
    def job_deliver_message(self, message_id):
        print('Try {0}/{1}'.format(self.request.retries, self.max_retries))
        ...
        ...
        ...

为了测试，我故意导致此任务失败。它确实重试了 10 次！但是，打印输出如下：

[2020-06-03 19:14:06,700: WARNING/ForkPoolWorker-15] Try 0/3
[2020-06-03 19:14:12,080: WARNING/ForkPoolWorker-1] Try 1/3
[2020-06-03 19:14:17,553: WARNING/ForkPoolWorker-3] Try 2/3
[2020-06-03 19:14:23,000: WARNING/ForkPoolWorker-5] Try 3/3
[2020-06-03 19:14:28,489: WARNING/ForkPoolWorker-7] Try 4/3
[2020-06-03 19:14:33,603: WARNING/ForkPoolWorker-9] Try 5/3
[2020-06-03 19:14:39,038: WARNING/ForkPoolWorker-11] Try 6/3
[2020-06-03 19:14:44,525: WARNING/ForkPoolWorker-13] Try 7/3
[2020-06-03 19:14:49,688: WARNING/ForkPoolWorker-15] Try 8/3
[2020-06-03 19:14:54,985: WARNING/ForkPoolWorker-1] Try 9/3
[2020-06-03 19:14:54,985: WARNING/ForkPoolWorker-1] Try 10/3

我错过了什么？我是不是从错误的地方打印了 max_retries 的值？

是的：我知道我可以只显示“10”……正如我明确设置的那样……但只是想知道为什么它在给出 10 时给我 3 并实际对它采取行动重试次数...

Answer 1

am I printing the value for the max_retries from the wrong place?

我认为这可能是问题的根源。你能深入研究 Celery 源代码和/或设置断点吗？

您可以尝试在 class-based task 中设置 max_retries，看看是否会得到不同的结果：

class BaseTaskWithRetry(Task):
    autoretry_for = (Exception,)
    retry_kwargs = {'max_retries': 5}

Answer 2

您正在使用默认设置为 3 的 self.max_retries。

重试方法有另一个变量 max_retries，在本例中设置为 10。

代码参考：task.py

Answer 3

我实施的重试略有不同，即我在我的一个项目中有以下芹菜任务：

@app.task(bind=True, max_retries=4, default_retry_delay=3)
def get_external_sentence_translation(self, translation_request):
    """
       Passing translation data to translator
    :param self
    :param translation_request:
    :return:
    """

    url = settings.TRANSLATOR_BULK_TRANSLATION_RESOURCE
    try:
        session = requests.Session()
        result = session.post(url=url, data=translation_request)
        if result.status_code == 500:
            self.retry()
        result_json = result.json(encoding='utf-8')
        # print (u'json: ' + json.dumps(result_json))
        if 'status' not in result_json:
            message = 'Auto translation in get_external_sentence_translation json status is undefined'
            logger.warning(message)
            raise RuntimeError(message)
        if 'result' not in result_json and result_json['status'] is False:
            message = 'Auto translation in get_external_sentence_translation json status is False'
            logger.warning(message)
            raise RuntimeError(message)
        session.close()
        if 'result' in result_json:
            validation_result = JsonEncoder.from_json(result_json['result'])
            return validation_result # result_json[u'result']
            return result_json
    except Exception as e:
        logger.error('Raised exception in get_external_sentence_translation')
        logger.error(e)
        try:
            raise self.retry()
        except MaxRetriesExceededError:
            return BulkTranslationDto(status='False').to_json()

这里我手动调用 retry() 并捕获 MaxRetriesExceededError 以停止重试

Django/Celery：max_retries 适用于重试，但打印值始终是 Celery 默认值 (3)

Django/Celery: max_retries works for re-try, but printed value is always the Celery default (3)

celery

django-celery