Huey 任务不是运行从 django 视图，而是从 shell

Question

我运行遇到了问题，想知道哪里出了问题。

问题

从网络视图调用时，Huey 任务不是运行。但是当从同一虚拟机的 shell 命令调用时，它运行良好。任务是在 huey 上正常注册。有问题的任务是 handshake.

VM：Web，使用视图：不工作
VM：网络，使用 Shell：工作
VM：工作人员，正在使用 Shell：正在工作

设置

我是运行 docker 中的一个应用程序。运行具有完全相同设置的相同图像的两个实例连接到相同的 redis 和相同的 postgres。我已验证两个实例都连接到同一个 Redis。

这两个实例是：

web.1: 运行正常线程模式下的 gunicorn Web 应用程序。
worker.1: 运行 python manage.py run_huey

设置

INSTALLED_APPS += ['huey.contrib.djhuey']

HUEY = {
    'name': 'myapp',  # Use db name for huey.
    'results': True,  # Store return values of tasks.
    'store_none': False,  # If a task returns None, do not save to results.
    'utc': True,  # Use UTC for all times internally.
    'blocking': True,  # Perform blocking pop rather than poll Redis.

    'consumer': {
        'workers': 2,
        'worker_type': 'thread',
        'initial_delay': 0.1,  # Smallest polling interval, same as -d.
        'backoff': 1.15,  # Exponential backoff using this rate, -b.
        'max_delay': 10.0,  # Max possible polling interval, -m.
        'scheduler_interval': 1,  # Check schedule every second, -s.
        'periodic': True,  # Enable crontab feature.
        'check_worker_health': True,  # Enable worker health checks.
        'health_check_interval': 1,  # Check worker health every second.
    },
}

# HUEY Overrides
HUEY['name'] = env("USER", default="myapp")
HUEY['immediate_use_memory'] = True


HUEY['immediate'] = False
HUEY['url'] = REDIS_LOCATION

任务

@huey.db_task()
def handshake(name: str):
    log = logging.getLogger(f"{__name__}::handshake")
    reply = f"Handshaked: {name}"
    log.info(reply)
    return reply

run_huey 的输出显示任务已注册（为清楚起见删除了一些行）。

2020-11-07T06:19:35.325436058Z app[worker.1]: [2020-11-07 11:49:35,325] INFO:huey.consumer:MainThread:Huey consumer started with 2 thread, PID 9 at 2020-11-07 06:19:35.325065
2020-11-07T06:19:35.325476428Z app[worker.1]: INFO 2020-11-07 11:49:35,325 consumer [9:139911526389568] - [-] - '-' - Huey consumer started with 2 thread, PID 9 at 2020-11-07 06:19:35.325065
2020-11-07T06:19:35.325939210Z app[worker.1]: [2020-11-07 11:49:35,325] INFO:huey.consumer:MainThread:Scheduler runs every 1 second(s).
2020-11-07T06:19:35.325952911Z app[worker.1]: INFO 2020-11-07 11:49:35,325 consumer [9:139911526389568] - [-] - '-' - Scheduler runs every 1 second(s).
2020-11-07T06:19:35.326318603Z app[worker.1]: [2020-11-07 11:49:35,326] INFO:huey.consumer:MainThread:Periodic tasks are enabled.
2020-11-07T06:19:35.326439763Z app[worker.1]: INFO 2020-11-07 11:49:35,326 consumer [9:139911526389568] - [-] - '-' - Periodic tasks are enabled.
2020-11-07T06:19:35.326648104Z app[worker.1]: [2020-11-07 11:49:35,326] INFO:huey.consumer:MainThread:The following commands are available:
2020-11-07T06:19:35.326662944Z app[worker.1]: + app.wms.tasks.generate_report_task
2020-11-07T06:19:35.326666544Z app[worker.1]: + app.wms.tasks.handshake
2020-11-07T06:19:35.326669924Z app[worker.1]: + app.orders.tasks.process_pending_bulk_orders
2020-11-07T06:19:35.326698025Z app[worker.1]: + app.returns.tasks.email_return_report
2020-11-07T06:19:35.326739964Z app[worker.1]: INFO 2020-11-07 11:49:35,326 consumer [9:139911526389568] - [-] - '-' - The following commands are available:
2020-11-07T06:19:35.326755324Z app[worker.1]: + app.wms.tasks.generate_report_task
2020-11-07T06:19:35.326758854Z app[worker.1]: + app.wms.tasks.handshake
2020-11-07T06:19:35.326762154Z app[worker.1]: + app.orders.tasks.process_pending_bulk_orders
2020-11-07T06:19:35.326799605Z app[worker.1]: + app.returns.tasks.email_return_report

测试一：调用handshake

的网页

from .tasks import handshake

def hello(request):
    r = handshake("Sam")
    val = r(blocking=True, timeout=20)
    return HttpResponse(f'Hello there: {val}')

网络日志中的输出：

2020-11-07T06:24:46.400137244Z app[web.1]: ERROR 2020-11-07 11:54:46,398 log [9:140443373860608] - [-] - '-' - Internal Server Error: /hello/
2020-11-07T06:24:46.400196084Z app[web.1]: Traceback (most recent call last):
2020-11-07T06:24:46.400205324Z app[web.1]:   File "/usr/local/lib/python3.8/site-packages/django/core/handlers/exception.py", line 34, in inner
2020-11-07T06:24:46.400212074Z app[web.1]:     response = get_response(request)
2020-11-07T06:24:46.400217884Z app[web.1]:   File "/usr/local/lib/python3.8/site-packages/django/core/handlers/base.py", line 115, in _get_response
2020-11-07T06:24:46.400223805Z app[web.1]:     response = self.process_exception_by_middleware(e, request)
2020-11-07T06:24:46.400228945Z app[web.1]:   File "/usr/local/lib/python3.8/site-packages/django/core/handlers/base.py", line 113, in _get_response
2020-11-07T06:24:46.400234465Z app[web.1]:     response = wrapped_callback(request, *callback_args, **callback_kwargs)
2020-11-07T06:24:46.400239975Z app[web.1]:   File "/usr/local/lib/python3.8/contextlib.py", line 75, in inner
2020-11-07T06:24:46.400244945Z app[web.1]:     return func(*args, **kwds)
2020-11-07T06:24:46.400249615Z app[web.1]:   File "/app/app/wms/views.py", line 79, in hello
2020-11-07T06:24:46.400255115Z app[web.1]:     val = r(blocking=True, timeout=20)
2020-11-07T06:24:46.400260144Z app[web.1]:   File "/usr/local/lib/python3.8/site-packages/huey/api.py", line 854, in __call__
2020-11-07T06:24:46.400265304Z app[web.1]:     return self.get(*args, **kwargs)
2020-11-07T06:24:46.400270114Z app[web.1]:   File "/usr/local/lib/python3.8/site-packages/huey/api.py", line 893, in get
2020-11-07T06:24:46.400301634Z app[web.1]:     result = self.get_raw_result(blocking, timeout, backoff, max_delay,
2020-11-07T06:24:46.400309535Z app[web.1]:   File "/usr/local/lib/python3.8/site-packages/huey/api.py", line 882, in get_raw_result
2020-11-07T06:24:46.400314785Z app[web.1]:     raise HueyException('timed out waiting for result')
2020-11-07T06:24:46.400319575Z app[web.1]: huey.exceptions.HueyException: timed out waiting for result

工作日志中没有任何记录。

测试 2：Django Shell 从 Web 运行

的同一实例调用 handshake

django@c2fcbcf604cd:/app$ python manage.py shell
(InteractiveConsole)
>>> from app.wms.tasks import handshake
>>> val = handshake('John')
>>> val
<Result: task 736642f1-0a91-4dff-9376-eca1af9ef00e>
>>> val()
'Handshaked: John'
>>>

在工作日志中：

2020-11-07T06:30:30.742538291Z app[worker.1]: [2020-11-07 12:00:30,742] INFO:huey:Worker-2:Executing app.wms.tasks.handshake: 736642f1-0a91-4dff-9376-eca1af9ef00e
2020-11-07T06:30:30.742663981Z app[worker.1]: INFO 2020-11-07 12:00:30,742 api [9:139911356086016] - [-] - '-' - Executing app.wms.tasks.handshake: 736642f1-0a91-4dff-9376-eca1af9ef00e
2020-11-07T06:30:30.743246684Z app[worker.1]: INFO 2020-11-07 12:00:30,743 tasks [9:139911356086016] - [-] - '-' - Handshaked: John
2020-11-07T06:30:30.743518145Z app[worker.1]: INFO 2020-11-07 12:00:30,743 tasks [9:139911356086016] - [-] - '-' - Handshaked: John
2020-11-07T06:30:30.744054668Z app[worker.1]: [2020-11-07 12:00:30,743] INFO:huey:Worker-2:app.wms.tasks.handshake: 736642f1-0a91-4dff-9376-eca1af9ef00e executed in 0.001s
2020-11-07T06:30:30.744117379Z app[worker.1]: INFO 2020-11-07 12:00:30,743 api [9:139911356086016] - [-] - '-' - app.wms.tasks.handshake: 736642f1-0a91-4dff-9376-eca1af9ef00e executed in 0.001s

那我错过了什么？我是否需要在 App 配置中添加一些代码来加载任务？我什至在 web.1 实例上尝试了运行 run_huey 但同样的问题。所以不是环境，不是redis。估计和web没有连接redis有关。

如果可以，请帮忙。我只是不确定我错过了什么。

谢谢阿米特

Answer 1

回答我自己。在 Charles（作者）回复后，我能够通过使用 hard-coded 名称来解决这个问题。

https://github.com/coleifer/huey/issues/561

Huey 任务不是运行从 django 视图，而是从 shell

Huey task not running from django view but works from shell

django

python-huey

Huey 任务不是 运行 从 django 视图，而是从 shell

Huey task not running from django view but works from shell

django

python-huey

Huey 任务不是运行从 django 视图，而是从 shell