Django 2, Celery 4.1 ModuleNotFoundError: No module named djcelery

Django 2, Celery 4.1 ModuleNotFoundError: No module named djcelery

我 运行 使用 Django 和 Celery 遇到 ModuleNotFound 错误。我有一个使用 django rest 框架的 post 端点,该框架运行 celery 任务来解析和存储 json。我可以很好地提供页面,但是当我执行 post 时,出现以下错误:

Exception Type: ModuleNotFoundError at /app/v1/results
Exception Value: No module named 'djcelery'

起初我以为我可能有版本控制问题,所以我检查了我的包,但我是最新的,没有发现任何突出的冲突。我确实看到 djcelery 是 celery 项目的加载器 init.py 中的引用:

https://github.com/celery/celery/blob/master/celery/loaders/init.py

amqp (2.2.2)
billiard (3.5.0.3)
celery (4.1.0)
certifi (2018.1.18)
Django (2.0.3)
django-celery-results (1.0.1)
django-filter (1.1.0)
django-pyodbc-azure (2.0.3.0)
djangorestframework (3.7.7)
kombu (4.1.0)
Markdown (2.6.11)
mod-wsgi (4.6.2)
pip (9.0.1)
pyodbc (4.0.22)
pytz (2018.3)
setuptools (38.5.1)
vine (1.1.4)
wheel (0.30.0)

我的项目遵循 Django 第一步的核心 http://docs.celeryproject.org/en/latest/django/first-steps-with-django.html#django-first-steps:

Proj
 - proj
   - __init__.py
   - celeryapp.py
   - settings.py
   - urls.py
 - app
   - models.py
   - tasks.py
   - views.py
   - urls.py
 - manage.py

对于我所有的代码,我都是从第一步教程中的内容开始的,但是对问题进行了一些更改 运行 并找到 post 以及尝试找到我可能的位置相对于 NotFoundError 导入不正确。我还将 from __future__ import absolute_import, unicode_literals 添加到项目中的所有文件。 在我的 settings.py 中,我有:

 # Celery settings
CELERY_BROKER_URL = 'amqp://user:pass@localhost:5672/proj_host'
CELERY_ACCEPT_CONTENT = ['application/json']
CELERY_TASK_SERIALIZER = 'json'
CELERY_RESULT_SERIALIZER = 'json'
CELERY_RESULT_BACKEND = 'django-db'

在 celeryapp.py 中(由于导入错误和其他 post 问题而从 celery.py 重命名)我有:

from __future__ import absolute_import, unicode_literals
import os
from django import apps
from django.conf import settings
from celery import Celery

# set the default Django settings module for the 'celery' program.
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'proj.settings')

app = Celery('proj')

# Using a string here means the worker don't have to serialize
# the configuration object to child processes.
# - namespace='CELERY' means all celery-related configuration keys
#   should have a `CELERY_` prefix.
app.config_from_object(settings)

# Load task modules from all registered Django app configs.
app.autodiscover_tasks(lambda: [n.name for n in apps.get_app_configs()])


@app.task(bind=True)
def debug_task(self):
    print('Request: {0!r}', format(self.request))

在我的 __init__.py 中我有:

from __future__ import absolute_import, unicode_literals

# This will make sure the app is always imported when
# Django starts so that shared_task will use this app.
from .celeryapp import app as celery_app

__all__ = ['celery_app']

终于views.py

from __future__ import absolute_import, unicode_literals
import json
from app.tasks import process_json_result
from rest_framework.views import APIView
from rest_framework.response import Response


class AppResultView(APIView):
    #check for errors then
    process_json_result.delay(json.dumps(request.data))
    return Response({"success": True, "message": "Ok", "payload": ""})

tasks.py

from __future__ import absolute_import, unicode_literals
import json
import logging
from proj.celeryapp import app
from celery.utils.log import get_task_logger
from app.models import Result

logger = get_task_logger(__name__)
logger.setLevel(logging.DEBUG)


@app.task(name="app.tasks.process_json_result")
def process_json_result(result):
    logger.info("Processing result")
    py_result = json.loads(result)
    # processing occurs

以及错误的较长回溯:

ModuleNotFoundError at /app/v1/results
No module named 'djcelery'

Request Method: POST
Request URL: http://testdeployserver/app/v1/results
Django Version: 2.0.3
Python Executable: 
Python Version: 3.6.4
Python Path: ['/var/www/python/proj', '/var/www/python/proj/proj', '/home/alex/anaconda3/envs/clean_django/lib/python36.zip', '/home/alex/anaconda3/envs/clean_django/lib/python3.6', '/home/alex/anaconda3/envs/clean_django/lib/python3.6/lib-dynload', '/home/alex/anaconda3/envs/clean_django/lib/python3.6/site-packages']
Server time: Tue, 13 Mar 2018 23:00:10 +0000
Installed Applications:
['django.contrib.admin',
 'django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'app',
 'rest_framework',
 'django_celery_results']
Installed Middleware:
['django.middleware.security.SecurityMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware']


Traceback:

File "/home/alex/anaconda3/envs/clean_django/lib/python3.6/site-packages/celery/local.py" in _get_current_object
  317.             return object.__getattribute__(self, '__thing')

During handling of the above exception ('PromiseProxy' object has no attribute '__thing'), another exception occurred:

File "/home/alex/anaconda3/envs/clean_django/lib/python3.6/site-packages/kombu/utils/objects.py" in __get__
  42.             return obj.__dict__[self.__name__]

During handling of the above exception ('data'), another exception occurred:

File "/home/alex/anaconda3/envs/clean_django/lib/python3.6/site-packages/kombu/utils/objects.py" in __get__
  42.             return obj.__dict__[self.__name__]

During handling of the above exception ('loader'), another exception occurred:

File "/home/alex/anaconda3/envs/clean_django/lib/python3.6/site-packages/django/core/handlers/exception.py" in inner
  35.             response = get_response(request)

File "/home/alex/anaconda3/envs/clean_django/lib/python3.6/site-packages/django/core/handlers/base.py" in _get_response
  128.                 response = self.process_exception_by_middleware(e, request)

File "/home/alex/anaconda3/envs/clean_django/lib/python3.6/site-packages/django/core/handlers/base.py" in _get_response
  126.                 response = wrapped_callback(request, *callback_args, **callback_kwargs)

File "/home/alex/anaconda3/envs/clean_django/lib/python3.6/site-packages/django/views/decorators/csrf.py" in wrapped_view
  54.         return view_func(*args, **kwargs)

File "/home/alex/anaconda3/envs/clean_django/lib/python3.6/site-packages/django/views/generic/base.py" in view
  69.             return self.dispatch(request, *args, **kwargs)

File "/home/alex/anaconda3/envs/clean_django/lib/python3.6/site-packages/rest_framework/views.py" in dispatch
  494.             response = self.handle_exception(exc)

File "/home/alex/anaconda3/envs/clean_django/lib/python3.6/site-packages/rest_framework/views.py" in handle_exception
  454.             self.raise_uncaught_exception(exc)

File "/home/alex/anaconda3/envs/clean_django/lib/python3.6/site-packages/rest_framework/views.py" in dispatch
  491.             response = handler(request, *args, **kwargs)

File "/var/www/python/proj/app/views.py" in post
  24.             process_app_result.delay(json.dumps(request.data))

File "/home/alex/anaconda3/envs/clean_django/lib/python3.6/site-packages/celery/local.py" in __getattr__
  146.         return getattr(self._get_current_object(), name)

File "/home/alex/anaconda3/envs/clean_django/lib/python3.6/site-packages/celery/local.py" in _get_current_object
  319.             return self.__evaluate__()

File "/home/alex/anaconda3/envs/clean_django/lib/python3.6/site-packages/celery/local.py" in __evaluate__
  349.             thing = Proxy._get_current_object(self)

File "/home/alex/anaconda3/envs/clean_django/lib/python3.6/site-packages/celery/local.py" in _get_current_object
  109.             return loc(*self.__args, **self.__kwargs)

File "/home/alex/anaconda3/envs/clean_django/lib/python3.6/site-packages/celery/app/base.py" in _task_from_fun
  462.             task.bind(self)  # connects task to this app

File "/home/alex/anaconda3/envs/clean_django/lib/python3.6/site-packages/celery/app/task.py" in bind
  320.                 setattr(cls, attr_name, conf[config_name])

File "/home/alex/anaconda3/envs/clean_django/lib/python3.6/site-packages/celery/utils/collections.py" in __getitem__
  431.                 return getitem(k)

File "/home/alex/anaconda3/envs/clean_django/lib/python3.6/site-packages/celery/utils/collections.py" in __getitem__
  280.                 return mapping[_key]

File "/home/alex/anaconda3/envs/clean_django/lib/python3.6/collections/__init__.py" in __getitem__
  987.         if key in self.data:

File "/home/alex/anaconda3/envs/clean_django/lib/python3.6/site-packages/kombu/utils/objects.py" in __get__
  44.             value = obj.__dict__[self.__name__] = self.__get(obj)

File "/home/alex/anaconda3/envs/clean_django/lib/python3.6/site-packages/celery/app/base.py" in data
  148.         return self.callback()

File "/home/alex/anaconda3/envs/clean_django/lib/python3.6/site-packages/celery/app/base.py" in _finalize_pending_conf
  911.         conf = self._conf = self._load_config()

File "/home/alex/anaconda3/envs/clean_django/lib/python3.6/site-packages/celery/app/base.py" in _load_config
  921.             self.loader.config_from_object(self._config_source)

File "/home/alex/anaconda3/envs/clean_django/lib/python3.6/site-packages/kombu/utils/objects.py" in __get__
  44.             value = obj.__dict__[self.__name__] = self.__get(obj)

File "/home/alex/anaconda3/envs/clean_django/lib/python3.6/site-packages/celery/app/base.py" in loader
  1209.         return get_loader_cls(self.loader_cls)(app=self)

File "/home/alex/anaconda3/envs/clean_django/lib/python3.6/site-packages/celery/loaders/__init__.py" in get_loader_cls
  21.     return symbol_by_name(loader, LOADER_ALIASES, imp=import_from_cwd)

File "/home/alex/anaconda3/envs/clean_django/lib/python3.6/site-packages/kombu/utils/imports.py" in symbol_by_name
  56.             module = imp(module_name, package=package, **kwargs)

File "/home/alex/anaconda3/envs/clean_django/lib/python3.6/site-packages/celery/utils/imports.py" in import_from_cwd
  101.         return imp(module, package=package)

File "/home/alex/anaconda3/envs/clean_django/lib/python3.6/importlib/__init__.py" in import_module
  126.     return _bootstrap._gcd_import(name[level:], package, level)

File "<frozen importlib._bootstrap>" in _gcd_import
  994. <source code not available>

File "<frozen importlib._bootstrap>" in _find_and_load
  971. <source code not available>

File "<frozen importlib._bootstrap>" in _find_and_load_unlocked
  941. <source code not available>

File "<frozen importlib._bootstrap>" in _call_with_frames_removed
  219. <source code not available>

File "<frozen importlib._bootstrap>" in _gcd_import
  994. <source code not available>

File "<frozen importlib._bootstrap>" in _find_and_load
  971. <source code not available>

File "<frozen importlib._bootstrap>" in _find_and_load_unlocked
  953. <source code not available>

Exception Type: ModuleNotFoundError at /app/v1/results
Exception Value: No module named 'djcelery'

从 Apache/mod_wsgi anaconda 环境路径中提取的回溯路径看起来是正确的,我在我的 anaconda 虚拟环境中看到了 celery 包文件,所以不确定什么没有被正确解决或者我的任务是否正确路径设置不适合 Celery。

今天早上回到问题并回滚所有更改,直到我符合 Django 第一步文档。那时我 运行 进入了我看到其他人 post 的 cannot import Celery from celery 问题。这让我想知道为什么这在 django 测试服务器上有效,但在我的部署服务器上却无效。然后我去查看 Apache 的 httpd conf 文件,发现我认为可能导致问题的原因:

WSGIDaemonProcess proj python-path=/var/www/python:/var/www/python/proj:/home/alex/anaconda3/envs/clean_django/lib/python3.6/site-packages

我想知道是不是我的项目在路径上的包之前导致了这个问题,所以我将其更改为:

WSGIDaemonProcess proj python-path=//home/alex/anaconda3/envs/clean_django/lib/python3.6/site-packages:var/www/python:/var/www/python/proj

而且我的项目现在已按预期部署到 Apache 和 Celery 运行。

TLDR 检查您的路径以及 Django 测试服务器和部署服务器上的路径解析之间的任何差异。确保包在您的路径中位于您的应用程序之前,尤其是混合使用 Celery。