django + gunicorn:异步工作者的问题(gevent)

django + gunicorn: problem with async workers (gevent)

我遇到了与此 post Django+gunicorn+nginx upload large file 502 error 完全相同的问题。但是所提供的解决方案对我来说并不能解决问题,也许是因为它已经很老了。

我在 python 3.6 中使用 django、gunicorn、supervisor 和 nginx,我根据 gunicorn 的文档安装了 gevent,我的 gunicorn 配置如下所示:

[program:gunicorn]
directory=/home/ubuntu/mysite
command=/home/ubuntu/env/bin/gunicorn --workers 3 --worker-class gevent --worker-connections=1000  --timeout 600 --bind unix:/home/ubuntu/mysite/app.sock app.wsgi:application

autostart=true
autorestart=true
stderr_logfile=/var/log/gunicorn/gunicorn.err.log
stdout_logfile=/var/log/gunicorn/gunicorn.out.log


[group:guni]
programs:gunicorn

在 gunicorn 日志中,我收到错误:

 File "/home/ubuntu/exo/lib/python3.6/site-packages/gunicorn/workers/ggevent.py", line 16, in <module>
    raise RuntimeError("gevent worker requires gevent 1.4 or higher")
RuntimeError: gevent worker requires gevent 1.4 or higher

[2020-05-16 13:22:40 +0000] [24451] [DEBUG] Current configuration:
  config: None
  bind: ['unix:/home/ubuntu/myapp/app.sock']
  backlog: 2048
  workers: 4
  worker_class: sync
  threads: 12
  worker_connections: 1000
  max_requests: 0
  max_requests_jitter: 0
  timeout: 600
  graceful_timeout: 90
  keepalive: 2
  limit_request_line: 4094
  limit_request_fields: 100
  limit_request_field_size: 8190
  reload: False
  reload_engine: auto
  reload_extra_files: []
  spew: False
  check_config: False
  preload_app: False
  sendfile: None
  reuse_port: False
  chdir: /home/ubuntu/myapp
  daemon: False
  raw_env: []
  pidfile: None
  worker_tmp_dir: None
  user: 0
  group: 0
  umask: 0
  initgroups: False
  tmp_upload_dir: None
  secure_scheme_headers: {'X-FORWARDED-PROTOCOL': 'ssl', 'X-FORWARDED-PROTO': 'https', 'X-FORWARDED-SSL': 'on'}
forwarded_allow_ips: ['127.0.0.1']
  accesslog: None
  disable_redirect_access_to_syslog: False
  access_log_format: %(h)s %(l)s %(u)s %(t)s "%(r)s" %(s)s %(b)s "%(f)s" "%(a)s"
  errorlog: -
  loglevel: DEBUG
  capture_output: False
  logger_class: gunicorn.glogging.Logger
  logconfig: None
  logconfig_dict: {}
  syslog_addr: udp://localhost:514
  syslog: False
  syslog_prefix: None
  syslog_facility: user
  enable_stdio_inheritance: False
  statsd_host: None
  dogstatsd_tags:
  statsd_prefix:
  proc_name: None
default_proc_name: myapp.wsgi:application
  pythonpath: None
  paste: None
  on_starting: <function OnStarting.on_starting at 0x7f7836ae4bf8>
  on_reload: <function OnReload.on_reload at 0x7f7836ae4d08>
  when_ready: <function WhenReady.when_ready at 0x7f7836ae4e18>
  pre_fork: <function Prefork.pre_fork at 0x7f7836ae4f28>
  post_fork: <function Postfork.post_fork at 0x7f7836b000d0>
  post_worker_init: <function PostWorkerInit.post_worker_init at 0x7f7836b001e0>
  worker_int: <function WorkerInt.worker_int at 0x7f7836b002f0>
  worker_abort: <function WorkerAbort.worker_abort at 0x7f7836b00400>
  pre_exec: <function PreExec.pre_exec at 0x7f7836b00510>
  pre_request: <function PreRequest.pre_request at 0x7f7836b00620>
  post_request: <function PostRequest.post_request at 0x7f7836b006a8>
  child_exit: <function ChildExit.child_exit at 0x7f7836b007b8>
  worker_exit: <function WorkerExit.worker_exit at 0x7f7836b008c8>
  nworkers_changed: <function NumWorkersChanged.nworkers_changed at 0x7f7836b009d8>
  on_exit: <function OnExit.on_exit at 0x7f7836b00ae8>
  proxy_protocol: False

作为这方面的新手,真的很难弄清楚到底发生了什么。

我已经安装了 gevent 见下文:

Installing collected packages: greenlet, psutil, gevent
Successfully installed gevent-20.5.0 greenlet-0.4.15 psutil-5.7.0

更新: 我在 /vev/lib/python3.6/site-packages/gunicorn/config.py

中修改了 gunicorn config.py 文件
BASE_DIR = "/path/to/base/dir/"
sys.path.append(BASE_DIR)


bind = '127.0.0.1:8000'
backlog = 2048


import multiprocessing
workers = multiprocessing.cpu_count() * 2 + 1
worker_class = 'gevent'
worker_connections = 1000
timeout = 300
keepalive = 2

然后修改我的 supervisor/conf/gunicorn.conf 文件:

[program:gunicorn]

directory=/home/ubuntu/mysite
command=/home/ubuntu/exo/bin/gunicorn  --config /home/ubuntu/venv/lib/python3.6/site-packages/gunicorn/config.py  unix:/home/ubuntu/mysite/app.sock mysite.wsgi:application

autostart=true
autorestart=true
stderr_logfile=/var/log/gunicorn/gunicorn.err.log
stdout_logfile=/var/log/gunicorn/gunicorn.out.log


[group:guni]
programs:gunicorn

这仍然给我同样的错误。我什至不确定 gunicorn 配置文件是否是添加修改的正确文件,但此时我无法尝试,也许新的眼睛可以解除阻塞,或者我也愿意接受任何人可能知道的替代品的

我终于知道了,希望这个答案对遇到同样问题的 post 的人有用。

1) 需要安装gevent如下:

python3 -m pip install gevent 

2) 在您的 env/python/site-packages/gunicorn/ 中将以下内容添加到 config.py,例如:

BASE_DIR = "/path/to/base/dir/"
sys.path.append(BASE_DIR)


bind = '127.0.0.1:8000'
backlog = 2048


import multiprocessing
workers = multiprocessing.cpu_count() * 2 + 1
worker_class = 'gevent'
worker_connections = 1000
timeout = 300
keepalive = 2

3) 然后在 gunicorn.conf 中,必须确保引用 config.py 文件的路径

[program:gunicorn]

directory=/home/ubuntu/mysite
command=/home/ubuntu/exo/bin/gunicorn  --config /home/ubuntu/venv/lib/python3.6/site-packages/gunicorn/config.py  --bind unix:/home/ubuntu/mysite/app.sock mysite.wsgi:application

autostart=true
autorestart=true
stderr_logfile=/var/log/gunicorn/gunicorn.err.log
stdout_logfile=/var/log/gunicorn/gunicorn.out.log


[group:guni]
programs:gunicorn

您还可以在命令中添加 sudo 以允许 supervisor 能够访问 root user,即有权访问包含 python 使用 pip3 下载的库,在我的机器中是 /usr/local/lib/python3.8/dist-packages/gevent,所以最终的 supervisor conf file 将是:

[program:gunicorn]

directory=/home/ubuntu/mysite
command=sudo /home/ubuntu/exo/bin/gunicorn  --config /home/ubuntu/venv/lib/python3.6/site-packages/gunicorn/config.py  --bind unix:/home/ubuntu/mysite/app.sock mysite.wsgi:application

autostart=true
autorestart=true
stderr_logfile=/var/log/gunicorn/gunicorn.err.log
stdout_logfile=/var/log/gunicorn/gunicorn.out.log

[group:guni]
programs:gunicorn

site-packages/gunicorn/中:

bind = '127.0.0.1:8000'
backlog = 2048

import multiprocessing
workers = multiprocessing.cpu_count() * 2 + 1
worker_class = 'gevent'
worker_connections = 1000
timeout = 300
keepalive = 2

Another solution:

您可以使用python为您在supervisor config file中使用的当前用户下载gevent,使用此命令:

pip install --user guni gevent 

请参阅此 Whosebug 答案,了解有关 --user

的更多信息