django + gunicorn:异步工作者的问题(gevent)
django + gunicorn: problem with async workers (gevent)
我遇到了与此 post Django+gunicorn+nginx upload large file 502 error 完全相同的问题。但是所提供的解决方案对我来说并不能解决问题,也许是因为它已经很老了。
我在 python 3.6 中使用 django、gunicorn、supervisor 和 nginx,我根据 gunicorn 的文档安装了 gevent,我的 gunicorn 配置如下所示:
[program:gunicorn]
directory=/home/ubuntu/mysite
command=/home/ubuntu/env/bin/gunicorn --workers 3 --worker-class gevent --worker-connections=1000 --timeout 600 --bind unix:/home/ubuntu/mysite/app.sock app.wsgi:application
autostart=true
autorestart=true
stderr_logfile=/var/log/gunicorn/gunicorn.err.log
stdout_logfile=/var/log/gunicorn/gunicorn.out.log
[group:guni]
programs:gunicorn
在 gunicorn 日志中,我收到错误:
File "/home/ubuntu/exo/lib/python3.6/site-packages/gunicorn/workers/ggevent.py", line 16, in <module>
raise RuntimeError("gevent worker requires gevent 1.4 or higher")
RuntimeError: gevent worker requires gevent 1.4 or higher
[2020-05-16 13:22:40 +0000] [24451] [DEBUG] Current configuration:
config: None
bind: ['unix:/home/ubuntu/myapp/app.sock']
backlog: 2048
workers: 4
worker_class: sync
threads: 12
worker_connections: 1000
max_requests: 0
max_requests_jitter: 0
timeout: 600
graceful_timeout: 90
keepalive: 2
limit_request_line: 4094
limit_request_fields: 100
limit_request_field_size: 8190
reload: False
reload_engine: auto
reload_extra_files: []
spew: False
check_config: False
preload_app: False
sendfile: None
reuse_port: False
chdir: /home/ubuntu/myapp
daemon: False
raw_env: []
pidfile: None
worker_tmp_dir: None
user: 0
group: 0
umask: 0
initgroups: False
tmp_upload_dir: None
secure_scheme_headers: {'X-FORWARDED-PROTOCOL': 'ssl', 'X-FORWARDED-PROTO': 'https', 'X-FORWARDED-SSL': 'on'}
forwarded_allow_ips: ['127.0.0.1']
accesslog: None
disable_redirect_access_to_syslog: False
access_log_format: %(h)s %(l)s %(u)s %(t)s "%(r)s" %(s)s %(b)s "%(f)s" "%(a)s"
errorlog: -
loglevel: DEBUG
capture_output: False
logger_class: gunicorn.glogging.Logger
logconfig: None
logconfig_dict: {}
syslog_addr: udp://localhost:514
syslog: False
syslog_prefix: None
syslog_facility: user
enable_stdio_inheritance: False
statsd_host: None
dogstatsd_tags:
statsd_prefix:
proc_name: None
default_proc_name: myapp.wsgi:application
pythonpath: None
paste: None
on_starting: <function OnStarting.on_starting at 0x7f7836ae4bf8>
on_reload: <function OnReload.on_reload at 0x7f7836ae4d08>
when_ready: <function WhenReady.when_ready at 0x7f7836ae4e18>
pre_fork: <function Prefork.pre_fork at 0x7f7836ae4f28>
post_fork: <function Postfork.post_fork at 0x7f7836b000d0>
post_worker_init: <function PostWorkerInit.post_worker_init at 0x7f7836b001e0>
worker_int: <function WorkerInt.worker_int at 0x7f7836b002f0>
worker_abort: <function WorkerAbort.worker_abort at 0x7f7836b00400>
pre_exec: <function PreExec.pre_exec at 0x7f7836b00510>
pre_request: <function PreRequest.pre_request at 0x7f7836b00620>
post_request: <function PostRequest.post_request at 0x7f7836b006a8>
child_exit: <function ChildExit.child_exit at 0x7f7836b007b8>
worker_exit: <function WorkerExit.worker_exit at 0x7f7836b008c8>
nworkers_changed: <function NumWorkersChanged.nworkers_changed at 0x7f7836b009d8>
on_exit: <function OnExit.on_exit at 0x7f7836b00ae8>
proxy_protocol: False
作为这方面的新手,真的很难弄清楚到底发生了什么。
我已经安装了 gevent 见下文:
Installing collected packages: greenlet, psutil, gevent
Successfully installed gevent-20.5.0 greenlet-0.4.15 psutil-5.7.0
更新:
我在 /vev/lib/python3.6/site-packages/gunicorn/config.py
中修改了 gunicorn config.py 文件
BASE_DIR = "/path/to/base/dir/"
sys.path.append(BASE_DIR)
bind = '127.0.0.1:8000'
backlog = 2048
import multiprocessing
workers = multiprocessing.cpu_count() * 2 + 1
worker_class = 'gevent'
worker_connections = 1000
timeout = 300
keepalive = 2
然后修改我的 supervisor/conf/gunicorn.conf 文件:
[program:gunicorn]
directory=/home/ubuntu/mysite
command=/home/ubuntu/exo/bin/gunicorn --config /home/ubuntu/venv/lib/python3.6/site-packages/gunicorn/config.py unix:/home/ubuntu/mysite/app.sock mysite.wsgi:application
autostart=true
autorestart=true
stderr_logfile=/var/log/gunicorn/gunicorn.err.log
stdout_logfile=/var/log/gunicorn/gunicorn.out.log
[group:guni]
programs:gunicorn
这仍然给我同样的错误。我什至不确定 gunicorn 配置文件是否是添加修改的正确文件,但此时我无法尝试,也许新的眼睛可以解除阻塞,或者我也愿意接受任何人可能知道的替代品的
我终于知道了,希望这个答案对遇到同样问题的 post 的人有用。
1) 需要安装gevent如下:
python3 -m pip install gevent
2) 在您的 env/python/site-packages/gunicorn/ 中将以下内容添加到 config.py,例如:
BASE_DIR = "/path/to/base/dir/"
sys.path.append(BASE_DIR)
bind = '127.0.0.1:8000'
backlog = 2048
import multiprocessing
workers = multiprocessing.cpu_count() * 2 + 1
worker_class = 'gevent'
worker_connections = 1000
timeout = 300
keepalive = 2
3) 然后在 gunicorn.conf 中,必须确保引用 config.py 文件的路径
[program:gunicorn]
directory=/home/ubuntu/mysite
command=/home/ubuntu/exo/bin/gunicorn --config /home/ubuntu/venv/lib/python3.6/site-packages/gunicorn/config.py --bind unix:/home/ubuntu/mysite/app.sock mysite.wsgi:application
autostart=true
autorestart=true
stderr_logfile=/var/log/gunicorn/gunicorn.err.log
stdout_logfile=/var/log/gunicorn/gunicorn.out.log
[group:guni]
programs:gunicorn
您还可以在命令中添加 sudo
以允许 supervisor
能够访问 root user
,即有权访问包含 python 使用 pip3
下载的库,在我的机器中是 /usr/local/lib/python3.8/dist-packages/gevent
,所以最终的 supervisor conf file
将是:
[program:gunicorn]
directory=/home/ubuntu/mysite
command=sudo /home/ubuntu/exo/bin/gunicorn --config /home/ubuntu/venv/lib/python3.6/site-packages/gunicorn/config.py --bind unix:/home/ubuntu/mysite/app.sock mysite.wsgi:application
autostart=true
autorestart=true
stderr_logfile=/var/log/gunicorn/gunicorn.err.log
stdout_logfile=/var/log/gunicorn/gunicorn.out.log
[group:guni]
programs:gunicorn
在site-packages/gunicorn/
中:
bind = '127.0.0.1:8000'
backlog = 2048
import multiprocessing
workers = multiprocessing.cpu_count() * 2 + 1
worker_class = 'gevent'
worker_connections = 1000
timeout = 300
keepalive = 2
Another solution:
您可以使用python为您在supervisor config file
中使用的当前用户下载gevent
,使用此命令:
pip install --user guni gevent
请参阅此 Whosebug 答案,了解有关 --user
、
的更多信息
我遇到了与此 post Django+gunicorn+nginx upload large file 502 error 完全相同的问题。但是所提供的解决方案对我来说并不能解决问题,也许是因为它已经很老了。
我在 python 3.6 中使用 django、gunicorn、supervisor 和 nginx,我根据 gunicorn 的文档安装了 gevent,我的 gunicorn 配置如下所示:
[program:gunicorn]
directory=/home/ubuntu/mysite
command=/home/ubuntu/env/bin/gunicorn --workers 3 --worker-class gevent --worker-connections=1000 --timeout 600 --bind unix:/home/ubuntu/mysite/app.sock app.wsgi:application
autostart=true
autorestart=true
stderr_logfile=/var/log/gunicorn/gunicorn.err.log
stdout_logfile=/var/log/gunicorn/gunicorn.out.log
[group:guni]
programs:gunicorn
在 gunicorn 日志中,我收到错误:
File "/home/ubuntu/exo/lib/python3.6/site-packages/gunicorn/workers/ggevent.py", line 16, in <module>
raise RuntimeError("gevent worker requires gevent 1.4 or higher")
RuntimeError: gevent worker requires gevent 1.4 or higher
[2020-05-16 13:22:40 +0000] [24451] [DEBUG] Current configuration:
config: None
bind: ['unix:/home/ubuntu/myapp/app.sock']
backlog: 2048
workers: 4
worker_class: sync
threads: 12
worker_connections: 1000
max_requests: 0
max_requests_jitter: 0
timeout: 600
graceful_timeout: 90
keepalive: 2
limit_request_line: 4094
limit_request_fields: 100
limit_request_field_size: 8190
reload: False
reload_engine: auto
reload_extra_files: []
spew: False
check_config: False
preload_app: False
sendfile: None
reuse_port: False
chdir: /home/ubuntu/myapp
daemon: False
raw_env: []
pidfile: None
worker_tmp_dir: None
user: 0
group: 0
umask: 0
initgroups: False
tmp_upload_dir: None
secure_scheme_headers: {'X-FORWARDED-PROTOCOL': 'ssl', 'X-FORWARDED-PROTO': 'https', 'X-FORWARDED-SSL': 'on'}
forwarded_allow_ips: ['127.0.0.1']
accesslog: None
disable_redirect_access_to_syslog: False
access_log_format: %(h)s %(l)s %(u)s %(t)s "%(r)s" %(s)s %(b)s "%(f)s" "%(a)s"
errorlog: -
loglevel: DEBUG
capture_output: False
logger_class: gunicorn.glogging.Logger
logconfig: None
logconfig_dict: {}
syslog_addr: udp://localhost:514
syslog: False
syslog_prefix: None
syslog_facility: user
enable_stdio_inheritance: False
statsd_host: None
dogstatsd_tags:
statsd_prefix:
proc_name: None
default_proc_name: myapp.wsgi:application
pythonpath: None
paste: None
on_starting: <function OnStarting.on_starting at 0x7f7836ae4bf8>
on_reload: <function OnReload.on_reload at 0x7f7836ae4d08>
when_ready: <function WhenReady.when_ready at 0x7f7836ae4e18>
pre_fork: <function Prefork.pre_fork at 0x7f7836ae4f28>
post_fork: <function Postfork.post_fork at 0x7f7836b000d0>
post_worker_init: <function PostWorkerInit.post_worker_init at 0x7f7836b001e0>
worker_int: <function WorkerInt.worker_int at 0x7f7836b002f0>
worker_abort: <function WorkerAbort.worker_abort at 0x7f7836b00400>
pre_exec: <function PreExec.pre_exec at 0x7f7836b00510>
pre_request: <function PreRequest.pre_request at 0x7f7836b00620>
post_request: <function PostRequest.post_request at 0x7f7836b006a8>
child_exit: <function ChildExit.child_exit at 0x7f7836b007b8>
worker_exit: <function WorkerExit.worker_exit at 0x7f7836b008c8>
nworkers_changed: <function NumWorkersChanged.nworkers_changed at 0x7f7836b009d8>
on_exit: <function OnExit.on_exit at 0x7f7836b00ae8>
proxy_protocol: False
作为这方面的新手,真的很难弄清楚到底发生了什么。
我已经安装了 gevent 见下文:
Installing collected packages: greenlet, psutil, gevent
Successfully installed gevent-20.5.0 greenlet-0.4.15 psutil-5.7.0
更新: 我在 /vev/lib/python3.6/site-packages/gunicorn/config.py
中修改了 gunicorn config.py 文件BASE_DIR = "/path/to/base/dir/"
sys.path.append(BASE_DIR)
bind = '127.0.0.1:8000'
backlog = 2048
import multiprocessing
workers = multiprocessing.cpu_count() * 2 + 1
worker_class = 'gevent'
worker_connections = 1000
timeout = 300
keepalive = 2
然后修改我的 supervisor/conf/gunicorn.conf 文件:
[program:gunicorn]
directory=/home/ubuntu/mysite
command=/home/ubuntu/exo/bin/gunicorn --config /home/ubuntu/venv/lib/python3.6/site-packages/gunicorn/config.py unix:/home/ubuntu/mysite/app.sock mysite.wsgi:application
autostart=true
autorestart=true
stderr_logfile=/var/log/gunicorn/gunicorn.err.log
stdout_logfile=/var/log/gunicorn/gunicorn.out.log
[group:guni]
programs:gunicorn
这仍然给我同样的错误。我什至不确定 gunicorn 配置文件是否是添加修改的正确文件,但此时我无法尝试,也许新的眼睛可以解除阻塞,或者我也愿意接受任何人可能知道的替代品的
我终于知道了,希望这个答案对遇到同样问题的 post 的人有用。
1) 需要安装gevent如下:
python3 -m pip install gevent
2) 在您的 env/python/site-packages/gunicorn/ 中将以下内容添加到 config.py,例如:
BASE_DIR = "/path/to/base/dir/"
sys.path.append(BASE_DIR)
bind = '127.0.0.1:8000'
backlog = 2048
import multiprocessing
workers = multiprocessing.cpu_count() * 2 + 1
worker_class = 'gevent'
worker_connections = 1000
timeout = 300
keepalive = 2
3) 然后在 gunicorn.conf 中,必须确保引用 config.py 文件的路径
[program:gunicorn]
directory=/home/ubuntu/mysite
command=/home/ubuntu/exo/bin/gunicorn --config /home/ubuntu/venv/lib/python3.6/site-packages/gunicorn/config.py --bind unix:/home/ubuntu/mysite/app.sock mysite.wsgi:application
autostart=true
autorestart=true
stderr_logfile=/var/log/gunicorn/gunicorn.err.log
stdout_logfile=/var/log/gunicorn/gunicorn.out.log
[group:guni]
programs:gunicorn
您还可以在命令中添加 sudo
以允许 supervisor
能够访问 root user
,即有权访问包含 python 使用 pip3
下载的库,在我的机器中是 /usr/local/lib/python3.8/dist-packages/gevent
,所以最终的 supervisor conf file
将是:
[program:gunicorn]
directory=/home/ubuntu/mysite
command=sudo /home/ubuntu/exo/bin/gunicorn --config /home/ubuntu/venv/lib/python3.6/site-packages/gunicorn/config.py --bind unix:/home/ubuntu/mysite/app.sock mysite.wsgi:application
autostart=true
autorestart=true
stderr_logfile=/var/log/gunicorn/gunicorn.err.log
stdout_logfile=/var/log/gunicorn/gunicorn.out.log
[group:guni]
programs:gunicorn
在site-packages/gunicorn/
中:
bind = '127.0.0.1:8000'
backlog = 2048
import multiprocessing
workers = multiprocessing.cpu_count() * 2 + 1
worker_class = 'gevent'
worker_connections = 1000
timeout = 300
keepalive = 2
Another solution:
您可以使用python为您在supervisor config file
中使用的当前用户下载gevent
,使用此命令:
pip install --user guni gevent
请参阅此 Whosebug 答案,了解有关 --user
、