某些 uwsgi 进程不会重新加载 django 应用程序(ansible、nginx、supervisor、uwsgi、django)
some uwsgi processes do not reload django application (ansible, nginx, supervisor, uwsgi, django)
情况:我想将我们的 Django 项目的代码更改部署到阶段或生产服务器。为此,我们有一个可靠的部署脚本,可以从 gitlab 中提取 django 项目的代码、迁移数据库、收集静态文件、重启不同的服务器等。
问题:当我通过 ansible 脚本部署时,一些 uwsgi 进程(平均 10 个中有 1 个)没有重新加载我的 django 应用程序。例如,在代码更新后静态资产发生变化并且静态资产包的文件名哈希值发生变化后,我可以重现这一点。然后服务器抛出一个 500 错误,因为 uwsgi 仍然尝试加载带有旧散列的包(通过 https://github.com/ezhome/django-webpack-loader)。当我在静态资产发生变化的部署后在浏览器中刷新网站时,服务器 returns 在大约 10 - 20% 的请求中出现 500 错误,因为它找不到旧的静态资产包(例如:捆绑.9290aAFKASE234.js).
当我在服务器上 运行 /etc/init.d/supervisor restart
时,问题立即消失,所有 uwsgi 进程似乎都正确重新加载。
相关nginx配置
location / {
{% if nginx_site_basic_auth %}
auth_basic "Test Server what.digital";
auth_basic_user_file "{{ project_root}}/nginx_passwdfile";
{% endif %}
uwsgi_pass {{ project_name }}_server;
uwsgi_read_timeout 1800s;
include /etc/nginx/uwsgi_params;
}
uwsgi 配置 (app.ini)
# uwsgi.ini file
[uwsgi]
# Django-related settings
# the base directory (full path)
chdir = {{ project_root }}
# Django's wsgi file
module = {{ project_django_wsgi }}
plugins = python
# the virtualenv (full path)
home = {{ project_venv }}
# process-related settings
# prevents some pretty crazy erratic behaviour
lazy-apps = true
# master
master = true
# maximum number of worker processes
processes = 10
pidfile = {{ wsgi_pid_file }}
#touch-reload = {{ project_root }}/config/wsgi.py
#daemonize = {{ project_root }}/{{ project_name }}-uwsgi-daemon.log
logto = {{ project_root }}/{{ project_name }}-uwsgi-error.log
log-5xx = true
disable-logging = true
harakiri = 120
no-orphans = true
max-requests = 50
# give appropriate permissions to socket file
chmod-socket = 666
# the socket (use the full path to be safe)
socket = {{ project_root }}/{{ project_name }}.sock
# http-socket = :8000
# clear environment on exit
vacuum = true
buffer-size = 32768
#
env = LANG=en_US.UTF-8
supervisor 模板(由 ansible 填充):
[program:uwsgi]
user = {{ user_name }}
command={{ project_venv }}/bin/uwsgi --ini {{ project_root }}/{{ uwsgi_conf_file }}
environment={% for name, value in env_vars.iteritems() -%}
{{ name }}="{{ value|replace('%', '%%') }}";
{%- endfor %}
autostart=true
autorestart=true
stderr_logfile = {{ project_logs }}/supervisor_uwsgi_err.log
stdout_logfile = {{ project_logs }}/supervisor_uwsgi_stdout.log
stopsignal=INT
相关ansible配置
- name: upload supervisor configuration
template: src=supervisor.j2 dest=/etc/supervisor/conf.d/{{ project_name }}.conf
become_user: root
#- name: make sure supervisord runs
# service: name=supervisor state=started
# become_user: root
#- name: reread supervisorctl config files
# command: supervisorctl reread
# become_user: root
- name: restart supervisor
# ansible's supervisorctl module doesnt really restart and some weird caching problems emerge
# shell: supervisorctl restart uwsgi
shell: /etc/init.d/supervisor restart
become_user: root
- name: make sure nginx is restarted
service: name=nginx state=restarted
become_user: root
我们终于找到了原因:django 默认使用本地缓存,当有多个 uwsgi 进程时使用它是不安全的。解决方案是设置一个 memcached 服务并为其配置 django 的 CACHES
,如下所述:https://docs.djangoproject.com/en/4.0/topics/cache/#memcached
情况:我想将我们的 Django 项目的代码更改部署到阶段或生产服务器。为此,我们有一个可靠的部署脚本,可以从 gitlab 中提取 django 项目的代码、迁移数据库、收集静态文件、重启不同的服务器等。
问题:当我通过 ansible 脚本部署时,一些 uwsgi 进程(平均 10 个中有 1 个)没有重新加载我的 django 应用程序。例如,在代码更新后静态资产发生变化并且静态资产包的文件名哈希值发生变化后,我可以重现这一点。然后服务器抛出一个 500 错误,因为 uwsgi 仍然尝试加载带有旧散列的包(通过 https://github.com/ezhome/django-webpack-loader)。当我在静态资产发生变化的部署后在浏览器中刷新网站时,服务器 returns 在大约 10 - 20% 的请求中出现 500 错误,因为它找不到旧的静态资产包(例如:捆绑.9290aAFKASE234.js).
当我在服务器上 运行 /etc/init.d/supervisor restart
时,问题立即消失,所有 uwsgi 进程似乎都正确重新加载。
相关nginx配置
location / {
{% if nginx_site_basic_auth %}
auth_basic "Test Server what.digital";
auth_basic_user_file "{{ project_root}}/nginx_passwdfile";
{% endif %}
uwsgi_pass {{ project_name }}_server;
uwsgi_read_timeout 1800s;
include /etc/nginx/uwsgi_params;
}
uwsgi 配置 (app.ini)
# uwsgi.ini file
[uwsgi]
# Django-related settings
# the base directory (full path)
chdir = {{ project_root }}
# Django's wsgi file
module = {{ project_django_wsgi }}
plugins = python
# the virtualenv (full path)
home = {{ project_venv }}
# process-related settings
# prevents some pretty crazy erratic behaviour
lazy-apps = true
# master
master = true
# maximum number of worker processes
processes = 10
pidfile = {{ wsgi_pid_file }}
#touch-reload = {{ project_root }}/config/wsgi.py
#daemonize = {{ project_root }}/{{ project_name }}-uwsgi-daemon.log
logto = {{ project_root }}/{{ project_name }}-uwsgi-error.log
log-5xx = true
disable-logging = true
harakiri = 120
no-orphans = true
max-requests = 50
# give appropriate permissions to socket file
chmod-socket = 666
# the socket (use the full path to be safe)
socket = {{ project_root }}/{{ project_name }}.sock
# http-socket = :8000
# clear environment on exit
vacuum = true
buffer-size = 32768
#
env = LANG=en_US.UTF-8
supervisor 模板(由 ansible 填充):
[program:uwsgi]
user = {{ user_name }}
command={{ project_venv }}/bin/uwsgi --ini {{ project_root }}/{{ uwsgi_conf_file }}
environment={% for name, value in env_vars.iteritems() -%}
{{ name }}="{{ value|replace('%', '%%') }}";
{%- endfor %}
autostart=true
autorestart=true
stderr_logfile = {{ project_logs }}/supervisor_uwsgi_err.log
stdout_logfile = {{ project_logs }}/supervisor_uwsgi_stdout.log
stopsignal=INT
相关ansible配置
- name: upload supervisor configuration
template: src=supervisor.j2 dest=/etc/supervisor/conf.d/{{ project_name }}.conf
become_user: root
#- name: make sure supervisord runs
# service: name=supervisor state=started
# become_user: root
#- name: reread supervisorctl config files
# command: supervisorctl reread
# become_user: root
- name: restart supervisor
# ansible's supervisorctl module doesnt really restart and some weird caching problems emerge
# shell: supervisorctl restart uwsgi
shell: /etc/init.d/supervisor restart
become_user: root
- name: make sure nginx is restarted
service: name=nginx state=restarted
become_user: root
我们终于找到了原因:django 默认使用本地缓存,当有多个 uwsgi 进程时使用它是不安全的。解决方案是设置一个 memcached 服务并为其配置 django 的 CACHES
,如下所述:https://docs.djangoproject.com/en/4.0/topics/cache/#memcached