MemoryError 使用 Nginx、uWSGI 和 web.py
MemoryError using Nginx, uWSGI, and web.py
我有一个 python Apple MDM 服务器，我正在尝试将其从 运行 从命令行进程转换为使用 Nginx 和 uWSGI 托管。我正在使用 web.py 进行路由。一切都运行宁宁最合适的顺利。我已经有了我认为对 Nginx 和 uWSGI 都很好的基本配置，但是今天早上 运行 将其设置不到 24 小时后，我在服务器收到的所有其他请求上都遇到了这个错误。有足够的可用内存，我正在使用 python.
的 64 位版本
编辑 1：SSL 在端口 8080 上通信，因为它前面有一个负载均衡器将 SSL 流量路由到该端口。
EDIT2：我已经将 uWSGI 中的进程设置从 3 更改为 1 以查看是否是问题所在。
EDIT3：这是来自 AWS 的新盒子。安装了我的服务器及其依赖项、nginx 和 uWSGI。我在使用 运行 将 uWSGI 作为服务时遇到了麻烦，在我最终获得 pip 版本 运行 之前安装和卸载了它的 aptitude 和 pip 版本 systemd。这就是现在 运行ning 的方式，它正在优雅地停止、启动和重新启动。
[ Tue Jun 14 2016 10:29:22 ] Traceback (most recent call last):
File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 237, in process
File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 565, in processor
File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 77, in reload_mapping
MemoryError
[ Tue Jun 14 2016 10:29:22 ]
[ Tue Jun 14 2016 10:29:22 ] Traceback (most recent call last):
File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 237, in process
File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 566, in processor
File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 237, in <lambda>
File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 246, in process
File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 477, in internalerror
MemoryError
[ Tue Jun 14 2016 10:29:22 ]
[ Tue Jun 14 2016 10:29:22 ] Traceback (most recent call last):
File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 237, in process
File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 581, in processor
File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 237, in <lambda>
File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 246, in process
File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 477, in internalerror
MemoryError
[ Tue Jun 14 2016 10:29:22 ]
[ Tue Jun 14 2016 10:29:22 ] Traceback (most recent call last):
File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 237, in process
File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 566, in processor
File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 237, in <lambda>
File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 246, in process
File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 477, in internalerror
MemoryError
[ Tue Jun 14 2016 10:29:22 ]
[ Tue Jun 14 2016 10:29:22 ] Traceback (most recent call last):
[ Tue Jun 14 2016 10:29:22 ] File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 279, in wsgi
[ Tue Jun 14 2016 10:29:22 ] [ Tue Jun 14 2016 10:29:22 ] result = self.handle_with_processors()
[ Tue Jun 14 2016 10:29:22 ] File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 249, in handle_with_processors
[ Tue Jun 14 2016 10:29:22 ] [ Tue Jun 14 2016 10:29:22 ] return process(self.processors)
[ Tue Jun 14 2016 10:29:22 ] File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 246, in process
[ Tue Jun 14 2016 10:29:22 ] [ Tue Jun 14 2016 10:29:22 ] raise self.internalerror()
[ Tue Jun 14 2016 10:29:22 ] File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 477, in internalerror
[ Tue Jun 14 2016 10:29:22 ] [ Tue Jun 14 2016 10:29:22 ] import debugerror
[ Tue Jun 14 2016 10:29:22 ] MemoryError[ Tue Jun 14 2016 10:29:22 ]
nginx.conf
user www-data;
worker_processes auto;
pid /run/nginx.pid;
events {
worker_connections 768;
# multi_accept on;
}
http {
##
# Basic Settings
##
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
# server_tokens off;
# server_names_hash_bucket_size 64;
# server_name_in_redirect off;
include /etc/nginx/mime.types;
default_type application/octet-stream;
##
# SSL Settings
##
ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
ssl_prefer_server_ciphers on;
##
# Logging Settings
##
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
##
# Gzip Settings
##
gzip on;
gzip_disable "msie6";
# gzip_vary on;
# gzip_proxied any;
# gzip_comp_level 6;
# gzip_buffers 16 8k;
# gzip_http_version 1.1;
# gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;
##
# Virtual Host Configs
##
include /etc/nginx/conf.d/*.conf;
include /etc/nginx/sites-enabled/*;
}
#mail {
# # See sample authentication script at:
# # http://wiki.nginx.org/ImapAuthenticateWithApachePhpScript
#
# # auth_http localhost/auth.php;
# # pop3_capabilities "TOP" "USER";
# # imap_capabilities "IMAP4rev1" "UIDPLUS";
#
# server {
# listen localhost:110;
# protocol pop3;
# proxy on;
# }
#
# server {
# listen localhost:143;
# protocol imap;
# proxy on;
# }
#}
启用站点
server {
listen 80;
location ^~ /static/ {
root /path/to/app/;
}
location / {
include uwsgi_params;
uwsgi_pass 127.0.0.1:9000;
}
}
server {
listen 8080;
ssl on;
ssl_certificate /path/to/app/certs/ssl.crt;
ssl_certificate_key /path/to/app/certs/ssl.key;
location ^~ /static/ {
root /path/to/app/static/;
}
location / {
include uwsgi_params;
uwsgi_pass 127.0.0.1:9000;
uwsgi_param UWSGI_CHDIR /path/to/app/;
uwsgi_param UWSGI_PYHOME /path/to/app/virtualenv/;
uwsgi_param UWSGI_SCRIPT server;
}
}
uWSGI ini
home = /path/to/app/virtualenv/
chdir = /path/to/app/
module = server:application
logto = /var/log/uwsgi/%n.log
rlogdate = [ %%a %%b %%d %%Y %%H:%%M:%%S ]
logformat = %(method) %(status) %(addr) %(uri) wid: %(wid)
log-encoder = format ${strftime:[ %%a %%b %%d %%Y %%H:%%M:%%S ]} ${msgnl}
gid = ubuntu
uid = ubuntu
vhost = false
socket = 127.0.0.1:9000
master = true
processes = 3
harakiri = 20
limit-as = 128
memory-report
no-orphans
有人告诉我我有一个相当糟糕的意外系统管理员综合症案例，所以任何 help/advice 都表示赞赏。只是为了更好的衡量，这里是 proc/meminfo
的输出
MemTotal: 7657772 kB
MemFree: 5971508 kB
MemAvailable: 7145084 kB
Buffers: 225528 kB
Cached: 900756 kB
SwapCached: 0 kB
Active: 1142912 kB
Inactive: 358288 kB
Active(anon): 380568 kB
Inactive(anon): 11956 kB
Active(file): 762344 kB
Inactive(file): 346332 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 48 kB
Writeback: 0 kB
AnonPages: 375008 kB
Mapped: 55840 kB
Shmem: 17616 kB
Slab: 129300 kB
SReclaimable: 106084 kB
SUnreclaim: 23216 kB
KernelStack: 3120 kB
PageTables: 6672 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 3828884 kB
Committed_AS: 602572 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 0 kB
VmallocChunk: 0 kB
HardwareCorrupted: 0 kB
AnonHugePages: 143360 kB
CmaTotal: 0 kB
CmaFree: 0 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
DirectMap4k: 71680 kB
DirectMap2M: 7923712 kB
您没有在您的 uWSGI .ini 中设置 max-requests。因此，您的 python 解释器永远存在并且可能会积累垃圾。尝试将其设置为某些内容，例如5000，希望泄漏会停止。
如果你以后得到更多的统计结果并且他们反驳了max-requests版本，请分享。祝你好运，James Fitter。
我有一个 python Apple MDM 服务器，我正在尝试将其从 运行 从命令行进程转换为使用 Nginx 和 uWSGI 托管。我正在使用 web.py 进行路由。一切都运行宁宁最合适的顺利。我已经有了我认为对 Nginx 和 uWSGI 都很好的基本配置，但是今天早上 运行 将其设置不到 24 小时后，我在服务器收到的所有其他请求上都遇到了这个错误。有足够的可用内存，我正在使用 python.
的 64 位版本编辑 1：SSL 在端口 8080 上通信，因为它前面有一个负载均衡器将 SSL 流量路由到该端口。
EDIT2：我已经将 uWSGI 中的进程设置从 3 更改为 1 以查看是否是问题所在。
EDIT3：这是来自 AWS 的新盒子。安装了我的服务器及其依赖项、nginx 和 uWSGI。我在使用 运行 将 uWSGI 作为服务时遇到了麻烦，在我最终获得 pip 版本 运行 之前安装和卸载了它的 aptitude 和 pip 版本 systemd。这就是现在 运行ning 的方式，它正在优雅地停止、启动和重新启动。
[ Tue Jun 14 2016 10:29:22 ] Traceback (most recent call last):
File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 237, in process
File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 565, in processor
File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 77, in reload_mapping
MemoryError
[ Tue Jun 14 2016 10:29:22 ]
[ Tue Jun 14 2016 10:29:22 ] Traceback (most recent call last):
File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 237, in process
File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 566, in processor
File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 237, in <lambda>
File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 246, in process
File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 477, in internalerror
MemoryError
[ Tue Jun 14 2016 10:29:22 ]
[ Tue Jun 14 2016 10:29:22 ] Traceback (most recent call last):
File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 237, in process
File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 581, in processor
File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 237, in <lambda>
File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 246, in process
File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 477, in internalerror
MemoryError
[ Tue Jun 14 2016 10:29:22 ]
[ Tue Jun 14 2016 10:29:22 ] Traceback (most recent call last):
File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 237, in process
File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 566, in processor
File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 237, in <lambda>
File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 246, in process
File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 477, in internalerror
MemoryError
[ Tue Jun 14 2016 10:29:22 ]
[ Tue Jun 14 2016 10:29:22 ] Traceback (most recent call last):
[ Tue Jun 14 2016 10:29:22 ] File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 279, in wsgi
[ Tue Jun 14 2016 10:29:22 ] [ Tue Jun 14 2016 10:29:22 ] result = self.handle_with_processors()
[ Tue Jun 14 2016 10:29:22 ] File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 249, in handle_with_processors
[ Tue Jun 14 2016 10:29:22 ] [ Tue Jun 14 2016 10:29:22 ] return process(self.processors)
[ Tue Jun 14 2016 10:29:22 ] File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 246, in process
[ Tue Jun 14 2016 10:29:22 ] [ Tue Jun 14 2016 10:29:22 ] raise self.internalerror()
[ Tue Jun 14 2016 10:29:22 ] File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 477, in internalerror
[ Tue Jun 14 2016 10:29:22 ] [ Tue Jun 14 2016 10:29:22 ] import debugerror
[ Tue Jun 14 2016 10:29:22 ] MemoryError[ Tue Jun 14 2016 10:29:22 ]
nginx.conf
user www-data;
worker_processes auto;
pid /run/nginx.pid;
events {
worker_connections 768;
# multi_accept on;
}
http {
##
# Basic Settings
##
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
# server_tokens off;
# server_names_hash_bucket_size 64;
# server_name_in_redirect off;
include /etc/nginx/mime.types;
default_type application/octet-stream;
##
# SSL Settings
##
ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
ssl_prefer_server_ciphers on;
##
# Logging Settings
##
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
##
# Gzip Settings
##
gzip on;
gzip_disable "msie6";
# gzip_vary on;
# gzip_proxied any;
# gzip_comp_level 6;
# gzip_buffers 16 8k;
# gzip_http_version 1.1;
# gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;
##
# Virtual Host Configs
##
include /etc/nginx/conf.d/*.conf;
include /etc/nginx/sites-enabled/*;
}
#mail {
# # See sample authentication script at:
# # http://wiki.nginx.org/ImapAuthenticateWithApachePhpScript
#
# # auth_http localhost/auth.php;
# # pop3_capabilities "TOP" "USER";
# # imap_capabilities "IMAP4rev1" "UIDPLUS";
#
# server {
# listen localhost:110;
# protocol pop3;
# proxy on;
# }
#
# server {
# listen localhost:143;
# protocol imap;
# proxy on;
# }
#}
启用站点
server {
listen 80;
location ^~ /static/ {
root /path/to/app/;
}
location / {
include uwsgi_params;
uwsgi_pass 127.0.0.1:9000;
}
}
server {
listen 8080;
ssl on;
ssl_certificate /path/to/app/certs/ssl.crt;
ssl_certificate_key /path/to/app/certs/ssl.key;
location ^~ /static/ {
root /path/to/app/static/;
}
location / {
include uwsgi_params;
uwsgi_pass 127.0.0.1:9000;
uwsgi_param UWSGI_CHDIR /path/to/app/;
uwsgi_param UWSGI_PYHOME /path/to/app/virtualenv/;
uwsgi_param UWSGI_SCRIPT server;
}
}
uWSGI ini
home = /path/to/app/virtualenv/
chdir = /path/to/app/
module = server:application
logto = /var/log/uwsgi/%n.log
rlogdate = [ %%a %%b %%d %%Y %%H:%%M:%%S ]
logformat = %(method) %(status) %(addr) %(uri) wid: %(wid)
log-encoder = format ${strftime:[ %%a %%b %%d %%Y %%H:%%M:%%S ]} ${msgnl}
gid = ubuntu
uid = ubuntu
vhost = false
socket = 127.0.0.1:9000
master = true
processes = 3
harakiri = 20
limit-as = 128
memory-report
no-orphans
有人告诉我我有一个相当糟糕的意外系统管理员综合症案例，所以任何 help/advice 都表示赞赏。只是为了更好的衡量，这里是 proc/meminfo
MemTotal: 7657772 kB
MemFree: 5971508 kB
MemAvailable: 7145084 kB
Buffers: 225528 kB
Cached: 900756 kB
SwapCached: 0 kB
Active: 1142912 kB
Inactive: 358288 kB
Active(anon): 380568 kB
Inactive(anon): 11956 kB
Active(file): 762344 kB
Inactive(file): 346332 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 48 kB
Writeback: 0 kB
AnonPages: 375008 kB
Mapped: 55840 kB
Shmem: 17616 kB
Slab: 129300 kB
SReclaimable: 106084 kB
SUnreclaim: 23216 kB
KernelStack: 3120 kB
PageTables: 6672 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 3828884 kB
Committed_AS: 602572 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 0 kB
VmallocChunk: 0 kB
HardwareCorrupted: 0 kB
AnonHugePages: 143360 kB
CmaTotal: 0 kB
CmaFree: 0 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
DirectMap4k: 71680 kB
DirectMap2M: 7923712 kB
您没有在您的 uWSGI .ini 中设置 max-requests。因此，您的 python 解释器永远存在并且可能会积累垃圾。尝试将其设置为某些内容，例如5000，希望泄漏会停止。
如果你以后得到更多的统计结果并且他们反驳了max-requests版本，请分享。祝你好运，James Fitter。