在 NGINX 和 Plumber API 之间添加负载平衡层 dockercloud/haproxy 后出现 502 Bad Gateway
502 Bad Gateway after adding load balancing layer dockercloud/haproxy between NGINX and Plumber API
我是管道工、dockercloud/haproxy 和 NGINX 的新手，并尝试按照 https://www.rplumber.io/articles/hosting.html 中的描述设置负载平衡。如下所示配置 nginx.conf 和 docker-compose.yml 后，我用 docker-compose up.
启动容器
docker-compose up
的日志
Starting api_plumber_1 ... done
Starting api_lb_1 ... done
Starting api_nginx_1 ... done
Attaching to api_plumber_1, api_lb_1, api_nginx_1
nginx_1 | /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
nginx_1 | /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
lb_1 | INFO:haproxy:dockercloud/haproxy 1.6.7 is running outside Docker Cloud
lb_1 | INFO:haproxy:Haproxy is running by docker-compose, loading HAProxy definition through docker api
lb_1 | INFO:haproxy:dockercloud/haproxy PID: 6
plumber_1 | ARGUMENT '/plumber.R' __ignored__
plumber_1 |
plumber_1 |
plumber_1 | R version 4.0.4 (2021-02-15) -- "Lost Library Book"
plumber_1 | Copyright (C) 2021 The R Foundation for Statistical Computing
plumber_1 | Platform: x86_64-pc-linux-gnu (64-bit)
plumber_1 |
plumber_1 | R is free software and comes with ABSOLUTELY NO WARRANTY.
plumber_1 | You are welcome to redistribute it under certain conditions.
plumber_1 | Type 'license()' or 'licence()' for distribution details.
plumber_1 |
plumber_1 | R is a collaborative project with many contributors.
plumber_1 | Type 'contributors()' for more information and
plumber_1 | 'citation()' on how to cite R or R packages in publications.
plumber_1 |
plumber_1 | Type 'demo()' for some demos, 'help()' for on-line help, or
plumber_1 | 'help.start()' for an HTML browser interface to help.
plumber_1 | Type 'q()' to quit R.
plumber_1 |
lb_1 | INFO:haproxy:=> Add task: Initial start - Compose Mode
plumber_1 | > pr <- plumber::plumb(rev(commandArgs())[1]); args <- list(host = '0.0.0.0', port = 8000); if (packageVersion('plumber') >= '1.0.0') { pr$setDocs(TRUE) } else { args$swagger <- TRUE }; do.call(pr$run, args)
nginx_1 | /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
plumber_1 | Loading required package: DBI
nginx_1 | 10-listen-on-ipv6-by-default.sh: info: IPv6 listen already enabled
nginx_1 | /docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
nginx_1 | /docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
nginx_1 | /docker-entrypoint.sh: Configuration complete; ready for start up
plumber_1 | Running plumber API at http://0.0.0.0:8000
plumber_1 | Running swagger Docs at http://127.0.0.1:8000/__docs__/
lb_1 | INFO:haproxy:=> Executing task: Initial start - Compose Mode
lb_1 | INFO:haproxy:==========BEGIN==========
lb_1 | INFO:haproxy:Linked service: api_plumber
lb_1 | INFO:haproxy:Linked container: api_plumber_1
lb_1 | INFO:haproxy:HAProxy configuration:
lb_1 | global
lb_1 | log 127.0.0.1 local0
lb_1 | log 127.0.0.1 local1 notice
lb_1 | log-send-hostname
lb_1 | maxconn 4096
lb_1 | pidfile /var/run/haproxy.pid
lb_1 | user haproxy
lb_1 | group haproxy
lb_1 | daemon
lb_1 | stats socket /var/run/haproxy.stats level admin
lb_1 | ssl-default-bind-options no-sslv3
lb_1 | ssl-default-bind-ciphers ECDHE-
lb_1 | defaults
lb_1 | balance roundrobin
lb_1 | log global
lb_1 | mode http
lb_1 | option redispatch
lb_1 | option httplog
lb_1 | option dontlognull
lb_1 | option forwardfor
lb_1 | timeout connect 5000
lb_1 | timeout client 50000
lb_1 | timeout server 50000
lb_1 | listen stats
lb_1 | bind :1936
lb_1 | mode http
lb_1 | stats enable
lb_1 | timeout connect 10s
lb_1 | timeout client 1m
lb_1 | timeout server 1m
lb_1 | stats hide-version
lb_1 | stats realm Haproxy\ Statistics
lb_1 | stats uri /
lb_1 | stats auth stats:stats
lb_1 | frontend default_port_80
lb_1 | bind :80
lb_1 | reqadd X-Forwarded-Proto:\ http
lb_1 | maxconn 4096
lb_1 | default_backend default_service
lb_1 | backend default_service
lb_1 | server api_plumber_1 api_plumber_1:8000 check inter 2000 rise 2 fall 3
lb_1 | INFO:haproxy:Launching HAProxy
lb_1 | INFO:haproxy:HAProxy has been launched(PID: 9)
lb_1 | INFO:haproxy:===========END===========
NGINX returns 如果我尝试在浏览器中访问 http://example.com/api/__docs__/ 会出现 502 Bad Gateway 错误。这是错误：
nginx_1 | 2021/03/31 11:25:06 [error] 23#23: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 123.456.789.123, server: example.com, request: "GET /api/__docs__/ HTTP/1.1", upstream: "http://172.18.0.3:8000/__docs__/", host: "example.com"
nginx_1 | 123.456.789.123 - - [31/Mar/2021:11:25:06 +0000] "GET /api/__docs__/ HTTP/1.1" 502 559 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.43.90 Safari/537.36"
当我直接使用 example.com:7000 向 lb 发送请求时，它工作正常。因此，我想我的 nginx.conf.
配置有误
docker-compose.yml
version: '2'
services:
plumber:
image: myapi:latest
command: /plumber.R
volumes:
- $PWD/plumber/plumber.R:/plumber.R
restart: always
nginx:
image: nginx
ports:
- "80:80"
volumes:
- $PWD/nginx/nginx.conf:/etc/nginx/nginx.conf:ro
restart: always
depends_on:
- plumber
- lb
lb:
image: 'dockercloud/haproxy:1.2.1''
links:
- plumber
volumes:
- /var/run/docker.sock:/var/run/docker.sock
restart: always
ports:
- "7000:80"
nginx.conf
events {
worker_connections 4096; ## Default: 1024
}
http {
default_type application/octet-stream;
sendfile on;
tcp_nopush on;
server_names_hash_bucket_size 128; # this seems to be required for some vhosts
server {
listen 80 default_server;
listen [::]:80 default_server ipv6only=on;
# root /usr/share/nginx/html;
index index.html index.htm;
server_name example.com;
location /api/ {
proxy_pass http://lb:7000/;
proxy_set_header Host $host;
}
}
}
根据日志，lb 服务正在侦听其 Docker 容器内的端口 80。将 nginx.conf 中的位置更改为
location /api/ {
proxy_pass http://lb/;
proxy_set_header Host $host;
}
错误消失。同样在docker-compose.yml部分
ports:
- "7000:80"
可以删除。
我是管道工、dockercloud/haproxy 和 NGINX 的新手，并尝试按照 https://www.rplumber.io/articles/hosting.html 中的描述设置负载平衡。如下所示配置 nginx.conf 和 docker-compose.yml 后，我用 docker-compose up.
docker-compose up
Starting api_plumber_1 ... done
Starting api_lb_1 ... done
Starting api_nginx_1 ... done
Attaching to api_plumber_1, api_lb_1, api_nginx_1
nginx_1 | /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
nginx_1 | /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
lb_1 | INFO:haproxy:dockercloud/haproxy 1.6.7 is running outside Docker Cloud
lb_1 | INFO:haproxy:Haproxy is running by docker-compose, loading HAProxy definition through docker api
lb_1 | INFO:haproxy:dockercloud/haproxy PID: 6
plumber_1 | ARGUMENT '/plumber.R' __ignored__
plumber_1 |
plumber_1 |
plumber_1 | R version 4.0.4 (2021-02-15) -- "Lost Library Book"
plumber_1 | Copyright (C) 2021 The R Foundation for Statistical Computing
plumber_1 | Platform: x86_64-pc-linux-gnu (64-bit)
plumber_1 |
plumber_1 | R is free software and comes with ABSOLUTELY NO WARRANTY.
plumber_1 | You are welcome to redistribute it under certain conditions.
plumber_1 | Type 'license()' or 'licence()' for distribution details.
plumber_1 |
plumber_1 | R is a collaborative project with many contributors.
plumber_1 | Type 'contributors()' for more information and
plumber_1 | 'citation()' on how to cite R or R packages in publications.
plumber_1 |
plumber_1 | Type 'demo()' for some demos, 'help()' for on-line help, or
plumber_1 | 'help.start()' for an HTML browser interface to help.
plumber_1 | Type 'q()' to quit R.
plumber_1 |
lb_1 | INFO:haproxy:=> Add task: Initial start - Compose Mode
plumber_1 | > pr <- plumber::plumb(rev(commandArgs())[1]); args <- list(host = '0.0.0.0', port = 8000); if (packageVersion('plumber') >= '1.0.0') { pr$setDocs(TRUE) } else { args$swagger <- TRUE }; do.call(pr$run, args)
nginx_1 | /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
plumber_1 | Loading required package: DBI
nginx_1 | 10-listen-on-ipv6-by-default.sh: info: IPv6 listen already enabled
nginx_1 | /docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
nginx_1 | /docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
nginx_1 | /docker-entrypoint.sh: Configuration complete; ready for start up
plumber_1 | Running plumber API at http://0.0.0.0:8000
plumber_1 | Running swagger Docs at http://127.0.0.1:8000/__docs__/
lb_1 | INFO:haproxy:=> Executing task: Initial start - Compose Mode
lb_1 | INFO:haproxy:==========BEGIN==========
lb_1 | INFO:haproxy:Linked service: api_plumber
lb_1 | INFO:haproxy:Linked container: api_plumber_1
lb_1 | INFO:haproxy:HAProxy configuration:
lb_1 | global
lb_1 | log 127.0.0.1 local0
lb_1 | log 127.0.0.1 local1 notice
lb_1 | log-send-hostname
lb_1 | maxconn 4096
lb_1 | pidfile /var/run/haproxy.pid
lb_1 | user haproxy
lb_1 | group haproxy
lb_1 | daemon
lb_1 | stats socket /var/run/haproxy.stats level admin
lb_1 | ssl-default-bind-options no-sslv3
lb_1 | ssl-default-bind-ciphers ECDHE-
lb_1 | defaults
lb_1 | balance roundrobin
lb_1 | log global
lb_1 | mode http
lb_1 | option redispatch
lb_1 | option httplog
lb_1 | option dontlognull
lb_1 | option forwardfor
lb_1 | timeout connect 5000
lb_1 | timeout client 50000
lb_1 | timeout server 50000
lb_1 | listen stats
lb_1 | bind :1936
lb_1 | mode http
lb_1 | stats enable
lb_1 | timeout connect 10s
lb_1 | timeout client 1m
lb_1 | timeout server 1m
lb_1 | stats hide-version
lb_1 | stats realm Haproxy\ Statistics
lb_1 | stats uri /
lb_1 | stats auth stats:stats
lb_1 | frontend default_port_80
lb_1 | bind :80
lb_1 | reqadd X-Forwarded-Proto:\ http
lb_1 | maxconn 4096
lb_1 | default_backend default_service
lb_1 | backend default_service
lb_1 | server api_plumber_1 api_plumber_1:8000 check inter 2000 rise 2 fall 3
lb_1 | INFO:haproxy:Launching HAProxy
lb_1 | INFO:haproxy:HAProxy has been launched(PID: 9)
lb_1 | INFO:haproxy:===========END===========
NGINX returns 如果我尝试在浏览器中访问 http://example.com/api/__docs__/ 会出现 502 Bad Gateway 错误。这是错误：
nginx_1 | 2021/03/31 11:25:06 [error] 23#23: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 123.456.789.123, server: example.com, request: "GET /api/__docs__/ HTTP/1.1", upstream: "http://172.18.0.3:8000/__docs__/", host: "example.com"
nginx_1 | 123.456.789.123 - - [31/Mar/2021:11:25:06 +0000] "GET /api/__docs__/ HTTP/1.1" 502 559 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.43.90 Safari/537.36"
当我直接使用 example.com:7000 向 lb 发送请求时，它工作正常。因此，我想我的 nginx.conf.
docker-compose.yml
version: '2'
services:
plumber:
image: myapi:latest
command: /plumber.R
volumes:
- $PWD/plumber/plumber.R:/plumber.R
restart: always
nginx:
image: nginx
ports:
- "80:80"
volumes:
- $PWD/nginx/nginx.conf:/etc/nginx/nginx.conf:ro
restart: always
depends_on:
- plumber
- lb
lb:
image: 'dockercloud/haproxy:1.2.1''
links:
- plumber
volumes:
- /var/run/docker.sock:/var/run/docker.sock
restart: always
ports:
- "7000:80"
nginx.conf
events {
worker_connections 4096; ## Default: 1024
}
http {
default_type application/octet-stream;
sendfile on;
tcp_nopush on;
server_names_hash_bucket_size 128; # this seems to be required for some vhosts
server {
listen 80 default_server;
listen [::]:80 default_server ipv6only=on;
# root /usr/share/nginx/html;
index index.html index.htm;
server_name example.com;
location /api/ {
proxy_pass http://lb:7000/;
proxy_set_header Host $host;
}
}
}
根据日志，lb 服务正在侦听其 Docker 容器内的端口 80。将 nginx.conf 中的位置更改为
location /api/ {
proxy_pass http://lb/;
proxy_set_header Host $host;
}
错误消失。同样在docker-compose.yml部分
ports:
- "7000:80"
可以删除。