Nodejs 服务器的 Nginx 反向代理 SSL_ERROR_RX_RECORD_TOO_LONG

Nginx reverse proxy for Nodejs server SSL_ERROR_RX_RECORD_TOO_LONG

我正在使用 AWS Beanstalk 配置一个多容器 docker 环境来为我的 PHP Docker 应用程序并行服务我的 NodeJS 服务器,运行 在端口 3000 上.

我有一个 运行 Express 服务器,监听 端口 3000。我现在希望能够在 https://nodejs.my-domain.com:3000 调用我的 NodeJS 服务器。 Nginx 现在应该终止 SSL 连接并将所有流量转发到我的 NodeJS Express 服务器。

到目前为止,无论是否使用 https,我都可以成功访问我的 PHP 应用程序。我也可以在 http://nodejs.my-domain.com:3000 访问我的 NodeJS 应用程序而不需要 SSL。但是,一旦我用 https 调用它,我就会收到 Broser 错误 SSL_ERROR_RX_RECORD_TOO_LONG.

Nginx 配置文件如下所示:

log_format healthd '$msec"$uri"'
          '$status"$request_time"$upstream_response_time"'
          '$http_x_forwarded_for';

upstream nodejs {
    server 127.0.0.1:3000;
    keepalive 256;
}

server {
    listen 80;
    listen [::]:80;
    listen 443 ssl;
    listen [::]:443 ssl;

    server_name nodejs.my-domain.com

    ssl_certificate /etc/nginx/certs/nginx-selfsigned.crt;
    ssl_certificate_key /etc/nginx/certs/nginx-selfsigned.key;
    ssl_session_timeout 1d;
    ssl_session_cache shared:MozSSL:10m;
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384;
    ssl_prefer_server_ciphers off;

    if ($time_iso8601 ~ "^(\d{4})-(\d{2})-(\d{2})T(\d{2})") {
        set $year ;
        set $month ;
        set $day ;
        set $hour ;
    }

    access_log /var/log/nginx/access.log main;
    access_log /var/log/nginx/healthd/application.log.$year-$month-$day-$hour healthd;

    location / {
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $http_host;
        proxy_set_header X-NginX-Proxy true;

        proxy_pass http://nodejs;
        proxy_redirect off;
    }
}

server {
    listen 80;
    listen [::]:80;
    listen 443 ssl http2 default_server;
    listen [::]:443 ssl http2 default_server;

    server_name localhost my-domain.com;
    root /var/www/public;

    ssl_certificate /etc/nginx/certs/nginx-selfsigned.crt;
    ssl_certificate_key /etc/nginx/certs/nginx-selfsigned.key;
    ssl_session_timeout 1d;
    ssl_session_cache shared:MozSSL:10m;
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384;
    ssl_prefer_server_ciphers off;

    if ($time_iso8601 ~ "^(\d{4})-(\d{2})-(\d{2})T(\d{2})") {
        set $year ;
        set $month ;
        set $day ;
        set $hour ;
    }

    access_log /var/log/nginx/access.log main;
    access_log /var/log/nginx/healthd/application.log.$year-$month-$day-$hour healthd;

    index index.php index.html index.htm;

    if ($ssl_protocol = "") {
        rewrite ^ https://$host$request_uri? permanent;
    }

    location / {
        try_files $uri $uri/ /index.php?$args;
    }

    location ~ [^/]\.php(/|$) {
        try_files $uri =404;
        fastcgi_split_path_info ^(.+?\.php)(/.*)$;
        include fastcgi_params;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        fastcgi_param PATH_INFO $fastcgi_path_info;
        fastcgi_param PATH_TRANSLATED $document_root$fastcgi_path_info;

        fastcgi_pass php:9000;
        fastcgi_index index.php;
    }
}

我不确定我必须注意哪些日志。以下是各种日志文件的一些输出:

我的 access.log 看起来像这样:

XX.X.XXX.X - - [18/Mar/2020:12:12:12 +0000] "GET / HTTP/1.1" 502 559 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36" "-"
XXX.XX.XX.XX - - [18/Mar/2020:12:27:09 +0000] "GET / HTTP/1.1" 502 157 "-" "Mozilla/5.0 zgrab/0.x" "-"
XX.XXX.XX.XXX - - [18/Mar/2020:12:54:16 +0000] "GET http://example.com/ HTTP/1.1" 502 157 "-" "AWS Security Scanner" "-"
XX.XXX.XX.XXX - - [18/Mar/2020:12:54:17 +0000] "GET http://XXX.XXX.XXX.XXX/ HTTP/1.1" 502 157 "-" "AWS Security Scanner" "-"
XX.XXX.XX.XXX - - [18/Mar/2020:12:54:17 +0000] "GET http://[::XXXX:XXXXX:XXXXX]/ HTTP/1.1" 502 157 "-" "AWS Security Scanner" "-"
XX.XXX.XX.XXX - - [18/Mar/2020:12:54:17 +0000] "GET http://XXX.XXX.XXX.XXX/latest/dynamic/instance-identity/document HTTP/1.1" 502 157 "-" "AWS Security Scanner" "-"
XX.XXX.XX.XXX - - [18/Mar/2020:12:54:18 +0000] "GET http://[::XXXX:XXXXX:XXXXX]/latest/dynamic/instance-identity/document HTTP/1.1" 502 157 "-" "AWS Security Scanner" "-"
XX.XXX.XX.XXX - - [18/Mar/2020:12:54:18 +0000] "GET / HTTP/1.1" 502 157 "-" "AWS Security Scanner" "-"
XX.XXX.XX.XXX - - [18/Mar/2020:12:54:18 +0000] "GET / HTTP/1.1" 502 157 "-" "AWS Security Scanner" "-"
XX.XXX.XX.XXX - - [18/Mar/2020:12:54:19 +0000] "GET / HTTP/1.1" 502 157 "-" "AWS Security Scanner" "-"
XX.XXX.XX.XXX - - [18/Mar/2020:12:54:19 +0000] "GET /latest/dynamic/instance-identity/document HTTP/1.1" 502 157 "-" "AWS Security Scanner" "-"
XX.XXX.XX.XXX - - [18/Mar/2020:12:54:19 +0000] "GET /latest/dynamic/instance-identity/document HTTP/1.1" 502 157 "-" "AWS Security Scanner" "-"
XX.XXX.XX.XXX - - [18/Mar/2020:12:54:20 +0000] "CONNECT X.XXX.XXX.XXX:80 HTTP/1.0" 400 157 "-" "-" "-"
XX.XXX.XX.XXX - - [18/Mar/2020:12:54:20 +0000] "CONNECT X.XXX.XXX.XXX:80 HTTP/1.0" 400 157 "-" "-" "-"
XX.XXX.XX.XXX - - [18/Mar/2020:12:54:20 +0000] "CONNECT X.XXX.XXX.XXX:80 HTTP/1.0" 400 157 "-" "-" "-"
XX.XXX.XX.XXX - - [18/Mar/2020:12:54:21 +0000] "CONNECT X.XXX.XXX.XXX:80 HTTP/1.0" 400 157 "-" "-" "-"
XX.XXX.XX.XXX - - [18/Mar/2020:12:54:21 +0000] "CONNECT X.XXX.XXX.XXX:80 HTTP/1.0" 400 157 "-" "-" "-"
XX.XXX.XX.XXX - - [18/Mar/2020:12:54:21 +0000] "\x16\x03\x01\x00\xD2\x01\x00\x00\xCE\x03\x03\x11\xB9\xBB\xFD\xF6a\xD4\xAFQ\x1F\xC0\x99j\xFA#\xBCX\xF9A}'\xC9\x00\xF9\x98K0\x88\xBA\xEA\xC0\x09\x00\x00b\xC00\xC0,\xC0/\xC0+\x00\x9F\x00\x9E\xC02\xC0.\xC01\xC0-\x00\xA5\x00\xA1\x00\xA4\x00\xA0\xC0(\xC0$\xC0\x14\xC0" 400 157 "-" "-" "-"
XX.XXX.XX.XXX - - [18/Mar/2020:12:54:22 +0000] "\x16\x03\x01\x00\xD2\x01\x00\x00\xCE\x03\x03\xD7\xED\xA5|\xF8u\xCA\x1C\xD17r\x8B1\xD5\x8F\xD07\x9C\xD7Y\x06h" 400 157 "-" "-" "-"
XX.XXX.XX.XXX - - [18/Mar/2020:12:54:22 +0000] "\x16\x03\x01\x00\xD2\x01\x00\x00\xCE\x03\x033':\xC6\xE6\x90\xA8M" 400 157 "-" "-" "-"
XX.XXX.XX.XXX - - [18/Mar/2020:12:54:22 +0000] "\x16\x03\x01\x00\xD2\x01\x00\x00\xCE\x03\x03" 400 157 "-" "-" "-"
XX.XXX.XX.XXX - - [18/Mar/2020:12:54:23 +0000] "\x16\x03\x01\x00\xD2\x01\x00\x00\xCE\x03\x03\xCB=\xFAi\xFA\x8F\x08\x1E\x98\xCEc\x19\x18\xDD\xA0\xAE\xC4{\x18E\xFD\xC2z\xC3\x97\xB5\x97\xFEW\xC0\xA6~\x00\x00b\xC00\xC0,\xC0/\xC0+\x00\x9F\x00\x9E\xC02\xC0.\xC01\xC0-\x00\xA5\x00\xA1\x00\xA4\x00\xA0\xC0(\xC0$\xC0\x14\xC0" 400 157 "-" "-" "-"

在我的 error.log 中,我发现了以下内容:

2020/03/18 11:01:40 [warn] 1#1: "ssl_stapling" ignored, issuer certificate not found for certificate "/etc/nginx/certs/nginx-selfsigned.crt"
2020/03/18 11:01:40 [warn] 1#1: "ssl_stapling" ignored, issuer certificate not found for certificate "/etc/nginx/certs/nginx-selfsigned.crt"
2020/03/18 11:14:44 [warn] 1#1: "ssl_stapling" ignored, issuer certificate not found for certificate "/etc/nginx/certs/nginx-selfsigned.crt"
2020/03/18 11:14:44 [warn] 1#1: "ssl_stapling" ignored, issuer certificate not found for certificate "/etc/nginx/certs/nginx-selfsigned.crt"

我的 healthd 登录配置似乎也有问题。我的 healthd/deamon.log:

中有很多这样的条目
# Logfile created on 2020-03-17 20:33:13 +0000 by logger.rb/47272
A, [2020-03-17T20:33:14.155980 #2972]   ANY -- : healthd daemon 1.0.3 initialized
W, [2020-03-17T20:33:14.249690 #2972]  WARN -- : log file "/var/log/nginx/healthd/application.log.2020-03-17-20" does not exist
W, [2020-03-17T20:33:14.249690 #2972]  WARN -- : log file "/var/log/nginx/healthd/application.log.2020-03-17-20" does not exist
[...]
A, [2020-03-17T20:34:03.782734 #4025]   ANY -- : healthd daemon 1.0.3 initialized
W, [2020-03-17T20:34:03.858118 #4025]  WARN -- : log file "/var/log/containers/nginx-proxy/healthd/application.log.2020-03-17-20" does not exist
W, [2020-03-17T20:34:03.858118 #4025]  WARN -- : log file "/var/log/containers/nginx-proxy/healthd/application.log.2020-03-17-20" does not exist
[...]

请尝试将 TLS1.2 添加到支持的 TLS 协议列表中。

ssl_protocols TLSv1.2 TLSv1.3;

您可以使用 openssl cli 查看支持的 TLS 版本。

openssl s_client -connect my-domain.com:443 -tls1_2

如果您获得证书链和握手,则支持 TLS 版本。

我在我的 CentOS7 虚拟机上用一个简单的 nodejs 应用测试了你的配置:

upstream nodejs {

        server localhost:3000;
        keepalive 256;
}


server {

        listen 443 http2 ssl default_server;
        listen [::]:443 http2 ssl;
        listen 80;
        listen [::]:80;
        ssl_certificate /etc/nginx/ssl/nginx.crt;
        ssl_certificate_key /etc/nginx/ssl/nginx.key;
        ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
        ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES1$
        ssl_prefer_server_ciphers off;

        location / {
                proxy_http_version 1.1;
                proxy_set_header Upgrade $http_upgrade;
                proxy_set_header Connection "upgrade";
                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header Host $http_host;
                proxy_set_header X-NginX-Proxy true;
                proxy_pass http://nodejs;
        }


}

curl -ivk https://localhost:443

的输出
* About to connect() to localhost port 443 (#0)
*   Trying ::1...
* Connected to localhost (::1) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* skipping SSL peer certificate verification
* SSL connection using TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
* Server certificate:
*   subject: O=Default Company Ltd,L=Default City,C=XX
*   start date: Mar 02 08:28:49 2020 GMT
*   expire date: Mar 02 08:28:49 2021 GMT
*   common name: (nil)
*   issuer: O=Default Company Ltd,L=Default City,C=XX
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: localhost
> Accept: */*
>
< HTTP/1.1 200 OK
HTTP/1.1 200 OK
< Server: nginx/1.17.6
Server: nginx/1.17.6
< Date: Mon, 02 Mar 2020 08:55:02 GMT
Date: Mon, 02 Mar 2020 08:55:02 GMT
< Content-Type: text/html; charset=utf-8
Content-Type: text/html; charset=utf-8
< Content-Length: 4
Content-Length: 4
< Connection: keep-alive
Connection: keep-alive
< X-Powered-By: Express
X-Powered-By: Express
< ETag: W/"4-5f2c/g6AOREdVLWI53srsMrUHDo"
ETag: W/"4-5f2c/g6AOREdVLWI53srsMrUHDo"

<
* Connection #0 to host localhost left intact
done

没问题。我建议以调试模式启动 nginx 以查看更详细的日志。

systemctl stop nginx.service && systemctl start nginx-debug.service

请注意:调试级别将创建巨大的日志文件。确保不要使用时间过长。

将此添加到您的配置中。 error_log /var/log/nginx/debug.log debug;

您使用的是哪个版本的 NGINX?

我找到了解决问题的办法。我遗漏了两件事:

  1. 我正在使用多容器 docker 环境,Nginx 和 Nodejs 都 运行 在它们自己的容器中。为了让 Nginx 能够访问我的 Nodejs 服务器,我必须在 Nginx 和 Nodejs 之间创建一个 link。但是在我的配置文件中,我在 Nodejs 部分而不是 Nginx 部分配置了 link。我现在将 "links": ["node"] 添加到 Dockerrun.aws.conf 中的 Nginx 部分,现在看起来像这样:
{
    "AWSEBDockerrunVersion": 2,
    "containerDefinitions": [
        {
            "name": "nginx-proxy",
            "image": "MY_NGINX_IMAGE_ON_PRIVATE_REGISTRY",
            "portMappings": [
                {
                    "containerPort": 80,
                    "hostPort": 80
                },
                {
                    "containerPort": 443,
                    "hostPort": 443
                }
            ],
            "links": ["node"]
            [...]
        },
        {
            "name": "node",
            "image": "MY_NODEJS_IMAGE_ON_PRIVATE_REGISTRY",
            "portMappings": [
                {
                    "containerPort": 3000,
                    "hostPort": 3000
                }
            ]
        }
        [...]
    ]
    [...]
}

  1. 此外,我没有将我的 Nginx 配置中的上游设置为 127.0.0.1:3000,而是不得不将上游设置为我的 Nodejs docker 容器,我将其命名为 nodeserver node:3000。所以我的 /var/nginx/conf.d/default.conf 现在看起来像这样:
[...]

upstream nodejs {
    server node:3000;
    keepalive 256;
}

server {
    listen 443 ssl;

    server_name websockets.my-domain.com;

    ssl_certificate /etc/letsencrypt/live/websockets.my-domain.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/websockets.my-domain.com/privkey.pem;

    [...]

    location / {
        proxy_http_version 1.1;
        proxy_set_header Upgrade ${DOLLAR}http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header X-Real-IP ${DOLLAR}remote_addr;
        proxy_set_header X-Forwarded-For ${DOLLAR}proxy_add_x_forwarded_for;
        proxy_set_header Host ${DOLLAR}http_host;
        proxy_set_header X-NginX-Proxy true;

        proxy_pass http://nodejs;
        proxy_redirect off;
    }
}

[...]