Varnish 503 后端获取失败

Varnish 503 Backend Fetch failed

我有一个服务器 运行 Ubuntu 20.04,nginx 和带有 letsencrypt SSL 的 varnish。

我已将 PHPMyAdmin 安装到默认服务器块,并为我的实际网站安装了另一个服务器块 运行 magento 2.4.3。这一切都很好,直到我安装了清漆,现在当我尝试加载任何一个站点时,我得到 503 后端获取失败错误,当我尝试从 URL 端口 8080 加载时,我得到一个“这个页面不能到达”错​​误。

我已经为两个站点配置了 nginx 来监听端口 8080,我已经将 varnish 设置为监听端口 80。我已经编辑了 magento 生成的 vcl,将主机和端口分别设置为 127.0.0.1 和 8080 作为显示在 https://devdocs.magento.com/guides/v2.4/config-guide/varnish/config-varnish-configure.html.

varnishlog 显示 backend is unhealthy 错误,但我不知道如何解决这个问题。
netstat -tulpn 的输出是:

Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:22007           0.0.0.0:*               LISTEN      48993/sshd: /usr/sb
tcp        0      0 0.0.0.0:443             0.0.0.0:*               LISTEN      302464/nginx: maste
tcp        0      0 127.0.0.1:6082          0.0.0.0:*               LISTEN      304955/varnishd
tcp        0      0 0.0.0.0:25672           0.0.0.0:*               LISTEN      42533/beam.smp
tcp        0      0 127.0.0.1:6379          0.0.0.0:*               LISTEN      44657/redis-server
tcp        0      0 127.0.0.1:9100          0.0.0.0:*               LISTEN      14734/noderig
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      304955/varnishd
tcp        0      0 0.0.0.0:8080            0.0.0.0:*               LISTEN      302464/nginx: maste
tcp        0      0 0.0.0.0:21              0.0.0.0:*               LISTEN      162503/pure-ftpd (S
tcp        0      0 127.0.0.53:53           0.0.0.0:*               LISTEN      843/systemd-resolve
tcp6       0      0 :::22007                :::*                    LISTEN      48993/sshd: /usr/sb
tcp6       0      0 :::443                  :::*                    LISTEN      302464/nginx: maste
tcp6       0      0 :::5672                 :::*                    LISTEN      42533/beam.smp
tcp6       0      0 :::3306                 :::*                    LISTEN      158505/mysqld
tcp6       0      0 ::1:6379                :::*                    LISTEN      44657/redis-server
tcp6       0      0 :::80                   :::*                    LISTEN      304955/varnishd
tcp6       0      0 :::8080                 :::*                    LISTEN      302464/nginx: maste

如您所见,varnishd 运行 在端口 80 上,nginx 在 8080 上,但它几乎就像端口 8080 没有监听一样。我启用了 ufw 并将“Nginx Full”设置为允许,我还尝试添加一条规则以允许 8080,但这没有任何区别。禁用清漆并再次尝试在端口 8080 上加载两个站点会导致相同的“无法访问页面”错误,所以我不认为这是清漆问题,而是 nginx 问题,但为什么它不在端口 8080 上监听?
例如,如果我尝试从本地服务器 curl 访问 localhost:8080,它会正常工作并按预期加载默认的 nginx 页面。我的 PHPMyAdmin 安装在一个文件夹中,也可以使用 curl 访问 localhost/phpmyadmin。这是否意味着它是防火墙问题?禁用 ufw 没有帮助。

我的 mysite.conf 我的 magento 网站是:

upstream fastcgi_backend {
        server  unix:/run/php/php7.4-fpm.sock;
}

server {
   listen 8080;
   listen [::]:8080;
   server_name staging.clicksaveandprint.com;
   return 404; # managed by Certbot
   set $MAGE_ROOT /var/www/clicksaveandprint.com;
   access_log  /var/log/nginx/clicksaveandprint.com.log;
   error_log   /var/log/nginx/clicksaveandprint.com.err;
   include /var/www/clicksaveandprint.com/nginx.conf;

   if ($host = staging.clicksaveandprint.com) {
        return 301 https://$host$request_uri;
    } # managed by Certbot

    # PHP entry point for main application
    location ~ (index|get|static|report|404|503)\.php$ {
        try_files $uri $uri/ =404;
        fastcgi_pass   fastcgi_backend;
        fastcgi_buffers 1024 4k;

        fastcgi_param  PHP_FLAG  "session.auto_start=off \n suhosin.session.cryptua=off";
        fastcgi_param  PHP_VALUE "memory_limit=4096M \n max_execution_time=18000";
        fastcgi_read_timeout 6000s;
        fastcgi_connect_timeout 6000s;

        fastcgi_index  index.php;
        fastcgi_param  SCRIPT_FILENAME  $document_root$fastcgi_script_name;
        fastcgi_param  HTTPS on;
        include        fastcgi_params;

        }
}
server {
    listen [::]:443 ssl ipv6only=on; # managed by Certbot
    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/staging.clicksaveandprint.com/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/staging.clicksaveandprint.com/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

    # OCSP stapling
    ssl_stapling                            on;
    ssl_stapling_verify                     on;

location / {
       proxy_pass http://127.0.0.1;
       proxy_set_header Host               $http_host;
       proxy_set_header X-Forwarded-Host   $http_host;
       proxy_set_header X-Real-IP          $remote_addr;
       proxy_set_header X-Forwarded-For    $proxy_add_x_forwarded_for;
       proxy_set_header Ssl-Offloaded "1";
       proxy_set_header X-Forwarded-Proto  https;
       proxy_set_header X-Forwarded-Port   443;
       #proxy_hide_header X-Varnish;
       #proxy_hide_header Via;
       proxy_set_header X-Forwarded-Proto $scheme;
       proxy_buffer_size                   128k;
       proxy_buffers                       4 256k;
       proxy_busy_buffers_size             256k;
       }

}

magento nginx.conf。这是 magento 的默认值,没有任何变化:


## Example configuration:
# upstream fastcgi_backend {
#    # use tcp connection
#    # server  127.0.0.1:9000;
#    # or socket
#    server   unix:/var/run/php/php7.4-fpm.sock;
# }
# server {
#    listen 80;
#    server_name mage.dev;
#    set $MAGE_ROOT /var/www/magento2;
#    set $MAGE_DEBUG_SHOW_ARGS 0;
#    include /vagrant/magento2/nginx.conf.sample;
# }
#
## Optional override of deployment mode. We recommend you use the
## command 'bin/magento deploy:mode:set' to switch modes instead.
##
## set $MAGE_MODE default; # or production or developer
##
## If you set MAGE_MODE in server config, you must pass the variable into the
## PHP entry point blocks, which are indicated below. You can pass
## it in using:
##
## fastcgi_param  MAGE_MODE $MAGE_MODE;
##
## In production mode, you should uncomment the 'expires' directive in the /static/ location block

# Modules can be loaded only at the very beginning of the Nginx config file, please move the line below to the main config file
# load_module /etc/nginx/modules/ngx_http_image_filter_module.so;

root $MAGE_ROOT/pub;

index index.php;
autoindex off;
charset UTF-8;
error_page 404 403 = /errors/404.php;
#add_header "X-UA-Compatible" "IE=Edge";


# Deny access to sensitive files
location /.user.ini {
    deny all;
}

# PHP entry point for setup application
location ~* ^/setup($|/) {
    root $MAGE_ROOT;
    location ~ ^/setup/index.php {
        fastcgi_pass   fastcgi_backend;

        fastcgi_param  PHP_FLAG  "session.auto_start=off \n suhosin.session.cryptua=off";
        fastcgi_param  PHP_VALUE "memory_limit=756M \n max_execution_time=600";
        fastcgi_read_timeout 600s;
        fastcgi_connect_timeout 600s;

        fastcgi_index  index.php;
        fastcgi_param  SCRIPT_FILENAME  $document_root$fastcgi_script_name;
        include        fastcgi_params;
    }

    location ~ ^/setup/(?!pub/). {
        deny all;
    }

    location ~ ^/setup/pub/ {
        add_header X-Frame-Options "SAMEORIGIN";
    }
}

# PHP entry point for update application
location ~* ^/update($|/) {
    root $MAGE_ROOT;

    location ~ ^/update/index.php {
        fastcgi_split_path_info ^(/update/index.php)(/.+)$;
        fastcgi_pass   fastcgi_backend;
        fastcgi_index  index.php;
        fastcgi_param  SCRIPT_FILENAME  $document_root$fastcgi_script_name;
        fastcgi_param  PATH_INFO        $fastcgi_path_info;
        include        fastcgi_params;
    }

    # Deny everything but index.php
    location ~ ^/update/(?!pub/). {
        deny all;
    }

    location ~ ^/update/pub/ {
        add_header X-Frame-Options "SAMEORIGIN";
    }
}

location / {
    try_files $uri $uri/ /index.php$is_args$args;
}

location /pub/ {
    location ~ ^/pub/media/(downloadable|customer|import|custom_options|theme_customization/.*\.xml) {
        deny all;
    }
    alias $MAGE_ROOT/pub/;
    add_header X-Frame-Options "SAMEORIGIN";
}

location /static/ {
    # Uncomment the following line in production mode
    # expires max;

    # Remove signature of the static files that is used to overcome the browser cache
    location ~ ^/static/version\d*/ {
        rewrite ^/static/version\d*/(.*)$ /static/ last;
    }

    location ~* \.(ico|jpg|jpeg|png|gif|svg|svgz|webp|avif|avifs|js|css|eot|ttf|otf|woff|woff2|html|json|webmanifest)$ {
        add_header Cache-Control "public";
        add_header X-Frame-Options "SAMEORIGIN";
        expires +1y;

        if (!-f $request_filename) {
            rewrite ^/static/(version\d*/)?(.*)$ /static.php?resource= last;
        }
    }
    location ~* \.(zip|gz|gzip|bz2|csv|xml)$ {
        add_header Cache-Control "no-store";
        add_header X-Frame-Options "SAMEORIGIN";
        expires    off;

        if (!-f $request_filename) {
           rewrite ^/static/(version\d*/)?(.*)$ /static.php?resource= last;
        }
    }
    if (!-f $request_filename) {
        rewrite ^/static/(version\d*/)?(.*)$ /static.php?resource= last;
    }
    add_header X-Frame-Options "SAMEORIGIN";
}

location /media/ {

## The following section allows to offload image resizing from Magento instance to the Nginx.
## Catalog image URL format should be set accordingly.
## See https://docs.magento.com/user-guide/configuration/general/web.html#url-options
#   location ~* ^/media/catalog/.* {
#
#       # Replace placeholders and uncomment the line below to serve product images from public S3
#       # See examples of S3 authentication at https://github.com/anomalizer/ngx_aws_auth
#       # resolver 8.8.8.8;
#       # proxy_pass https://<bucket-name>.<region-name>.amazonaws.com;
#
#       set $width "-";
#       set $height "-";
#       if ($arg_width != '') {
#           set $width $arg_width;
#       }
#       if ($arg_height != '') {
#           set $height $arg_height;
#       }
#       image_filter resize $width $height;
#       image_filter_jpeg_quality 90;
#   }

    try_files $uri $uri/ /get.php$is_args$args;

    location ~ ^/media/theme_customization/.*\.xml {
        deny all;
    }

    location ~* \.(ico|jpg|jpeg|png|gif|svg|svgz|webp|avif|avifs|js|css|eot|ttf|otf|woff|woff2)$ {
        add_header Cache-Control "public";
        add_header X-Frame-Options "SAMEORIGIN";
        expires +1y;
        try_files $uri $uri/ /get.php$is_args$args;
    }
    location ~* \.(zip|gz|gzip|bz2|csv|xml)$ {
        add_header Cache-Control "no-store";
        add_header X-Frame-Options "SAMEORIGIN";
        expires    off;
        try_files $uri $uri/ /get.php$is_args$args;
    }
    add_header X-Frame-Options "SAMEORIGIN";
}

location /media/customer/ {
    deny all;
}

location /media/downloadable/ {
    deny all;
}

location /media/import/ {
    deny all;
}

location /media/custom_options/ {
    deny all;
}

location /errors/ {
    location ~* \.xml$ {
        deny all;
    }
}

# PHP entry point for main application
location ~ ^/(index|get|static|errors/report|errors/404|errors/503|health_check)\.php$ {
    try_files $uri =404;
    fastcgi_pass   fastcgi_backend;
    fastcgi_buffers 16 16k;
    fastcgi_buffer_size 32k;

    fastcgi_param  PHP_FLAG  "session.auto_start=off \n suhosin.session.cryptua=off";
    fastcgi_param  PHP_VALUE "memory_limit=756M \n max_execution_time=18000";
    fastcgi_read_timeout 600s;
    fastcgi_connect_timeout 600s;

    fastcgi_index  index.php;
    fastcgi_param  SCRIPT_FILENAME  $document_root$fastcgi_script_name;
    include        fastcgi_params;
}

gzip on;
gzip_disable "msie6";

gzip_comp_level 6;
gzip_min_length 1100;
gzip_buffers 16 8k;
gzip_proxied any;
gzip_types
    text/plain
    text/css
    text/js
    text/xml
    text/javascript
    application/javascript
    application/x-javascript
    application/json
    application/xml
    application/xml+rss
    image/svg+xml;
gzip_vary on;

# Banned locations (only reached if the earlier PHP entry point regexes don't match)
location ~* (\.php$|\.phtml$|\.htaccess$|\.git) {
    deny all;
}

编辑: 将端口 8080 添加到 ufw 现在允许我的 phpmyadmin 在连接到端口 8080 时工作,但是当我连接到端口 80 上的任何内容时仍然出现 503 错误清漆启用。

sudo varnishlog -g raw -i backend_health 的输出:

varnishlog -g raw -i backend_health
         0 Backend_health - default Still sick 4---X-R- 0 5 10 0.000624 0.000000 HTTP/1.1 404 Not Found
         0 Backend_health - default Still sick 4---X-R- 0 5 10 0.000621 0.000000 HTTP/1.1 404 Not Found
         0 Backend_health - default Still sick 4---X-R- 0 5 10 0.000652 0.000000 HTTP/1.1 404 Not Found
         0 Backend_health - default Still sick 4---X-R- 0 5 10 0.000210 0.000000 HTTP/1.1 404 Not Found
         0 Backend_health - default Still sick 4---X-R- 0 5 10 0.000214 0.000000 HTTP/1.1 404 Not Found

命令 sudo varnishlog -g request -q "RespStatus == 503" 的输出:

sudo varnishlog -g request -q "RespStatus == 503"
*   << Request  >> 11
-   Begin          req 10 rxreq
-   Timestamp      Start: 1641551164.244422 0.000000 0.000000
-   Timestamp      Req: 1641551164.244422 0.000000 0.000000
-   VCL_use        boot
-   ReqStart       127.0.0.1 39792 a0
-   ReqMethod      GET
-   ReqURL         /
-   ReqProtocol    HTTP/1.0
-   ReqHeader      Host: staging.clicksaveandprint.com
-   ReqHeader      X-Forwarded-Host: staging.clicksaveandprint.com
-   ReqHeader      X-Real-IP: 81.107.192.226
-   ReqHeader      X-Forwarded-For: 81.107.192.226
-   ReqHeader      Ssl-Offloaded: 1
-   ReqHeader      X-Forwarded-Proto: https
-   ReqHeader      X-Forwarded-Port: 443
-   ReqHeader      X-Forwarded-Proto: https
-   ReqHeader      Connection: close
-   ReqHeader      Cache-Control: max-age=0
-   ReqHeader      Upgrade-Insecure-Requests: 1
-   ReqHeader      User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36
-   ReqHeader      Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
-   ReqHeader      Sec-GPC: 1
-   ReqHeader      Sec-Fetch-Site: none
-   ReqHeader      Sec-Fetch-Mode: navigate
-   ReqHeader      Sec-Fetch-User: ?1
-   ReqHeader      Sec-Fetch-Dest: document
-   ReqHeader      Accept-Encoding: gzip, deflate, br
-   ReqHeader      Accept-Language: en-GB,en-US;q=0.9,en;q=0.8
-   ReqHeader      Cookie: __tawkuuid=e::clicksaveandprint.com::n8RwZ5uipY2iBtEtSiR+wc7TP1jlMwD51gtYez+PiWZ4Xr/t3VxVX/BMIhUzfv0U::2; _ga=GA1.2.428376681.1618304758; _ga_01K4WXXY6B=GS1.1.1641392250.103.1.1641392407.60; form_key=VKzAhiJUg3HyT7jg; AMCVS_8F99160E571FC0427F0001
-   ReqUnset       X-Forwarded-For: 81.107.192.226
-   ReqHeader      X-Forwarded-For: 81.107.192.226, 127.0.0.1
-   VCL_call       RECV
-   ReqHeader      grace: none
-   ReqURL         /
-   ReqUnset       Accept-Encoding: gzip, deflate, br
-   ReqHeader      Accept-Encoding: gzip
-   VCL_return     hash
-   VCL_call       HASH
-   VCL_return     lookup
-   VCL_call       MISS
-   VCL_return     fetch
-   Link           bereq 12 fetch
-   Timestamp      Fetch: 1641551164.245036 0.000614 0.000614
-   RespProtocol   HTTP/1.1
-   RespStatus     503
-   RespReason     Backend fetch failed
-   RespHeader     Date: Fri, 07 Jan 2022 10:26:04 GMT
-   RespHeader     Server: Varnish
-   RespHeader     content-type: text/html; charset=utf-8
-   RespHeader     Retry-After: 5
-   RespHeader     X-Varnish: 11
-   RespHeader     Age: 0
-   RespHeader     Via: 1.1 varnish (Varnish/6.2)
-   VCL_call       DELIVER
-   RespUnset      Age: 0
-   RespHeader     Pragma: no-cache
-   RespHeader     Expires: -1
-   RespHeader     Cache-Control: no-store, no-cache, must-revalidate, max-age=0
-   RespUnset      Server: Varnish
-   RespUnset      X-Varnish: 11
-   RespUnset      Via: 1.1 varnish (Varnish/6.2)
-   VCL_return     deliver
-   Timestamp      Process: 1641551164.245126 0.000704 0.000090
-   Filters
-   RespHeader     Content-Length: 279
-   RespHeader     Connection: close
-   Timestamp      Resp: 1641551164.245228 0.000807 0.000103
-   ReqAcct        1233 0 1233 264 279 543
-   End
**  << BeReq    >> 12
--  Begin          bereq 11 fetch
--  VCL_use        boot
--  Timestamp      Start: 1641551164.244718 0.000000 0.000000
--  BereqMethod    GET
--  BereqURL       /
--  BereqProtocol  HTTP/1.0
--  BereqHeader    Host: staging.clicksaveandprint.com
--  BereqHeader    X-Forwarded-Host: staging.clicksaveandprint.com
--  BereqHeader    X-Real-IP: 81.107.192.226
--  BereqHeader    Ssl-Offloaded: 1
--  BereqHeader    X-Forwarded-Proto: https
--  BereqHeader    X-Forwarded-Port: 443
--  BereqHeader    X-Forwarded-Proto: https
--  BereqHeader    Upgrade-Insecure-Requests: 1
--  BereqHeader    User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36
--  BereqHeader    Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
--  BereqHeader    Sec-GPC: 1
--  BereqHeader    Sec-Fetch-Site: none
--  BereqHeader    Sec-Fetch-Mode: navigate
--  BereqHeader    Sec-Fetch-User: ?1
--  BereqHeader    Sec-Fetch-Dest: document
--  BereqHeader    Accept-Language: en-GB,en-US;q=0.9,en;q=0.8
--  BereqHeader    Cookie: __tawkuuid=e::clicksaveandprint.com::n8RwZ5uipY2iBtEtSiR+wc7TP1jlMwD51gtYez+PiWZ4Xr/t3VxVX/BMIhUzfv0U::2; _ga=GA1.2.428376681.1618304758; _ga_01K4WXXY6B=GS1.1.1641392250.103.1.1641392407.60; form_key=VKzAhiJUg3HyT7jg; AMCVS_8F99160E571FC0427F0001
--  BereqHeader    X-Forwarded-For: 81.107.192.226, 127.0.0.1
--  BereqHeader    grace: none
--  BereqHeader    Accept-Encoding: gzip
--  BereqProtocol  HTTP/1.1
--  BereqHeader    X-Varnish: 12
--  VCL_call       BACKEND_FETCH
--  VCL_return     fetch
--  FetchError     backend default: unhealthy
--  Timestamp      Beresp: 1641551164.244824 0.000107 0.000107
--  Timestamp      Error: 1641551164.244834 0.000116 0.000010
--  BerespProtocol HTTP/1.1
--  BerespStatus   503
--  BerespReason   Service Unavailable
--  BerespReason   Backend fetch failed
--  BerespHeader   Date: Fri, 07 Jan 2022 10:26:04 GMT
--  BerespHeader   Server: Varnish
--  VCL_call       BACKEND_ERROR
--  BerespHeader   content-type: text/html; charset=utf-8
--  BerespHeader   Retry-After: 5
--  VCL_return     deliver
--  Storage        malloc Transient
--  Length         279
--  BereqAcct      0 0 0 0 0 0
--  End

尝试像这样增加 first_byte_timeout 参数:

backend www {
...
    .first_byte_timeout = 100s;
...
}

能否请您提供以下命令的输出?

sudo varnishlog -g raw -i backend_health

此命令将提供有关健康探测检查状态的日志记录信息。

您得到 503 Backend fetch failed 的事实可能意味着后端不可用。

如果后端运行状况日志记录不确定,我们始终可以查看导致该 503 Backend fetch failed 错误的事务。

您可以 运行 为此执行以下命令:

sudo varnishlog -g request -q "RespStatus == 503"

请提供一些日志输出:

  • 来自后端健康日志命令的几行
  • 触发 503 Backend fetch failed
  • 的请求的完整日志事务

更新

我检查了您的 VSL 日志和运行状况检查的输出 returns HTTP 404 Not Found 错误。这意味着找不到 /health_check.php 资源。

这是 Magento 中的常见错误。如 https://www.varnish-software.com/developers/tutorials/configuring-varnish-magento/#fixing-the-backend-health-checks-for-magento-24 中所述,健康检查端点取决于您的 Magento 版本。

请将您的健康探测器的 URL 从 /health_check.php 替换为 /pub/health_check.php,反之亦然。

只需确保健康检查 URL returns HTTP 200 OK 和 Varnish 应该能够认为后端是健康的。