清漆4：总是错过
Varnish 4: Always miss
我目前正在为我的域设置 Varnish 4。
除了 DOC 请求之外，我的 Varnish 像 charm 一样缓存所有文件。
但我不明白，为什么 DOC 请求总是 "MISS".
希望大家能帮帮我。
所有详细信息如下。如果您需要更多，请告诉我。
非常感谢您的帮助！
此致，
赫姆斯
清漆版本：
/ # varnishd -V
varnishd (varnish-4.1.3 revision 5e3b6d2)
Copyright (c) 2006 Verdens Gang AS
Copyright (c) 2006-2015 Varnish Software AS
卷曲请求：
$ curl -s -D - "https://www.example.com" -o /dev/null
HTTP/1.1 200 OK
Accept-Ranges: bytes
Age: 0
Cache-Control: max-age=0
Content-Type: text/html; charset=UTF-8
Date: Wed, 15 Nov 2017 20:25:14 GMT
Expires: Wed, 15 Nov 2017 20:25:14 GMT
Vary: Accept-Encoding
X-Cache: MISS
X-Cache-Age:
X-Cache-Grace:
X-Cache-Ttl-Remaining:
X-Hits: 0
X-Host: example.com
X-Varnish-Cookie-Debug: Cleaned request cookies:
Transfer-Encoding: chunked
清漆日志：
* << BeReq >> 786983
- Begin bereq 786982 pass
- Timestamp Start: 1510849297.605099 0.000000 0.000000
- BereqMethod GET
- BereqURL /
- BereqProtocol HTTP/1.1
- BereqHeader Host: www.example.com
- BereqHeader User-Agent: curl/7.26.0
- BereqHeader Accept: */*
- BereqHeader X-Forwarded-Host: www.example.com
- BereqHeader X-Forwarded-Port: 443
- BereqHeader X-Forwarded-Proto: https
- BereqHeader X-Forwarded-Server: lb-proxy-1
- BereqHeader X-Real-Ip: XXX.XX.XXX.XXX
- BereqHeader X-Forwarded-For: XXX.XX.XXX.XXX
- BereqHeader Accept-Encoding: gzip
- BereqHeader X-Device: desktop
- BereqHeader X-Varnish: 786983
- VCL_call BACKEND_FETCH
- VCL_return fetch
- BackendOpen 23 boot.default XXX.XX.XXX.XXX 80 10.42.70.133 39122
- BackendStart XXX.XX.XXX.XXX 80
- Timestamp Bereq: 1510849297.619798 0.014699 0.014699
- Timestamp Beresp: 1510849297.676958 0.071859 0.057160
- BerespProtocol HTTP/1.1
- BerespStatus 200
- BerespReason OK
- BerespHeader Date: Thu, 16 Nov 2017 16:21:37 GMT
- BerespHeader Server: Apache
- BerespHeader Cache-Control: max-age=0
- BerespHeader Expires: Thu, 16 Nov 2017 16:21:37 GMT
- BerespHeader Vary: Accept-Encoding
- BerespHeader Content-Encoding: gzip
- BerespHeader Content-Length: 3252
- BerespHeader Content-Type: text/html; charset=UTF-8
- TTL RFC 0 10 -1 1510849298 1510849298 1510849297 1510849297 0
- VCL_call BACKEND_RESPONSE
- BerespHeader X-Varnish-Cookie-Debug: Cleaned request cookies:
- BerespHeader x-url: /
- BerespHeader x-host: www.example.com
- BerespHeader X-Cache: ZIP
- BereqHeader stale-while-revalidate: max-age=0
- TTL VCL -1 7200 0 1510849298
- TTL VCL 120 7200 0 1510849298
- VCL_return deliver
- Storage malloc Transient
- ObjProtocol HTTP/1.1
- ObjStatus 200
- ObjReason OK
- ObjHeader Date: Thu, 16 Nov 2017 16:21:37 GMT
- ObjHeader Server: Apache
- ObjHeader Cache-Control: max-age=0
- ObjHeader Expires: Thu, 16 Nov 2017 16:21:37 GMT
- ObjHeader Vary: Accept-Encoding
- ObjHeader Content-Encoding: gzip
- ObjHeader Content-Length: 3252
- ObjHeader Content-Type: text/html; charset=UTF-8
- ObjHeader X-Varnish-Cookie-Debug: Cleaned request cookies:
- ObjHeader x-url: /
- ObjHeader x-host: www.example.com
- ObjHeader X-Cache: ZIP
- Fetch_Body 3 length stream
- Gzip u F - 3252 10723 80 80 25946
- BackendReuse 23 boot.default
- Timestamp BerespBody: 1510849297.677148 0.072049 0.000190
- Length 3252
- BereqAcct 328 0 328 247 3252 3499
- End
* << Request >> 786982
- Begin req 786981 rxreq
- Timestamp Start: 1510849297.605008 0.000000 0.000000
- Timestamp Req: 1510849297.605008 0.000000 0.000000
- ReqStart 10.42.115.200 35666
- ReqMethod GET
- ReqURL /
- ReqProtocol HTTP/1.1
- ReqHeader Host: www.example.com
- ReqHeader User-Agent: curl/7.26.0
- ReqHeader Accept: */*
- ReqHeader X-Forwarded-For: XXX.XX.XXX.XXX
- ReqHeader X-Forwarded-Host: www.example.com
- ReqHeader X-Forwarded-Port: 443
- ReqHeader X-Forwarded-Proto: https
- ReqHeader X-Forwarded-Server: lb-proxy-1
- ReqHeader X-Real-Ip: XXX.XX.XXX.XXX
- ReqHeader Accept-Encoding: gzip
- ReqUnset X-Forwarded-For: XXX.XX.XXX.XXX
- ReqHeader X-Forwarded-For: XXX.XX.XXX.XXX, 10.42.115.200
- VCL_call RECV
- ReqUnset X-Forwarded-For: XXX.XX.XXX.XXX, 10.42.115.200
- ReqHeader X-Forwarded-For: XXX.XX.XXX.XXX
- ReqHeader Cookie:
- ReqUnset Cookie:
- ReqHeader Cookie:
- ReqUnset Cookie:
- ReqHeader Cookie:
- ReqUnset Cookie:
- ReqHeader Cookie:
- ReqUnset Cookie:
- ReqHeader Cookie:
- ReqUnset Cookie:
- ReqHeader Cookie:
- ReqUnset Cookie:
- ReqHeader Cookie:
- ReqUnset Cookie:
- ReqHeader Cookie:
- ReqUnset Cookie:
- ReqHeader Cookie:
- ReqUnset Cookie:
- ReqHeader Cookie:
- ReqUnset Cookie:
- ReqUnset Accept-Encoding: gzip
- ReqHeader Accept-Encoding: gzip
- ReqHeader X-Device: desktop
- VCL_return hash
- VCL_call HASH
- VCL_return lookup
- Debug "XXXX HIT-FOR-PASS"
- HitPass 786980
- VCL_call PASS
- VCL_return fetch
- Link bereq 786983 pass
- Timestamp Fetch: 1510849297.677182 0.072175 0.072175
- RespProtocol HTTP/1.1
- RespStatus 200
- RespReason OK
- RespHeader Date: Thu, 16 Nov 2017 16:21:37 GMT
- RespHeader Server: Apache
- RespHeader Cache-Control: max-age=0
- RespHeader Expires: Thu, 16 Nov 2017 16:21:37 GMT
- RespHeader Vary: Accept-Encoding
- RespHeader Content-Encoding: gzip
- RespHeader Content-Length: 3252
- RespHeader Content-Type: text/html; charset=UTF-8
- RespHeader X-Varnish-Cookie-Debug: Cleaned request cookies:
- RespHeader x-url: /
- RespHeader x-host: www.example.com
- RespHeader X-Cache: ZIP
- RespHeader X-Varnish: 786982
- RespHeader Age: 0
- RespHeader Via: 1.1 varnish-v4
- VCL_call DELIVER
- RespUnset x-url: /
- RespUnset x-host: www.example.com
- RespUnset X-Cache: ZIP
- RespHeader X-Cache: MISS
- RespHeader X-Hits: 0
- RespHeader X-Host: example.com
- RespUnset Server: Apache
- RespUnset X-Varnish: 786982
- RespUnset Via: 1.1 varnish-v4
- RespHeader X-Cache-TTL-Remaining:
- RespHeader X-Cache-Age:
- RespHeader X-Cache-Grace:
- VCL_return deliver
- Timestamp Process: 1510849297.677222 0.072215 0.000040
- RespHeader Accept-Ranges: bytes
- Debug "RES_MODE 2"
- RespHeader Connection: keep-alive
- Timestamp Resp: 1510849297.677275 0.072268 0.000053
- ReqAcct 290 0 290 446 3252 3698
- End
这是我的 VCL：
vcl 4.0;
import std;
include "cache_invalidation.vcl";
include "custom_subroutines.vcl";
backend default {
.host = "VARNISH_BACKEND_ADDRESS"; # IP or Hostname of backend
.port = "VARNISH_BACKEND_PORT"; # Port Apache or whatever is listening
.connect_timeout = 5s; # Wait a maximum of 5s for backend connection (Apache, Nginx, etc...)
.first_byte_timeout = 10s; # Wait a maximum of 10s for the first byte to come from your backend
.between_bytes_timeout = 2s; # Wait a maximum of 2s between each bytes sent
.max_connections = 300; # Max parallel connections to our backend
}
sub vcl_recv {
# IP forwarding.
if (req.restarts == 0) {
if (req.http.X-Forwarded-For && !req.http.X-Real-IP) {
set req.http.X-Real-IP = regsub(req.http.X-Forwarded-For, ".*\b(?!10|127|172)(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}).*", "");
}else{
set req.http.X-Forwarded-For = req.http.X-Real-IP;
}
}
# Enforce redirect to HTTPS
if (req.http.X-Forwarded-Proto !~ "(?i)https") {
return(synth(720, "https://" + req.http.host + req.url));
}
# Remove the Google Analytics added parameters, useless for our backend
if (req.url ~ "(\?|&)(utm_source|utm_medium|utm_campaign|utm_content|gclid|cx|ie|cof|siteurl)=") {
set req.url = regsuball(req.url, "&(utm_source|utm_medium|utm_campaign|utm_content|gclid|cx|ie|cof|siteurl)=([A-z0-9_\-\.%25]+)", "");
set req.url = regsuball(req.url, "\?(utm_source|utm_medium|utm_campaign|utm_content|gclid|cx|ie|cof|siteurl)=([A-z0-9_\-\.%25]+)", "?");
set req.url = regsub(req.url, "\?&", "?");
set req.url = regsub(req.url, "\?$", "");
}
# Remove anchor or hash references
if(req.url ~ "\#"){
set req.url = regsub(req.url, "\#.*$", "");
}
# Remove incomplete query string
if(req.url ~ "\?$"){
set req.url = regsub(req.url, "\?$", "");
}
# Remove Google Analytics Cookies, our server does not need it
set req.http.Cookie = regsuball(req.http.Cookie, "__utm.=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "_ga=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "_gat=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "utmctr=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "utmcmd.=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "utmccn.=[^;]+(; )?", "");
# Remove DoubleClick offensive cookies
set req.http.Cookie = regsuball(req.http.Cookie, "__gads=[^;]+(; )?", "");
# Remove the Quant Capital cookies (added by some plugin, all __qca)
set req.http.Cookie = regsuball(req.http.Cookie, "__qc.=[^;]+(; )?", "");
# Remove the AddThis cookies
set req.http.Cookie = regsuball(req.http.Cookie, "__atuv.=[^;]+(; )?", "");
# Remove a ";" prefix in the cookie if present
set req.http.Cookie = regsuball(req.http.Cookie, "^;\s*", "");
# Are there cookies left with only spaces or that are empty?
if (req.http.cookie ~ "^\s*$") {
unset req.http.cookie;
}
if (req.url ~ "^[^?]*\.(bmp|bz2|css|doc|eot|flv|gif|gz|ico|jpeg|jpg|js|less|pdf|png|rtf|swf|txt|woff|xml)(\?.*)?$") {
unset req.http.Cookie;
return (hash);
}
if (req.url ~ "^[^?]*\.(mp[34]|rar|tar|tgz|gz|wav|zip|bz2|xz|7z|avi|mov|ogm|mpe?g|mk[av]|webm)(\?.*)?$") {
unset req.http.Cookie;
return (hash);
}
if(req.http.Accept-Encoding)
{
if(req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp[34]|ogg)$")
{
# No point compressing these extensions
unset req.http.Accept-Encoding;
} elsif(req.http.Accept-Encoding ~ "gzip")
{
set req.http.Accept-Encoding = "gzip";
} elsif(req.http.Accept-Encoding ~ "deflate")
{
set req.http.Accept-Encoding = "deflate";
} elsif(req.http.Accept-Encoding ~ "sdch")
{
set req.http.Accept-Encoding = "sdch";
} else
{
# Unknown Algorithm
unset req.http.Accept-Encoding;
}
}
}
# At this point, it is our last chance to make some decision (set|unset|mod) about some header that the backend should see.
sub vcl_backend_fetch {
unset bereq.http.X-Varnish-SESS-ID;
}
sub vcl_hit {
# Custom Headers for debug purposes
set req.http.X-Cache-TTL-Remaining = obj.ttl;
set req.http.X-Cache-Age = obj.keep - obj.ttl;
set req.http.X-Cache-Grace = obj.grace;
}
sub vcl_backend_response {
if (bereq.url ~ "^[^?]*\.(mp[34]|rar|tar|tgz|gz|wav|zip|bz2|xz|7z|avi|mov|ogm|mpe?g|mk[av]|webm)(\?.*)?$") {
unset beresp.http.set-cookie;
set beresp.do_stream = true; # Check memory usage it'll grow in fetch_chunksize blocks (128k by default) if the backend doesn't send a Content-Length header, so only enable it for big objects
set beresp.do_gzip = false; # Don't try to compress it for storage
}
# Do not Gzip image files in Varnish
if (beresp.http.url ~ "\.(jpg|jpeg|png|gif|gz|tgz|bz2|tbz|mp3|mp4|ogg|swf)$") {
set beresp.do_gzip = false;
}
else {
set beresp.do_gzip = true;
set beresp.http.X-Cache = "ZIP";
}
# GZip the cached content if possible
if (beresp.http.content-type ~ "text") {
set beresp.do_gzip = true;
}
# If we are performing a redirection 301 / 302 from the backend and our web server and varnish instances are in the same node, apache mod_rewrite could append it's port
# typically varnish :80 apache/nginx/lighthttp :8080, so the redirect can then often redirect the end-user to a URL on :8080, where it should be :80.
if(beresp.status == 301 || beresp.status == 302) {
set beresp.http.Location = regsub(beresp.http.Location, ":[0-9]+", "");
}
# The example below is explained in detail in load_balancing.vcl as part of one of the strategies that we can take to handle Varnish Grace Mode
set bereq.http.stale-while-revalidate = regsub(beresp.http.Cache-Control, ".*stale-while-revalidate\=([0-9]+).*", "");
if(std.real(bereq.http.stale-while-revalidate, 0.0) > 0)
{
set beresp.grace = std.duration(bereq.http.stale-while-revalidate + "s", 10s);
} else {
set beresp.grace = 2h;
}
}
sub vcl_deliver {
# At this point this is our last change to set or unset the response headers before deliver it
if (obj.hits > 0)
{
set resp.http.X-Cache = "HIT";
} else
{
set resp.http.X-Cache = "MISS";
}
set resp.http.X-Hits = obj.hits;
# Remove www of our host in the backend response
set resp.http.X-Host = regsub(req.http.host, "^www\.", "");
# Remove PHP , Apache and Varnish versions
unset resp.http.x-powered-by;
unset resp.http.Server;
unset resp.http.X-Varnish;
unset resp.http.Via;
# Here we are ensuring to add this headers in the response (defined in the hit subroutine) for debug purposes
set resp.http.X-Cache-TTL-Remaining = req.http.X-Cache-TTL-Remaining;
set resp.http.X-Cache-Age = req.http.X-Cache-Age;
set resp.http.X-Cache-Grace = req.http.X-Cache-Grace;
}
您提供的清漆日志与您正在执行的 curl 请求不匹配（例如错误的用户代理）。
如果它仍然相关，在您的 varnishlog 中您可以看到后端请求包含 headers :
BereqHeader Cache-Control: no-cache
BereqHeader Pragma: no-cache
后端响应包含以下内容
BerespHeader Cache-Control: max-age=0
这会阻止 Varnish 缓存响应。
你能分享 Request（不是 BackendRequest）的清漆日志部分吗？
没有它，我的猜测可能是你总是错过，因为 builtin.vcl 使你的请求成为 pass（尽管如此，你没有在你的 conf 中做任何 return (pass)后端请求来自 pass 状态
Begin bereq 1048612 pass
可以通过builtin.vcl中的这两条规则触发：
if (req.method != "GET" && req.method != "HEAD") {
/* We only deal with GET and HEAD by default */
return (pass);
}
if (req.http.Authorization || req.http.Cookie) {
/* Not cacheable by default */
return (pass);
}
编辑
这很奇怪。你不应该有任何使用 Curl 的 Cookie。
尝试确定您的 cookie 处理代码的范围并使用 req.http.Cookie（带有大写 C），如下所示：
if (req.http.Cookie) {
# Remove Google Analytics Cookies, our server does not need it
set req.http.Cookie = regsuball(req.http.Cookie, "__utm.=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "_ga=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "_gat=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "utmctr=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "utmcmd.=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "utmccn.=[^;]+(; )?", "");
# Remove DoubleClick offensive cookies
set req.http.Cookie = regsuball(req.http.Cookie, "__gads=[^;]+(; )?", "");
# Remove the Quant Capital cookies (added by some plugin, all __qca)
set req.http.Cookie = regsuball(req.http.Cookie, "__qc.=[^;]+(; )?", "");
# Remove the AddThis cookies
set req.http.Cookie = regsuball(req.http.Cookie, "__atuv.=[^;]+(; )?", "");
# Remove a ";" prefix in the cookie if present
set req.http.Cookie = regsuball(req.http.Cookie, "^;\s*", "");
# Are there cookies left with only spaces or that are empty?
if (req.http.Cookie ~ "^\s*$") {
unset req.http.Cookie;
}
}
这里还有一件事，在你的vcl日志中你可以看到Varnish在做Hit-For-Pass，也就是Varnish在缓存你的请求没有被缓存的事实。如果你想真正测试，你需要事先清除内容。
最后一件事：您的后端服务器 (apache) 以 Cache-Control : max-age =0 header 响应，这会阻止缓存，如果您想缓存内容，您应该更改它。
我目前正在为我的域设置 Varnish 4。 除了 DOC 请求之外，我的 Varnish 像 charm 一样缓存所有文件。 但我不明白，为什么 DOC 请求总是 "MISS".
希望大家能帮帮我。
所有详细信息如下。如果您需要更多，请告诉我。
非常感谢您的帮助！
此致， 赫姆斯
清漆版本：
/ # varnishd -V
varnishd (varnish-4.1.3 revision 5e3b6d2)
Copyright (c) 2006 Verdens Gang AS
Copyright (c) 2006-2015 Varnish Software AS
卷曲请求：
$ curl -s -D - "https://www.example.com" -o /dev/null
HTTP/1.1 200 OK
Accept-Ranges: bytes
Age: 0
Cache-Control: max-age=0
Content-Type: text/html; charset=UTF-8
Date: Wed, 15 Nov 2017 20:25:14 GMT
Expires: Wed, 15 Nov 2017 20:25:14 GMT
Vary: Accept-Encoding
X-Cache: MISS
X-Cache-Age:
X-Cache-Grace:
X-Cache-Ttl-Remaining:
X-Hits: 0
X-Host: example.com
X-Varnish-Cookie-Debug: Cleaned request cookies:
Transfer-Encoding: chunked
清漆日志：
* << BeReq >> 786983
- Begin bereq 786982 pass
- Timestamp Start: 1510849297.605099 0.000000 0.000000
- BereqMethod GET
- BereqURL /
- BereqProtocol HTTP/1.1
- BereqHeader Host: www.example.com
- BereqHeader User-Agent: curl/7.26.0
- BereqHeader Accept: */*
- BereqHeader X-Forwarded-Host: www.example.com
- BereqHeader X-Forwarded-Port: 443
- BereqHeader X-Forwarded-Proto: https
- BereqHeader X-Forwarded-Server: lb-proxy-1
- BereqHeader X-Real-Ip: XXX.XX.XXX.XXX
- BereqHeader X-Forwarded-For: XXX.XX.XXX.XXX
- BereqHeader Accept-Encoding: gzip
- BereqHeader X-Device: desktop
- BereqHeader X-Varnish: 786983
- VCL_call BACKEND_FETCH
- VCL_return fetch
- BackendOpen 23 boot.default XXX.XX.XXX.XXX 80 10.42.70.133 39122
- BackendStart XXX.XX.XXX.XXX 80
- Timestamp Bereq: 1510849297.619798 0.014699 0.014699
- Timestamp Beresp: 1510849297.676958 0.071859 0.057160
- BerespProtocol HTTP/1.1
- BerespStatus 200
- BerespReason OK
- BerespHeader Date: Thu, 16 Nov 2017 16:21:37 GMT
- BerespHeader Server: Apache
- BerespHeader Cache-Control: max-age=0
- BerespHeader Expires: Thu, 16 Nov 2017 16:21:37 GMT
- BerespHeader Vary: Accept-Encoding
- BerespHeader Content-Encoding: gzip
- BerespHeader Content-Length: 3252
- BerespHeader Content-Type: text/html; charset=UTF-8
- TTL RFC 0 10 -1 1510849298 1510849298 1510849297 1510849297 0
- VCL_call BACKEND_RESPONSE
- BerespHeader X-Varnish-Cookie-Debug: Cleaned request cookies:
- BerespHeader x-url: /
- BerespHeader x-host: www.example.com
- BerespHeader X-Cache: ZIP
- BereqHeader stale-while-revalidate: max-age=0
- TTL VCL -1 7200 0 1510849298
- TTL VCL 120 7200 0 1510849298
- VCL_return deliver
- Storage malloc Transient
- ObjProtocol HTTP/1.1
- ObjStatus 200
- ObjReason OK
- ObjHeader Date: Thu, 16 Nov 2017 16:21:37 GMT
- ObjHeader Server: Apache
- ObjHeader Cache-Control: max-age=0
- ObjHeader Expires: Thu, 16 Nov 2017 16:21:37 GMT
- ObjHeader Vary: Accept-Encoding
- ObjHeader Content-Encoding: gzip
- ObjHeader Content-Length: 3252
- ObjHeader Content-Type: text/html; charset=UTF-8
- ObjHeader X-Varnish-Cookie-Debug: Cleaned request cookies:
- ObjHeader x-url: /
- ObjHeader x-host: www.example.com
- ObjHeader X-Cache: ZIP
- Fetch_Body 3 length stream
- Gzip u F - 3252 10723 80 80 25946
- BackendReuse 23 boot.default
- Timestamp BerespBody: 1510849297.677148 0.072049 0.000190
- Length 3252
- BereqAcct 328 0 328 247 3252 3499
- End
* << Request >> 786982
- Begin req 786981 rxreq
- Timestamp Start: 1510849297.605008 0.000000 0.000000
- Timestamp Req: 1510849297.605008 0.000000 0.000000
- ReqStart 10.42.115.200 35666
- ReqMethod GET
- ReqURL /
- ReqProtocol HTTP/1.1
- ReqHeader Host: www.example.com
- ReqHeader User-Agent: curl/7.26.0
- ReqHeader Accept: */*
- ReqHeader X-Forwarded-For: XXX.XX.XXX.XXX
- ReqHeader X-Forwarded-Host: www.example.com
- ReqHeader X-Forwarded-Port: 443
- ReqHeader X-Forwarded-Proto: https
- ReqHeader X-Forwarded-Server: lb-proxy-1
- ReqHeader X-Real-Ip: XXX.XX.XXX.XXX
- ReqHeader Accept-Encoding: gzip
- ReqUnset X-Forwarded-For: XXX.XX.XXX.XXX
- ReqHeader X-Forwarded-For: XXX.XX.XXX.XXX, 10.42.115.200
- VCL_call RECV
- ReqUnset X-Forwarded-For: XXX.XX.XXX.XXX, 10.42.115.200
- ReqHeader X-Forwarded-For: XXX.XX.XXX.XXX
- ReqHeader Cookie:
- ReqUnset Cookie:
- ReqHeader Cookie:
- ReqUnset Cookie:
- ReqHeader Cookie:
- ReqUnset Cookie:
- ReqHeader Cookie:
- ReqUnset Cookie:
- ReqHeader Cookie:
- ReqUnset Cookie:
- ReqHeader Cookie:
- ReqUnset Cookie:
- ReqHeader Cookie:
- ReqUnset Cookie:
- ReqHeader Cookie:
- ReqUnset Cookie:
- ReqHeader Cookie:
- ReqUnset Cookie:
- ReqHeader Cookie:
- ReqUnset Cookie:
- ReqUnset Accept-Encoding: gzip
- ReqHeader Accept-Encoding: gzip
- ReqHeader X-Device: desktop
- VCL_return hash
- VCL_call HASH
- VCL_return lookup
- Debug "XXXX HIT-FOR-PASS"
- HitPass 786980
- VCL_call PASS
- VCL_return fetch
- Link bereq 786983 pass
- Timestamp Fetch: 1510849297.677182 0.072175 0.072175
- RespProtocol HTTP/1.1
- RespStatus 200
- RespReason OK
- RespHeader Date: Thu, 16 Nov 2017 16:21:37 GMT
- RespHeader Server: Apache
- RespHeader Cache-Control: max-age=0
- RespHeader Expires: Thu, 16 Nov 2017 16:21:37 GMT
- RespHeader Vary: Accept-Encoding
- RespHeader Content-Encoding: gzip
- RespHeader Content-Length: 3252
- RespHeader Content-Type: text/html; charset=UTF-8
- RespHeader X-Varnish-Cookie-Debug: Cleaned request cookies:
- RespHeader x-url: /
- RespHeader x-host: www.example.com
- RespHeader X-Cache: ZIP
- RespHeader X-Varnish: 786982
- RespHeader Age: 0
- RespHeader Via: 1.1 varnish-v4
- VCL_call DELIVER
- RespUnset x-url: /
- RespUnset x-host: www.example.com
- RespUnset X-Cache: ZIP
- RespHeader X-Cache: MISS
- RespHeader X-Hits: 0
- RespHeader X-Host: example.com
- RespUnset Server: Apache
- RespUnset X-Varnish: 786982
- RespUnset Via: 1.1 varnish-v4
- RespHeader X-Cache-TTL-Remaining:
- RespHeader X-Cache-Age:
- RespHeader X-Cache-Grace:
- VCL_return deliver
- Timestamp Process: 1510849297.677222 0.072215 0.000040
- RespHeader Accept-Ranges: bytes
- Debug "RES_MODE 2"
- RespHeader Connection: keep-alive
- Timestamp Resp: 1510849297.677275 0.072268 0.000053
- ReqAcct 290 0 290 446 3252 3698
- End
这是我的 VCL：
vcl 4.0;
import std;
include "cache_invalidation.vcl";
include "custom_subroutines.vcl";
backend default {
.host = "VARNISH_BACKEND_ADDRESS"; # IP or Hostname of backend
.port = "VARNISH_BACKEND_PORT"; # Port Apache or whatever is listening
.connect_timeout = 5s; # Wait a maximum of 5s for backend connection (Apache, Nginx, etc...)
.first_byte_timeout = 10s; # Wait a maximum of 10s for the first byte to come from your backend
.between_bytes_timeout = 2s; # Wait a maximum of 2s between each bytes sent
.max_connections = 300; # Max parallel connections to our backend
}
sub vcl_recv {
# IP forwarding.
if (req.restarts == 0) {
if (req.http.X-Forwarded-For && !req.http.X-Real-IP) {
set req.http.X-Real-IP = regsub(req.http.X-Forwarded-For, ".*\b(?!10|127|172)(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}).*", "");
}else{
set req.http.X-Forwarded-For = req.http.X-Real-IP;
}
}
# Enforce redirect to HTTPS
if (req.http.X-Forwarded-Proto !~ "(?i)https") {
return(synth(720, "https://" + req.http.host + req.url));
}
# Remove the Google Analytics added parameters, useless for our backend
if (req.url ~ "(\?|&)(utm_source|utm_medium|utm_campaign|utm_content|gclid|cx|ie|cof|siteurl)=") {
set req.url = regsuball(req.url, "&(utm_source|utm_medium|utm_campaign|utm_content|gclid|cx|ie|cof|siteurl)=([A-z0-9_\-\.%25]+)", "");
set req.url = regsuball(req.url, "\?(utm_source|utm_medium|utm_campaign|utm_content|gclid|cx|ie|cof|siteurl)=([A-z0-9_\-\.%25]+)", "?");
set req.url = regsub(req.url, "\?&", "?");
set req.url = regsub(req.url, "\?$", "");
}
# Remove anchor or hash references
if(req.url ~ "\#"){
set req.url = regsub(req.url, "\#.*$", "");
}
# Remove incomplete query string
if(req.url ~ "\?$"){
set req.url = regsub(req.url, "\?$", "");
}
# Remove Google Analytics Cookies, our server does not need it
set req.http.Cookie = regsuball(req.http.Cookie, "__utm.=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "_ga=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "_gat=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "utmctr=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "utmcmd.=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "utmccn.=[^;]+(; )?", "");
# Remove DoubleClick offensive cookies
set req.http.Cookie = regsuball(req.http.Cookie, "__gads=[^;]+(; )?", "");
# Remove the Quant Capital cookies (added by some plugin, all __qca)
set req.http.Cookie = regsuball(req.http.Cookie, "__qc.=[^;]+(; )?", "");
# Remove the AddThis cookies
set req.http.Cookie = regsuball(req.http.Cookie, "__atuv.=[^;]+(; )?", "");
# Remove a ";" prefix in the cookie if present
set req.http.Cookie = regsuball(req.http.Cookie, "^;\s*", "");
# Are there cookies left with only spaces or that are empty?
if (req.http.cookie ~ "^\s*$") {
unset req.http.cookie;
}
if (req.url ~ "^[^?]*\.(bmp|bz2|css|doc|eot|flv|gif|gz|ico|jpeg|jpg|js|less|pdf|png|rtf|swf|txt|woff|xml)(\?.*)?$") {
unset req.http.Cookie;
return (hash);
}
if (req.url ~ "^[^?]*\.(mp[34]|rar|tar|tgz|gz|wav|zip|bz2|xz|7z|avi|mov|ogm|mpe?g|mk[av]|webm)(\?.*)?$") {
unset req.http.Cookie;
return (hash);
}
if(req.http.Accept-Encoding)
{
if(req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp[34]|ogg)$")
{
# No point compressing these extensions
unset req.http.Accept-Encoding;
} elsif(req.http.Accept-Encoding ~ "gzip")
{
set req.http.Accept-Encoding = "gzip";
} elsif(req.http.Accept-Encoding ~ "deflate")
{
set req.http.Accept-Encoding = "deflate";
} elsif(req.http.Accept-Encoding ~ "sdch")
{
set req.http.Accept-Encoding = "sdch";
} else
{
# Unknown Algorithm
unset req.http.Accept-Encoding;
}
}
}
# At this point, it is our last chance to make some decision (set|unset|mod) about some header that the backend should see.
sub vcl_backend_fetch {
unset bereq.http.X-Varnish-SESS-ID;
}
sub vcl_hit {
# Custom Headers for debug purposes
set req.http.X-Cache-TTL-Remaining = obj.ttl;
set req.http.X-Cache-Age = obj.keep - obj.ttl;
set req.http.X-Cache-Grace = obj.grace;
}
sub vcl_backend_response {
if (bereq.url ~ "^[^?]*\.(mp[34]|rar|tar|tgz|gz|wav|zip|bz2|xz|7z|avi|mov|ogm|mpe?g|mk[av]|webm)(\?.*)?$") {
unset beresp.http.set-cookie;
set beresp.do_stream = true; # Check memory usage it'll grow in fetch_chunksize blocks (128k by default) if the backend doesn't send a Content-Length header, so only enable it for big objects
set beresp.do_gzip = false; # Don't try to compress it for storage
}
# Do not Gzip image files in Varnish
if (beresp.http.url ~ "\.(jpg|jpeg|png|gif|gz|tgz|bz2|tbz|mp3|mp4|ogg|swf)$") {
set beresp.do_gzip = false;
}
else {
set beresp.do_gzip = true;
set beresp.http.X-Cache = "ZIP";
}
# GZip the cached content if possible
if (beresp.http.content-type ~ "text") {
set beresp.do_gzip = true;
}
# If we are performing a redirection 301 / 302 from the backend and our web server and varnish instances are in the same node, apache mod_rewrite could append it's port
# typically varnish :80 apache/nginx/lighthttp :8080, so the redirect can then often redirect the end-user to a URL on :8080, where it should be :80.
if(beresp.status == 301 || beresp.status == 302) {
set beresp.http.Location = regsub(beresp.http.Location, ":[0-9]+", "");
}
# The example below is explained in detail in load_balancing.vcl as part of one of the strategies that we can take to handle Varnish Grace Mode
set bereq.http.stale-while-revalidate = regsub(beresp.http.Cache-Control, ".*stale-while-revalidate\=([0-9]+).*", "");
if(std.real(bereq.http.stale-while-revalidate, 0.0) > 0)
{
set beresp.grace = std.duration(bereq.http.stale-while-revalidate + "s", 10s);
} else {
set beresp.grace = 2h;
}
}
sub vcl_deliver {
# At this point this is our last change to set or unset the response headers before deliver it
if (obj.hits > 0)
{
set resp.http.X-Cache = "HIT";
} else
{
set resp.http.X-Cache = "MISS";
}
set resp.http.X-Hits = obj.hits;
# Remove www of our host in the backend response
set resp.http.X-Host = regsub(req.http.host, "^www\.", "");
# Remove PHP , Apache and Varnish versions
unset resp.http.x-powered-by;
unset resp.http.Server;
unset resp.http.X-Varnish;
unset resp.http.Via;
# Here we are ensuring to add this headers in the response (defined in the hit subroutine) for debug purposes
set resp.http.X-Cache-TTL-Remaining = req.http.X-Cache-TTL-Remaining;
set resp.http.X-Cache-Age = req.http.X-Cache-Age;
set resp.http.X-Cache-Grace = req.http.X-Cache-Grace;
}
您提供的清漆日志与您正在执行的 curl 请求不匹配（例如错误的用户代理）。
如果它仍然相关，在您的 varnishlog 中您可以看到后端请求包含 headers :
BereqHeader Cache-Control: no-cache
BereqHeader Pragma: no-cache
后端响应包含以下内容
BerespHeader Cache-Control: max-age=0
这会阻止 Varnish 缓存响应。
你能分享 Request（不是 BackendRequest）的清漆日志部分吗？
没有它，我的猜测可能是你总是错过，因为 builtin.vcl 使你的请求成为 pass（尽管如此，你没有在你的 conf 中做任何 return (pass)后端请求来自 pass 状态
Begin bereq 1048612 pass
可以通过builtin.vcl中的这两条规则触发：
if (req.method != "GET" && req.method != "HEAD") {
/* We only deal with GET and HEAD by default */
return (pass);
}
if (req.http.Authorization || req.http.Cookie) {
/* Not cacheable by default */
return (pass);
}
编辑
这很奇怪。你不应该有任何使用 Curl 的 Cookie。 尝试确定您的 cookie 处理代码的范围并使用 req.http.Cookie（带有大写 C），如下所示：
if (req.http.Cookie) {
# Remove Google Analytics Cookies, our server does not need it
set req.http.Cookie = regsuball(req.http.Cookie, "__utm.=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "_ga=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "_gat=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "utmctr=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "utmcmd.=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "utmccn.=[^;]+(; )?", "");
# Remove DoubleClick offensive cookies
set req.http.Cookie = regsuball(req.http.Cookie, "__gads=[^;]+(; )?", "");
# Remove the Quant Capital cookies (added by some plugin, all __qca)
set req.http.Cookie = regsuball(req.http.Cookie, "__qc.=[^;]+(; )?", "");
# Remove the AddThis cookies
set req.http.Cookie = regsuball(req.http.Cookie, "__atuv.=[^;]+(; )?", "");
# Remove a ";" prefix in the cookie if present
set req.http.Cookie = regsuball(req.http.Cookie, "^;\s*", "");
# Are there cookies left with only spaces or that are empty?
if (req.http.Cookie ~ "^\s*$") {
unset req.http.Cookie;
}
}
这里还有一件事，在你的vcl日志中你可以看到Varnish在做Hit-For-Pass，也就是Varnish在缓存你的请求没有被缓存的事实。如果你想真正测试，你需要事先清除内容。
最后一件事：您的后端服务器 (apache) 以 Cache-Control : max-age =0 header 响应，这会阻止缓存，如果您想缓存内容，您应该更改它。