Gzip header 强制文件下载

Question

我正在尝试对所有回复进行 gzip 压缩。在main.go

mux := mux.NewRouter()
mux.Use(middlewareHeaders)
mux.Use(gzipHandler)

然后我有中间件：

func gzipHandler(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        gz := gzip.NewWriter(w)
        defer gz.Close()
        gzr := gzipResponseWriter{Writer: gz, ResponseWriter: w}
        next.ServeHTTP(gzr, r)
    })
}

func middlewareHeaders(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        w.Header().Set("Cache-Control", "max-age=2592000") // 30 days
        w.Header().Set("Content-Encoding", "gzip")
        w.Header().Set("Strict-Transport-Security", "max-age=63072000; includeSubDomains; preload")
        w.Header().Set("Access-Control-Allow-Headers", "Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token")
        w.Header().Set("Access-Control-Allow-Methods", "POST")
        w.Header().Set("Access-Control-Allow-Origin", "origin")
        w.Header().Set("Access-Control-Allow-Credentials", "true")
        w.Header().Set("Access-Control-Expose-Headers", "AMP-Access-Control-Allow-Source-Origin")
        w.Header().Set("AMP-Access-Control-Allow-Source-Origin", os.Getenv("DOMAIN"))
        next.ServeHTTP(w, r)
    })
}

当我卷曲网站时，我得到

curl -v https://example.com
*   Trying 44.234.222.27:443...
* TCP_NODELAY set
* Connected to example.com (XX.XXX.XXX.XX) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=example.com
*  start date: Mar 16 00:00:00 2021 GMT
*  expire date: Apr 16 23:59:59 2022 GMT
*  subjectAltName: host "example.com" matched cert's "example.com"
*  issuer: C=GB; ST=Greater Manchester; L=Salford; O=Sectigo Limited; CN=Sectigo RSA Domain Validation Secure Server CA
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x55cadcebfe10)
> GET / HTTP/2
> Host: example.com
> user-agent: curl/7.68.0
> accept: */*
> 
* Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
< HTTP/2 200 
< date: Mon, 07 Jun 2021 20:13:19 GMT
< access-control-allow-credentials: true
< access-control-allow-headers: Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token
< access-control-allow-methods: POST
< access-control-allow-origin: origin
< access-control-expose-headers: AMP-Access-Control-Allow-Source-Origin
< amp-access-control-allow-source-origin: example.com
< cache-control: max-age=2592000
< content-encoding: gzip
< strict-transport-security: max-age=63072000; includeSubDomains; preload
< vary: Accept-Encoding
< 
Warning: Binary output can mess up your terminal. Use "--output -" to tell 
Warning: curl to output it to your terminal anyway, or consider "--output 
Warning: <FILE>" to save to a file.
* Failed writing body (0 != 3506)
* stopped the pause stream!
* Connection #0 to host example.com left intact

启用 gzip 处理程序和 gzip header 浏览器想要下载文件。

谁能发现我的错误？

Answer 1

1。您应该只在客户要求时 gzip。

Accept-Encoding: gzip 从未被请求，但你 gzip 无论如何都会响应。

所以 curl 按原样还给你。

2。鉴于您的浏览器的行为，这听起来像是双重压缩。也许你有一些 HTTP 反向代理，它已经处理了对浏览器的压缩，但不压缩后端流量。因此，您可能根本不需要在后端进行任何 gzip 压缩 - 尝试 curl --compressed 来确认这一点。

3。您应该从响应中过滤掉 Content-Length。 Content-Length 是压缩后的 HTTP 响应的最终大小，因此该值在压缩过程中会发生变化。

4。您不应该盲目地对所有 URI 应用压缩。一些处理程序已经执行 gzipping（例如 prometheus /metrics），而有些处理程序没有意义进行压缩（例如 .png、.zip、.gz）。至少在将请求传递到处理程序链之前从请求中剥离 Accept-Encoding: gzip，以避免双 gzipping。

5。 Go 中的透明 gzipping 之前已经实现了。快速搜索显示 this gist（针对上面的第 4 点进行了调整）：

package main

import (
    "compress/gzip"
    "io"
    "io/ioutil"
    "net/http"
    "strings"
    "sync"
)

var gzPool = sync.Pool{
    New: func() interface{} {
        w := gzip.NewWriter(ioutil.Discard)
        return w
    },
}

type gzipResponseWriter struct {
    io.Writer
    http.ResponseWriter
}

func (w *gzipResponseWriter) WriteHeader(status int) {
    w.Header().Del("Content-Length")
    w.ResponseWriter.WriteHeader(status)
}

func (w *gzipResponseWriter) Write(b []byte) (int, error) {
    return w.Writer.Write(b)
}

func Gzip(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        if !strings.Contains(r.Header.Get("Accept-Encoding"), "gzip") {
            next.ServeHTTP(w, r)
            return
        }

        w.Header().Set("Content-Encoding", "gzip")

        gz := gzPool.Get().(*gzip.Writer)
        defer gzPool.Put(gz)

        gz.Reset(w)
        defer gz.Close()

        r.Header.Del("Accept-Encoding")
        next.ServeHTTP(&gzipResponseWriter{ResponseWriter: w, Writer: gz}, r)
    })
}

注意 - 以上不支持分块编码和尾部。所以还有改进的机会。

Gzip header 强制文件下载

Gzip header forces file download

go

http-headers

gzip

mux