FiddlerCore 解码 sdch 响应

Question

我从一个我想用 FiddlerCore 解析的站点收到了一个奇怪的响应。在 chrome 开发人员工具中，如果我检查响应，它看起来完全正常，但在 fiddler 中则不然。代码片段如下（以前工作正常）

String html = oSession.GetResponseBodyAsString();

Returns 以下不是 html，请注意这是一个示例而不是完整的巨大字符串。

JRHwJNeR[=11=]���[=11=][=11=]\u0001��D[=11=]�2�\b[=11=]�\u0016�7]<!DOCTYPE html>\n win\">

也有这样的“\n”和html乱七八糟的

\n\n\n\n\n  \n    <meta name=\"treeID\" content=\"dwedxE+pgRQAWIHiFSsAAA==\">\n

回复headers如下：

Cache-Control:no-cache, no-store
Connection:keep-alive
Content-Encoding:sdch, gzip
Content-Language:en-US
Content-Type:text/html;charset=UTF-8
Date:Fri, 28 Oct 2016 10:17:02 GMT
Expires:Thu, 01 Jan 1970 00:00:00 GMT
Pragma:no-cache
Server:Apache-Coyote/1.1
Set-Cookie:lidc="b=VB87:g=518:u=60:i=1477649823:t=1477731496:s=AQG-LTdly5mcIjAtiRHIOrKE1TiRWW-l"; Expires=Sat, 29 Oct 2016 08:58:16 GMT; domain=.thedomain.com; Path=/
Set-Cookie:_lipt=deleteMe; Expires=Thu, 01-Jan-1970 00:00:10 GMT; Path=/
Strict-Transport-Security:max-age=0
Transfer-Encoding:chunked
Vary:Accept-Encoding, Avail-Dictionary
X-Content-Type-Options:nosniff
X-Frame-Options:sameorigin
X-FS-UUID:882b3366afaa811400a04937a92b0000
X-Li-Fabric:prod-lva1
X-Li-Pop:prod-tln1-scalable
X-LI-UUID:iCszZq+qgRQAoEk3qSsAAA==
X-XSS-Protection:1; mode=block

Fiddler 启动代码：

  Fiddler.FiddlerApplication.AfterSessionComplete += FiddlerApplication_OnAfterSessionComplete;
   Fiddler.FiddlerApplication.BeforeResponse += delegate(Fiddler.Session oS) {
         oS.utilDecodeResponse(); 
   };

    Fiddler.FiddlerApplication.Startup(0, FiddlerCoreStartupFlags.Default);


  }

最初我以为它是 chunked/gzipped 所以我添加了 utilDecodeResponse();到没有效果的 onBeforeResponse！

为了涵盖所有基础，我还尝试在 UTF-8、Unicode、Bigendian 等中手动解码 responseBodyBytes，只是在响应内容类型不正确并禁用 javascript 并加载页面的情况下证明这不是一些时髦的模板，这也没有什么区别。

有什么想法吗？

更新：

根据下面Developer&NineBerry提供的资料，解决方法如下：

为了防止响应被 SDCH 编码，您可以像这样添加一个处理程序：

    Fiddler.FiddlerApplication.BeforeRequest += delegate (Fiddler.Session oS)
    {
        oS.oRequest["Accept-Encoding"] = "gzip, deflate, br";
    };

应该注意的是，这并不适合所有情况，因为您手动设置 headers 而不是检查 SDCH 是否存在然后将其删除，就我的目的而言，这工作正常，但是要使用 fiddler 的一般代理功能，您需要更多逻辑。

Answer 1

内容编码显示为 SDCH - 共享字典压缩；因此在这种情况下，手动解码 UTF-8、Unicode、Bigendian 等中的 responseBodyBytes 将不起作用。

您可以在此处找到有关 SDCH 的更多详细信息 -SDCH Ref 1 & SDCH Ref 2

以上网站摘录：

Shared Dictionary Compression is a content-encoding method which was proposed back in 2008 by Google, and is implemented in Chrome and supported by a number of Google servers. The full proposal can be obtained here -https://lists.w3.org/Archives/Public/ietf-http-wg/2008JulSep/att-0441/Shared_Dictionary_Compression_over_HTTP.pdf. Rather than replicate the contents of the document in this blog post, I’ll try and summarise as concisely as possible:
The whole idea of the protocol is to reduce redundancy across HTTP connections. The amount of ‘common data’ across HTTP responses is obviously significant - for example you will often see a website use common header/footers across a number of HTML pages. If the client were to store this common data locally in a ‘dictionary’, the server would only need to instruct the client how to reconstruct the page using that dictionary.

FiddlerCore 解码 sdch 响应

FiddlerCore decoding an sdch response

c#

fiddlercore