如何使用 Puppeteer 获取请求的原始下载大小？

Question

即所有资源（包括 video/media）下载的数据总量，类似于 Chrome DevTools 的网络选项卡返回的数据。

Answer 1

截至 2018 年 1 月，似乎没有任何方法可以处理所有资源类型（侦听 response 事件 fails for videos），并且可以正确计算压缩资源.

最好的解决方法似乎是监听 Network.dataReceived 事件，然后手动处理该事件：

const resources = {};
page._client.on('Network.dataReceived', (event) => {
  const request = page._networkManager._requestIdToRequest.get(
    event.requestId
  );
  if (request && request.url().startsWith('data:')) {
    return;
  }
  const url = request.url();
  // encodedDataLength is supposed to be the amount of data received
  // over the wire, but it's often 0, so just use dataLength for consistency.
  // https://chromedevtools.github.io/devtools-protocol/tot/Network/#event-dataReceived
  // const length = event.encodedDataLength > 0 ?
  //     event.encodedDataLength : event.dataLength;
  const length = event.dataLength;
  if (url in resources) {
    resources[url] += length;
  } else {
    resources[url] = length;
  }
});

// page.goto(...), etc.

// totalCompressedBytes is unavailable; see comment above
const totalUncompressedBytes = Object.values(resources).reduce((a, n) => a + n, 0);

Answer 2

const imgaes_width = await page.$$eval('img', anchors => [].map.call(anchors, img => img.width));
const imgaes_height = await page.$$eval('img', anchors => [].map.call(anchors, img => img.height));

Answer 3

如果您使用的是 puppeteer，那么您有服务器端节点...为什么不通过流或流来传输请求，然后计算内容大小？

还有https://github.com/watson/request-stats

您可能还想调用 page.waitForNavigation，因为您可能正在处理异步计时问题

Answer 4

的解决方案即使在2021年也能完美运行。只需更换：

page._networkManager -> 页面。_frameManager._networkManager

适合我的完整示例：

const resources = {};
page._client.on('Network.dataReceived', (event) => {
  const request = page._frameManager._networkManager._requestIdToRequest.get(
    event.requestId
  );
  if (request && request.url().startsWith('data:')) {
    return;
  }
  const url = request.url();
  const length = event.dataLength;
  if (url in resources) {
    resources[url] += length;
  } else {
    resources[url] = length;
  }
});

await page.goto('

const totalUncompressedBytes = Object.values(resources).reduce((a, n) => a + n, 0);
console.log(totalUncompressedBytes);

如何使用 Puppeteer 获取请求的原始下载大小？

How can I get the raw download size of a request using Puppeteer?

javascript

request

puppeteer