响应缓冲区数据在 Node JS 中被截断

Question

我正在尝试弄清楚如何在 Node JS 中使用 http 和 https 内置模块的 request 函数（我知道第三方包装器，但我正在尝试自己弄清楚）。我正在运行解决响应中的缓冲区数据最后被部分截断的问题。使用 cURL.

进行测试时不会出现此问题

这是我一直在使用的代码：

const { request: httpRequest } = require("http");
const { request: httpsRequest } = require("https");
const parseURLData = (url, init) => {
  const { hostname, protocol, port: somePort, pathname, search } = new URL(url);
  const port = +somePort || (protocol === "https:" ? 443 : 80);
  const options = { hostname, port, path: pathname + search, ...init };
  return [protocol, options];
};
const makeRequest = (url, init = { method: "GET" }) => {
  const [protocol, options] = argumentsToOptions(url, init);
  const request = protocol === "http:" ? httpRequest : httpsRequest;
  return new Promise((resolve, reject) =>
    request(options, (res) => {
      res.on("error", reject);
      resolve(res);
    })
      .on("error", reject)
      .end()
  );
};
// not using `async/await` while testing
makeRequest("https://jsonplaceholder.typicode.com/users/1/")
  .then((res) => 
    new Promise((resolve) =>
      res.on("data", (buffer) => {
        resolve(buffer.toString("utf8")); // part of data is cut off
        // resolve(JSON.parse(buffer.toString()));
      })
    )
  )
  .then(console.log)
  .catch(console.error);

这是预期的输出（来自 cURL）：

{
  "id": 1,
  "name": "Leanne Graham",
  "username": "Bret",
  "email": "Sincere@april.biz",
  "address": {
    "street": "Kulas Light",
    "suite": "Apt. 556",
    "city": "Gwenborough",
    "zipcode": "92998-3874",
    "geo": {
      "lat": "-37.3159",
      "lng": "81.1496"
    }
  },
  "phone": "1-770-736-8031 x56442",
  "website": "hildegard.org",
  "company": {
    "name": "Romaguera-Crona",
    "catchPhrase": "Multi-layered client-server neural-net",
    "bs": "harness real-time e-markets"
  }
}

这是实际输出，由于某种原因每次我运行代码时都略有不同：

{
  "id": 1,
  "name": "Leanne Graham",
  "username": "Bret",
  "email": "Sincere@april.biz",
  "address": {
    "street": "Kulas Light",
    "suite": "Apt. 556",
    "city": "Gwenborough",
    "zipcode": "92998-3874",
    "geo": {
      "lat": "-37.3159",
      "lng": "81.1496"
    }
  },
  "p

这个问题的正确解决方法是什么？如何从请求中获取所有数据？

Answer 1

缓冲区只应在 end 事件触发时处理，否则您可能正在处理不完整的缓冲区。

makeRequest("https://jsonplaceholder.typicode.com/users/1/")
  .then((res) => 
    new Promise((resolve) => {
        let totalBuffer = "";

        res.on("data", (buffer) => {
            totalBuffer += buffer.toString("utf8");    
        });

        res.on("end", () => resolve(totalBuffer));
    })
  )
  .then(console.log)
  .catch(console.error);

当文件超过 1mb 时，响应几乎总是被截断成几段，因此有必要使用 end 事件来指示流已处理所有可用数据。

https://nodejs.org/api/http.html 查找“JSON 获取示例”

响应缓冲区数据在 Node JS 中被截断

Response Buffer data is truncated in Node JS

javascript

node.js

http

https

buffer