Node js Puppeteer 转到页面数组

Question

我尝试从我的数组中逐页浏览，但得到这个：

(node:4196) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 request listeners added. Use emitter.setMaxListeners() to increase limit (node:4196) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 framedetached listeners adde d. Use emitter.setMaxListeners() to increase limit (node:4196) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 lifecycleevent listeners add ed. Use emitter.setMaxListeners() to increase limit (node:4196) UnhandledPromiseRejectionWarning: Error: Protocol error (Page.navigate): Target closed. at Promise (D:\Kutz\irrParse\node_modules\puppeteer\lib\Connection.js:198:56) at new Promise () at CDPSession.send (D:\Kutz\irrParse\node_modules\puppeteer\lib\Connection.js:197:12) at navigate (D:\Kutz\irrParse\node_modules\puppeteer\lib\Page.js:520:39) at Page.goto (D:\Kutz\irrParse\node_modules\puppeteer\lib\Page.js:500:7) at uniqueLinks.forEach (D:\Kutz\irrParse\scrape.js:26:16) at Array.forEach () at D:\Kutz\irrParse\scrape.js:25:15 at at process._tickCallback (internal/process/next_tick.js:118:7) (node:4196) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (r ejection id: 1) (node:4196) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise reject ions that are not handled will terminate the Node.js process with a non-zero exit code. (node:4196) UnhandledPromiseRejectionWarning: Error: Navigation Timeout Exceeded: 30000ms exceeded at Promise.then (D:\Kutz\irrParse\node_modules\puppeteer\lib\NavigatorWatcher.js:71:21) at

const puppeteer = require("puppeteer");
var forEach = require('async-foreach').forEach;


const url = "https://reddit.com/r/programming";
const linkSelector = ".content a.title";

(async () => {
  // Launch chrome process
  const browser = await puppeteer.launch({headless: true});
  const page = await browser.newPage();

  await page.goto(url, { waitUntil: "load" });

  // This runs the `document.querySelectorAll` within the page and passes
  // the result to function
  const links = await page.$$eval(linkSelector, links => {
    return links.map((link) => link.href);
  });

  // Make sure we get the unique set of links only
  const uniqueLinks = [...links];
  //console.log(uniqueLinks[0]);

  uniqueLinks.forEach(async (link) => {
    await page.goto(link, { waitUntil: "load" });
  });

  // Kill the browser process
  await browser.close();
})();

forEach() 中抛出错误

Answer 1

不幸的是，Array.prototype.forEach 的迭代器函数并没有像您将其定义为异步时所期望的那样以异步方式执行。使用 for 循环应该适用于您要执行的操作。

for (let i = 0; i < uniqueLinks.length; i ++) {
  await page.goto(uniqueLinks[i], { waitUntil: "load" });
}

Node js Puppeteer 转到页面数组

Node js Puppeteer goto array of pages

javascript

node.js

puppeteer