如何在我的分析结果中添加 URL 以及其他被废弃的数据

Question

我想一次抓取多个网站。因此，我更愿意将 URL 与其他被废弃的数据一起写入结果中。但我不知道怎么办。

const puppeteer = require('puppeteer');

(async () => {
const browser = await puppeteer.launch()
const page = await browser.newPage()
await page.goto('https://www.amazon.com/')


 await page.waitForTimeout( 10000 );
const localStorageData = await page.evaluate(() => {
let json = {};
for (let i = 0; i < localStorage.length; i++) {
  const key = localStorage.key(i);
  json[key] = localStorage.getItem(key);
}
return json;
});
const data = {};
for (let entry of Object.entries(data)) {
data[entry.key] = entry.value;
}
console.log(localStorageData)

await browser.close()
})()

Answer 1

您只需将您使用的 URL 添加到您的 JSON:

const puppeteer = require("puppeteer");

(async () => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();

  const url = "https://www.amazon.com/";
  await page.goto(url);

  await page.waitForTimeout(10000);
  const localStorageData = await page.evaluate((url) => {
    let json = {};
    for (let i = 0; i < localStorage.length; i++) {
      const key = localStorage.key(i);
      json[key] = localStorage.getItem(key);
    }
    return json;
  });
  localStorageData["url"] = url;
  const data = {};
  for (let entry of Object.entries(data)) {
    data[entry.key] = entry.value;
  }
  console.log(localStorageData);

  await browser.close();
})();

如何在我的分析结果中添加 URL 以及其他被废弃的数据

How to add URL in the result of my analysis along with other data that get scrapped

javascript

scrape