使用 puppeteer 和 MutationObserver 检测 DOM 变化

Question

我想检测某些加载页面上的 DOM 变化（例如，本地新闻页面上添加的新文章）并在检测后做一些事情（发送电子邮件）。在此示例中，我试图检测子节点是否已添加或从父节点（目标 div 节点）中删除，并在检测后在控制台中输出一些内容。

需要实现暴露功能吗？如果是，我该怎么做？

我正在使用 pupetter 和 MutationObserver，但它不起作用。

这是我的代码：

const puppeteer = require('puppeteer');

(async () => {
  const browser = await puppeteer.launch({headless: false});
  const page = await browser.newPage();
  await page.goto('https://www.example.com/');

  await page.evaluate(() => {
    const target = document.querySelector("body > div.container.upscore-pos-1 > section > div:nth-child(3) > div:nth-child(2) > div.row.hidden-xs > div > section");
    const observer = new MutationObserver( mutations => {
      for (const mutation of mutations) {
          if (mutation.type === 'childList') {
              console.log('Mutation Detected: A child node has been added or removed.');
            }
      }
    });
    observer.observe(target, { childList: true });
  });

})();

提前致谢！

Answer 1

也许page.exposeFunction()是最简单的方法：

'use strict';

const puppeteer = require('puppeteer');

(async () => {
  const browser = await puppeteer.launch({ headless: false });
  const page = await browser.newPage();
  await page.goto('https://www.example.com/');

  await page.exposeFunction('puppeteerLogMutation', () => {
    console.log('Mutation Detected: A child node has been added or removed.');
  });

  await page.evaluate(() => {
    const target = document.querySelector('body');
    const observer = new MutationObserver( mutations => {
      for (const mutation of mutations) {
        if (mutation.type === 'childList') {
          puppeteerLogMutation();
        }
      }
    });
    observer.observe(target, { childList: true });
  });

  await page.evaluate(() => {
    document.querySelector('body').appendChild(document.createElement('br'));
  });

})();

但 page.waitForSelector()（或其他一些 waitFor... 方法）在某些情况下也足够了：

'use strict';

const puppeteer = require('puppeteer');

(async () => {
  const browser = await puppeteer.launch({ headless: false });
  const page = await browser.newPage();
  await page.goto('https://www.example.com/');

  await page.evaluate(() => {
    setTimeout(() => {
      document.querySelector('body')
        .appendChild(document.createElement('br')).className = 'puppeteer-test';
    }, 5000);
  });

  await page.waitForSelector('body br.puppeteer-test');

  console.log('Mutation Detected: A child node has been added or removed.');
})();

Answer 2

你可以等待那些 console.logs 并抓住它们：

let msg = await page.evaluate(() => new Promise((resolve, reject) => {
  console.log = (msg) => resolve(msg)
}))

或者您可以添加一个 page.on('console') 侦听器

使用 puppeteer 和 MutationObserver 检测 DOM 变化

Detecting DOM changes with puppeteer and MutationObserver

dom

node.js

mutation-observers

puppeteer