使用axios时无法修复由cheerio引起的错误

Question

我正在尝试使用 axios 和 cheerio 从 webpage 获取不同帖子的标题，但脚本总是抛出此错误 Error: cheerio.load() expects a string。我该如何解决？

const axios = require('axios');
const cheerio = require('cheerio');

const startLink = 'https://old.reddit.com/r/programming/';

const getPostTitles = (link) =>{
        const resp = axios.get(link);
        const $ = cheerio.load(resp.data);
        const postTitles = [];

        $('div > p.title > a').each((_idx, el) => {
            const postTitle = $(el).text()
            postTitles.push(postTitle)
        });

        return postTitles;
};

const results = getPostTitles(startLink);
console.log(results);

Answer 1

Axios.get() returns 一个承诺，所以你需要 await 它，或者使用 .then() 来获得结果。我在下面使用了 async/await 语法，这将显示 post 标题：

const axios = require('axios');
const cheerio = require('cheerio');

const startLink = 'https://old.reddit.com/r/programming/';

const getPostTitles = async (link) => {
    const resp = await axios.get(link);
    const $ = cheerio.load(resp.data);
    const postTitles = [];

    $('div > p.title > a').each((_idx, el) => {
        const postTitle = $(el).text()
        postTitles.push(postTitle)
    });

    return postTitles;
};

async function testGetPostTitles() {
    const results = await getPostTitles(startLink);
    console.log(results);
}

testGetPostTitles();

使用axios时无法修复由cheerio引起的错误

Can't fix an error caused by cheerio while using axios

javascript

node.js

web-scraping

cheerio

axios