如何使用 BeautifulSoup 抓取下一页？

Question

我想一边看本站的漫画一边刷下一章：
https://www.mangaworld.io/，这不简单link所以我不能用计数器，它就像 1、2、3...9、a、b、c、d。我试过计数器但没有结果。

Answer 1

您可以使用此示例如何对请求进行分页：

import requests
from bs4 import BeautifulSoup


url = "https://www.mangaworld.io"

params = {"page": 1}

for page in range(1, 4):  # <--- increase the page number here
    print("Page {}...".format(page))
    print("-" * 80)
    params["page"] = page
    soup = BeautifulSoup(
        requests.get(url, params=params).content, "html.parser"
    )

    for a in soup.select(".comics-grid a.manga-title"):
        print(a.get_text(strip=True))

打印：

Page 1...
--------------------------------------------------------------------------------
Isekai Kenkokuki
Taming My Boss
Omukae Shibuya-kun
Touch to Unlock
Amagami-san Chi no Enmusubi
Clover
Kirei ni shite moraemasu ka
DanMachi: Sword Oratoria
Kishuku Gakkou no Juliet
Kaiju No. 8
Four Knights of the Apocalypse
Creepy Cat
Little Witch Academia
Blue Box
SHY
Okaeri Alice
Page 2...
--------------------------------------------------------------------------------
Tokyo卍Revengers
Kamonohashi Ron no Kindan Suiri
Stairway of Time
Koi ka Mahou ka Wakaranai!
One Piece
JoJo's Bizarre Adventure Part 8 - JoJolion
Silent War
Sweet Spooky Darling
Martial Peak
Tale of a Scribe Who Retires to the Countryside
Oshi no Ko
Goblin Slayer
Fairy Tail: 100 Years Quest
Boku no Hero Academia
Evil Heroes
Ao no Exorcist
Page 3...
--------------------------------------------------------------------------------
Positively Yours
Fire Brigade of Flames
Whispering You a Love Song
Kiss x Sis
Zense Coupling
Gallery L
The Bloody Merchant Empress and the Cold Husband's Forceful Doting
Isekai Maou to Shoukan Shoujo Dorei Majutsu
Time Lover
Lunar Lover
Kanojo mo Kanojo
Himekishi ga Classmate!
Leveling Up, by Only Eating!
Dr. Stone
Murim Login
Candy Flurry

如何使用 BeautifulSoup 抓取下一页？

How can I scrape the next page using BeautifulSoup?

python

web-scraping

beautifulsoup

screen-scraping