Beautiful Soup 与亚马逊有问题，它

Question

我正在尝试从亚马逊页面获取名称和奖品，这是代码：

import requests
from bs4 import BeautifulSoup

URL = "https://www.amazon.it/Nuovo-Apple-iPhone-SE-64GB/dp/B087616RMM/ref=sr_1_1_sspa?__mk_it_IT=%C3%85M%C3%85%C5%BD%C3%95%C3%91&dchild=1&keywords=iphone&qid=1597409499&s=electronics&sr=1-1-spons&psc=1&spLa=ZW5jcnlwdGVkUXVhbGlmaWVyPUExNElPQVhUUTFTOTFOJmVuY3J5cHRlZElkPUEwMTA3ODI0M1RZVTc2MTdRM1A3SiZlbmNyeXB0ZWRBZElkPUEwODI4MDExMTdYMlhUOFlUTVY0TCZ3aWRnZXROYW1lPXNwX2F0ZiZhY3Rpb249Y2xpY2tSZWRpcmVjdCZkb05vdExvZ0NsaWNrPXRydWU="

URL2 = "https://www.amazon.it/intermittenze-della-morte-Jos%C3%A9-Saramago-ebook/dp/B019KBH3CC/ref=sr_1_1?__mk_it_IT=%C3%85M%C3%85%C5%BD%C3%95%C3%91&dchild=1&keywords=saramago&qid=1597410061&sr=8-1"

headers = {"User-Agent": 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:79.0) Gecko/20100101 Firefox/79.0'}



page = requests.get(URL2, headers=headers)
soup = BeautifulSoup(page.content, 'lxml')

title = soup.find(id='productTitle').get_text().strip()
price = soup.find(id='priceblock_ourprice').get_text().strip()

print(title)
print(price)

问题是 URL 有效，但 URL2 无效。我该如何解决？谢谢:)

Answer 1

在获取文本之前，您必须检查是否找到所需的元素，如果找到，您可以提取文本：

title = soup.find(id='productTitle')
if title:
    title = title.get_text().strip()

price = soup.find(id='priceblock_ourprice')
if price:
    price = price.get_text().strip()

请注意亚马逊有一些不同的页面布局，所以如果你想制作通用的爬虫，你将不得不涵盖所有这些

Beautiful Soup 与亚马逊有问题，它

Beautiful Soup has problems with amazon,it

python

amazon

beautifulsoup

web-scraping