BeautifulSoup <small> 标签

Question

我想抓取 <> 元素之间的文本。 HTML 我想在其中抓取“文本”的小代码（当时是：27.00）。 HTML 是 <> class="product-views-price-old" 是：£27.00 <> 我的代码是：

from bs4 import BeautifulSoup
import requests
url = "https://www.petshop.co.uk/Dog"
r = requests.get(url)
soup = BeautifulSoup(r.content)
for old_price in soup.find_all("small", class_ = "product-views-price-old"):
    print(old_price)

上面的代码什么也没给我。甚至没有错误。如何抓取 <> 标签之间的文本？

Answer 1

内容是动态提供的，因此您不会通过 requests 以这种方式获取它 - 看看这个 selenium 代码。

要删除文本和空格，您可以执行以下操作：

.get_text(strip=True).replace('Was: ','')

例子

from selenium import webdriver
from bs4 import BeautifulSoup
import time

url = "https://www.petshop.co.uk/Dog"
driver = webdriver.Chrome('C:\Program Files\ChromeDriver\chromedriver.exe')
driver.get(url)
time.sleep(3)

html = driver.page_source
soup = BeautifulSoup(html,'html.parser')
for old_price in soup.find_all("small", class_ = "product-views-price-old"):
    print(old_price.get_text(strip=True).replace('Was: ',''))

driver.quit()

输出

£2.20
£18.61
£27.00
£38.39
£38.39
£20.65
£1.30
£67.99
£20.65
£1.30
£54.95
£30.99

Answer 2

您不需要 selenium 和 beautifulsoup。如果你去 Network tab，就会有一个 API。获得响应后，您需要确定密钥以获取值。

https://www.petshop.co.uk/api/items?c=3934951&commercecategoryurl=%2FDog&country=GB&currency=GBP&fieldset=search&include=facets&language=en&limit=100&n=2&offset=0&pricelevel=5&sort=custitem_bb1_qtysold%3Adesc

import requests
url = "https://www.petshop.co.uk/api/items?c=3934951&commercecategoryurl=%2FDog&country=GB&currency=GBP&fieldset=search&include=facets&language=en&limit=100&n=2&offset=0&pricelevel=5&sort=custitem_bb1_qtysold%3Adesc"
r = requests.get(url).json()
for item in r['items']:
       print(item['pricelevel2_formatted'])

输出：

£2.20
£18.61
£27.00
£5.92
£38.39
£38.39
£20.65
£1.30
£67.99
£20.65
£1.30
£54.95
£30.99
£57.95
£22.00
£46.55
£9.60
£1.99
£32.99
£30.99
£54.95
£8.21
£38.39
£57.95
£32.99
£2.65
£20.65
£10.50
£18.48
£10.50
£3.75
£2.99
£33.99
£25.00
£23.99
£1.39
£54.95
£36.99
£27.00
£49.50
£38.39
£39.59
£67.99
£32.99
£40.70
£29.69
£39.94
£31.49
£59.99
£38.39
£25.99
£67.99
£38.39
£25.99
£49.50
£39.59
£1.30
£12.90
£1.00
£44.99
£22.99
£69.99
£15.50
£2.99
£20.99
£32.99
£38.39
£15.99
£42.99
£27.12
£46.55
£52.49
£2.99
£1.99
£51.59
£2.99
£25.99
£2.99
£49.50
£18.84
£40.74
£44.99
£20.99
£39.56
£2.99
£7.09
£26.99
£18.61
£19.99
£43.99
£16.50
£12.00
£36.29
£40.40
£2.99
£35.99
£59.99
£5.50
£8.99
£57.95

BeautifulSoup <small> 标签

BeautifulSoup <small> tag

beautifulsoup

request

web-scraping

python-3.x

python-requests