BeautifulSoup findAll 不返回结果

Question

我想获取此页面上的产品名称和价格。我几乎重复了我为价格的产品名称所做的完全相同的事情，但我没有得到任何东西。

from urllib.request import Request, urlopen
from bs4 import BeautifulSoup as bSoup

header = {'User-Agent':'Mozilla/5.0 (Windows NT 10.0; WOW64; rv:77.0) Gecko/20100101 Firefox/77.0'}
url = "https://www.walmart.ca/search?q=lettuce"
req = Request (url = url, headers = header)

client = urlopen (req)
pageHtml = client.read()
client.close()

pageSoup = bSoup(pageHtml, 'html.parser')

products = pageSoup.findAll ("div", {"class":"css-155zfob e175iya63"})
print (len(products)) #prints 15, like expected
for product  in products:
    pass

prices = pageSoup.findAll ("div", {"class":"css-8frhg8 e175iya65"})
print (len (prices))#prints 0 and idk why :/
for price in prices:
    pass

Answer 1

页面 https://www.walmart.ca/search?q=lettuce 没有 return 您期望的内容：

curl -s -H 'User-Agent':'Mozilla/5.0 (Windows NT 10.0; WOW64; rv:77.0) Gecko/20100101 Firefox/77.0' 'https://www.walmart.ca/search?q=lettuce' | grep 'css-8frhg8'

您可能在浏览器中看到 class，其中内容通过 JavaScript 在运行时部分呈现。这意味着您需要使用可以模拟具有 JavaScript 支持的浏览器的库。

BeautifulSoup findAll 不返回结果

BeautfulSoup findAll not returning results

python

urllib

beautifulsoup

python-3.x