我似乎无法使用 BeautifulSoup 在同一个 class 中找到多个值

Question

我正在尝试学习如何抓取网站。目前我正在抓取：https://finance.yahoo.com/screener/unsaved/f491bcb6-de80-4813-b50e-d6dc8e2f5623?dependentField=sector&dependentValues=Consumer%20Cyclical&offset=0&count=100

我正在尝试找零钱，但它与 stock_price 共享相同的 class。

因此，我尝试使用不同的 class：C($positiveColor) 和 C($negativeColor)。但是，当我使用这些 classes 时，我收到一个错误 AttributeError: 'NoneType' object has no attribute 'text'.

这是因为有变化为 0 且没有明显变化 class。我怎样才能使用 BeatifulSoup 获得 0？

是的，我知道我可以测试 None 然后我可以将它设置为 0，但我希望能够使用 BeautifulSoup.

谢谢:)

import requests

headers = {'User-Agent'      : 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36', 'Accept'          : 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8', 
 'Accept-Language' : 'en-US,en;q=0.5', 'DNT'             : '1', # Do Not Track Request Header 'Connection'      : 'close'
}


from bs4 import BeautifulSoup

URL = 'https://finance.yahoo.com/screener/unsaved/f491bcb6-de80-4813-b50e-d6dc8e2f5623?dependentField=sector&dependentValues=Consumer%20Cyclical&offset=0&count=100'
page = requests.get(URL, headers=headers, timeout=5)

soup = BeautifulSoup(page.content, "html.parser")

results = soup.find(id="screener-results")

stock_ = results.find_all("tr", class_="simpTblRow")

for stock_ in stock_:
  stock_symbol = stock_.find('a', class_='Fw(600) C($linkColor)')
  stock_name = stock_.find('td', class_='Va(m) Ta(start) Px(10px) Fz(s)')  
  stock_price = stock_.find('td', class_='Va(m) Ta(end) Pstart(20px) Fw(600) Fz(s)')
  stock_change = stock_.find('span', class_='C($positiveColor)')
  if stock_change == None:
    stock_change = stock_.find('span', class_='C($negativeColor)')
  
  print(stock_symbol.text.strip() + '\n' + stock_name.text.strip() + '\nCurrent Price: $' + stock_price.text.strip() + '\nChange: ' + stock_change.text.strip(), end="\n"*2)

Answer 1

在大多数情况下，select 元素不是 class 是更好的策略，因为它们通常非常动态，如果可用，请关注更多“静态”属性。

如果要更改，只需使用 <fin-streamer>

的 data-field 属性

stock_change = stock_.find('fin-streamer', {'data-field':'regularMarketChange'})

例子

...
URL = 'https://finance.yahoo.com/screener/unsaved/f491bcb6-de80-4813-b50e-d6dc8e2f5623?dependentField=sector&dependentValues=Consumer%20Cyclical&offset=0&count=100'
page = requests.get(URL, headers=headers, timeout=5)    
soup = BeautifulSoup(page.content, "html.parser")    
results = soup.find(id="screener-results")

stock_ = results.find_all("tr", class_="simpTblRow")

for stock_ in stock_:
    stock_symbol = stock_.find('td', {'aria-label':'Name'})
    stock_name = stock_.find('td', {'aria-label':'Name'})
    stock_price = stock_.find('fin-streamer', {'data-field':'regularMarketPrice'})
    stock_change = stock_.find('fin-streamer', {'data-field':'regularMarketChange'})
    print(stock_symbol.text.strip() + '\n' + stock_name.text.strip() + '\nCurrent Price: $' + stock_price.text.strip() + '\nChange: ' + stock_change.text.strip(), end="\n"*2)

输出

...
Great Wall Motor Company Limited
Great Wall Motor Company Limited
Current Price: .80
Change: -0.51

Mahindra & Mahindra Limited
Mahindra & Mahindra Limited
Current Price: .00
Change: 0.00

Sime Darby Berhad
Sime Darby Berhad
Current Price: [=12=].5690
Change: +0.0050
...

我似乎无法使用 BeautifulSoup 在同一个 class 中找到多个值

I can't seem to find multiple values within the same class using BeatufulSoup

python

lxml

beautifulsoup

web-scraping

例子

输出