使用 Beautifulsoup 和选择器检索内容

Question

正在尝试检索 html 中嵌入的内容（文本）。未获取内容。

正在尝试使用格式中的选择器来查找 price_box:
price_box = soup2.find('div', attrs={'title class': 'Fw(600)'})

# Import libraries
import requests
import urllib.request
import time
from bs4 import BeautifulSoup

# Set the URL you want to webscrape from
url = 'https://finance.yahoo.com/quote/NVDA?p=NVDA'

# Connect to the URL
response = requests.get(url)

# Parse HTML and save to BeautifulSoup object¶
soup = BeautifulSoup(response.text, "html.parser")

beta = soup.find('h1')
#print (beta)

link = beta.contents

variable = 'NVDA - NVIDIA Corporation'
test = 'NVDA - NVIDIA Corporation'

#<..>

url2 = 'https://finance.yahoo.com/calendar/earnings?from=2019-09-01&to=2019-09-07&day=2019-09-01'

response2 = requests.get(url2)
soup2 = BeautifulSoup(response2.text, "html.parser")
# alpha = soup2.find('')

# div = soup.find('a', {class_ ='D(ib) '})
# text = div.string

price_box = soup2.find('div', attrs={'title class': 'Fw(600)'})
#price = price_box.text
print("Price Box: "+ str(price_box)) # THIS IS WHAT I WANT

希望看到 "Senea"。而不是看到 "None" - “价格框： None

Answer 1

很多内容都是动态的。您可以轻松地用正则表达式输出该信息

import requests, re

p = re.compile(r'"YFINANCE:(.*?)"')
r = requests.get('https://finance.yahoo.com/calendar/earnings?from=2019-09-01&to=2019-09-07&day=2019-09-01&guccounter=1')
print(p.findall(r.text)[0])

另一种方法是完全避免动态外观类

import requests
from bs4 import BeautifulSoup as bs

r = requests.get('https://finance.yahoo.com/calendar/earnings?from=2019-09-01&to=2019-09-07&day=2019-09-01&guccounter=1')
soup = bs(r.content, 'lxml')
print(soup.select_one('#cal-res-table a').text)

正在阅读：

css selectors

使用 Beautifulsoup 和选择器检索内容

Retrieving content using Beautifulsoup and selectors

python

urllib

beautifulsoup

web-scraping

python-requests