是什么导致我的 Splinter 'click()' 调用期间出现 Python NoneType 错误？

Question

当尝试从多个 Politico 州网页（例如 this one 抓取县数据时，我得出的结论是最好的方法是先单击展开县列表的按钮，然后再抓取 table body的数据（存在时）。然而，我点击按钮的尝试失败了：

from bs4 import BeautifulSoup as bs
import requests
from splinter import Browser

state_page_url = "https://www.politico.com/2020-election/results/washington/"
executable_path = {'executable_path': 'chrome-driver/chromedriver.exe'}
browser = Browser('chrome', **executable_path, headless=False)
browser.visit(state_page_url)
    
state_soup = bs(browser.html, 'html.parser')
reveal_button = state_soup.find('button', class_='jsx-3713440361')

if (reveal_button == None):
    # Steps to take when the button isn't present
    # ...
else:
    reveal_button.click()

在 else-condition 之后返回的错误是针对我的 click() 调用的："TypeError: NoneType object is not callable"。这对我来说没有意义，因为我认为 if-statement 暗示 reveal_button 不是 NoneType。我是在误解错误消息，reveal_button 是如何设置的，还是在制作 state_soup 后误解了我正在处理的内容？

Answer 1

根据问题的评论线程和 this solution 类似问题，我遇到了以下修复：

from bs4 import BeautifulSoup as bs
import requests
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By

# Navigate the page to click the desired button
state_page_url = "https://www.politico.com/2020-election/results/alabama/"
driver = webdriver.Chrome(executable_path='chrome-driver/chromedriver.exe')
driver.get(state_page_url)
button_list = driver.find_elements(By.CLASS_NAME, 'jsx-3713440361')

if button_list == []:
    # Actions to take when no button is found
    # ...
else:
    button_list[-1].click() # The index was determined through trial/error specific to the web page

    # Now to grab the table and its data
    state_soup = bs(driver.page_source)
    state_county_results_table = state_soup.find('tbody', class_='jsx-3713440361')

请注意，它需要 selenium 来进行导航和交互，而 BeautifulSoup4 用于解析它以获得我需要的信息

是什么导致我的 Splinter 'click()' 调用期间出现 Python NoneType 错误？

What Caused the Python NoneType Error During My Splinter 'click()' Call?

web-scraping

beautifulsoup

click

splinter