Python & Selenium：如何避免来自 dividend.com 的 StaleElementReferenceException 错误

Question

我想使用 Python 和 Selenium 从 https://www.dividend.com/dividend-champions/ 获取公司名称。

但是有些公司名打印出来后出现如下错误

StaleElementReferenceException: Message: stale element reference: element is not attached to the page document

如何避免这个错误？

奇怪的是，在出现这个错误之前打印了一些公司名称。

如果你能给我一些提示，我将不胜感激。

Python

driver.get('https://www.dividend.com/dividend-champions/')
companies = driver.find_elements(by=By.CLASS_NAME, value="m-table-long-name")
print(len(companies)) #30
for company in companies:
  print(company.text)

Answer 1

StaleElementReferenceException表示选择的元素可能不再在htmldom.However中，这个异常背后有很多原因，比如元素选择不正确，加载时间等等。所以你需要使用 webdriverwait.

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager

from selenium.webdriver.chrome.options import Options

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC


option = webdriver.ChromeOptions()
option.add_argument("start-maximized")

#chrome to stay open
option.add_experimental_option("detach", True)

driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()),options=option)
driver.get('https://www.dividend.com/dividend-champions/')


names = WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, '//*[@class="mp-table-body"]/div[@class="mp-table-body-row-container mp-table-row t-static"]')))

for name in names:
    company = name.find_element(By.XPATH,'.//a[@class="m-table-body-link-text m-table-stocks-name-col m-table-long-name"]').text
    print(company)

输出：

Eversource Energy
Andersons Inc.
J.M. Smucker Co.
Cambridge Bancorp
Factset Research Systems Inc.
Graco Inc.
First Of Long Island Corp.
Stepan Co.
MDU Resources Group Inc
W. P. Carey Inc
Church & Dwight Co., Inc.
Lincoln Electric Holdings, Inc.
Matthews International Corp. - Ordinary Shares - Class A
New Jersey Resources Corporation
Bank OZK
Caterpillar Inc.
Brown & Brown, Inc.
Carrier Global Corp
Polaris Inc
International Business Machines Corp.
NextEra Energy Inc
Realty Income Corp.
RenaissanceRe Holdings Ltd
Cullen Frost Bankers Inc.
Otis Worldwide Corp
Roper Technologies Inc
West Pharmaceutical Services, Inc.
Bancfirst Corp.
Albemarle Corp.
Aptargroup Inc.

WebdriverManager

Python & Selenium：如何避免来自 dividend.com 的 StaleElementReferenceException 错误

Python & Selenium: How to avoid StaleElementReferenceException error from dividend.com

python

selenium

screen-scraping