Python Selenium Error: Message: stale element reference: element is not attached to the page document

Question

我有一个客户想要从网站上抓取一些信息。该循环第一次工作，但第二次发生错误。有什么帮助吗？我也不建议去实际的网站，它是粗略的。

代码：

URL = 'https://avbebe.com/archives/category/高清中字/page/5'
driver = webdriver.Chrome(executable_path=PATH, options=options)
driver.get(URL)

time.sleep(5)

Vids = WebDriverWait(driver, 10).until(EC.visibility_of_any_elements_located((By.CLASS_NAME, 'entry-thumbnails-link')))
for title in Vids:
    time.sleep(3)
    actions = ActionChains(driver)
    WebDriverWait(driver, 10).until(EC.presence_of_all_elements_located((By.CLASS_NAME, 'entry-thumbnails-link')))
    driver.execute_script("arguments[0].click();", title)#Where error occurs
    time.sleep(5)
    VidUrl = driver.current_url
    VidTitle = driver.find_element_by_xpath('//*[@id="post-69331"]/h1/a').text
    try:
        VidTags = driver.find_elements_by_class_name('tags')
        for tag in VidTags:
            VidTag = tag.find_element_by_tag_name('a').text
        
    except NoSuchElementException or StaleElementReferenceException:
        pass

    with open('data.csv', 'w', newline='', encoding = "utf-8") as f:
        fieldnames = ['Title', 'Tags', 'URL']
        thewriter = csv.DictWriter(f, fieldnames=fieldnames)

        thewriter.writeheader()
        thewriter.writerow({'Title': VidTitle, 'Tags': VidTag, 'URL': VidUrl})
    driver.get(URL)
    print('done')

错误：

selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: element is not attached to the page document

完整的终端回溯：

done Traceback (most recent call last): File "c:\Users\Heage\Coding\Freelancing\Clients\Genewang-webscrape\main.py", line 24, in driver.execute_script("arguments[0].click();", title) File "C:\Users\Heage\AppData\Local\Programs\Python\Python39\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 634, in execute_script return self.execute(command, { File "C:\Users\Heage\AppData\Local\Programs\Python\Python39\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute self.error_handler.check_response(response) File "C:\Users\Heage\AppData\Local\Programs\Python\Python39\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response raise exception_class(message, screen, stacktrace) selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: element is not attached to the page document (Session info: chrome=91.0.4472.114)

Answer 1

可以使用普通的点击方式，将元素存入列表，获取元素索引，点击索引。

您可以使用以下代码更新您的代码

video_text = driver.find_elements_by_class_name("entry-thumbnails-link")
    WebDriverWait(driver, 10).until(EC.presence_of_all_elements_located((By.CLASS_NAME, 'entry-thumbnails-link')))
video_text[0].click()

完整代码如下所示

Vids = WebDriverWait(driver, 10).until(EC.visibility_of_any_elements_located((By.CLASS_NAME, 'entry-thumbnails-link')))
for title in Vids:
    time.sleep(3)
    actions = ActionChains(driver)
    video_text = driver.find_elements_by_class_name("entry-thumbnails-link")
    WebDriverWait(driver, 10).until(EC.presence_of_all_elements_located((By.CLASS_NAME, 'entry-thumbnails-link')))
    video_text[0].click()
    time.sleep(5)
    VidUrl = driver.current_url
    VidTitle = driver.find_element_by_xpath('//*[@id="post-69331"]/h1/a').text
    try:
        VidTags = driver.find_elements_by_class_name('tags')
        for tag in VidTags:
            VidTag = tag.find_element_by_tag_name('a').text

    except NoSuchElementException or StaleElementReferenceException:
        pass

    with open('data.csv', 'w', newline='', encoding="utf-8") as f:
        fieldnames = ['Title', 'Tags', 'URL']
        thewriter = csv.DictWriter(f, fieldnames=fieldnames)

        thewriter.writeheader()
        thewriter.writerow({'Title': VidTitle, 'Tags': VidTag, 'URL': VidUrl})
    driver.get(URL)
    print('done')

也尽量不要使用这么多sleep()。

Python Selenium Error: Message: stale element reference: element is not attached to the page document

Python Selenium Error: Message: stale element reference: element is not attached to the page document

python

selenium

element

web-scraping

selenium-webdriver