"Message: no such element: Unable to locate element: {"method":"xpath","selector":"//button[normalize-space()="LOAD MORE"]"}

Question

我正在尝试编写一个抓取工具，我希望抓取工具向下滚动，直到到达“加载更多”按钮，然后单击它，但它告诉我不存在这样的按钮。这是代码：

SCROLL_PAUSE_TIME = 5

# Get scroll height
last_height = driver.execute_script("return document.body.scrollHeight")

# Keep scrolling
while True:
    # Scroll to bottom
    driver.execute_script(
        "window.scrollTo(0, document.body.scrollHeight);")

    # Wait to load page
    time.sleep(SCROLL_PAUSE_TIME)

    # Calculate new scroll height and compare with last scroll height
    new_height = driver.execute_script("return document.body.scrollHeight")
    if new_height == last_height:
        break
    last_height = new_height

driver.find_element_by_xpath(
    '//button[normalize-space()="LOAD MORE"]').click()
time.sleep(SCROLL_PAUSE_TIME)
while True:
    # Scroll to bottom
    driver.execute_script(
        "window.scrollTo(0, document.body.scrollHeight);")

    # Wait to load page
    time.sleep(SCROLL_PAUSE_TIME)

    # Calculate new scroll height and compare with last scroll height
    new_height = driver.execute_script("return document.body.scrollHeight")
    if new_height == last_height:
        break
    last_height = new_height

这里是 url：https://messari.io/governor/daos?types=Protocol

Answer 1

经过测试的工作解决方案

import time
from selenium import webdriver
from selenium.common.exceptions import NoSuchElementException 

driver = webdriver.Firefox()

driver.get("https://messari.io/governor/daos?types=Protocol")



def check_exists_by_xpath(xpath):
    try:
        driver.find_element_by_xpath(xpath)
    except NoSuchElementException:
        return False
    return True


xpath="//*[contains(text(), 'Load More')]"

循环以继续滚动直到找到按钮并暂停 3 秒以加载页面。

while  check_exists_by_xpath(xpath)==False :

    time.sleep(3)
    driver.execute_script("window.scrollTo(0, document.body.scrollHeight)")
    print("scrolling")



print('button successfully found , clicking ')
driver.find_element_by_xpath(xpath).click()
print('finish')

输出

scrolling
scrolling
scrolling
scrolling
scrolling
scrolling
scrolling
scrolling
scrolling
button successfully found , clicking 
finish