在带有 Selenium 的网页中单击 "Show more deals"

Question

我想单击以下页面上的每个 'Show 10 more deals'：“https://www.uswitch.com/broadband/compare/deals_and_offers/”，但它似乎不起作用。我遇到了以下错误：

 AttributeError: 'NoneType' object has no attribute 'find_element'

我的代码如下：

from selenium import webdriver
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By

url = "https://www.uswitch.com/broadband/compare/deals_and_offers/"

driver = webdriver.Chrome(r'C:\temp\chromedriver.exe')

browser = driver.get(url)
while True:
    button = WebDriverWait(browser,10).until(EC.presence_of_element_located((By.PARTIAL_LINK_TEXT, 'Show 10 more deals')))
button.click()

有什么想法吗？

Answer 1

尝试以下方法，它使用 CSS 属性 = 值选择器通过值

定位按钮的 data-event-action 属性

driver.find_element_by_css_selector('[data-event-action="Show 10 more products"]').click()

如果需要，将 driver 替换为 browser。

Answer 2

要在页面 https://www.uswitch.com/broadband/compare/deals_and_offers/ 上单击文本为 显示 10 多笔交易 的元素，您可以使用以下解决方案：

代码块：

from selenium import webdriver
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By

url = "https://www.uswitch.com/broadband/compare/deals_and_offers/"
options = webdriver.ChromeOptions() 
options.add_argument("start-maximized")
options.add_argument('disable-infobars')
browser = webdriver.Chrome(chrome_options=options, executable_path=r'C:\Utility\BrowserDrivers\chromedriver.exe')
browser.get(url)
while True:
    try:
        browser.execute_script("return arguments[0].scrollIntoView(true);", WebDriverWait(browser,20).until(EC.visibility_of_element_located((By.XPATH, "//button[@class='us-btn us-btn--action' and contains(.,'Show 10 more deals')]"))))
        browser.execute_script("arguments[0].click();", WebDriverWait(browser,20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button.us-btn.us-btn--action[name='visible_products']"))))
        print("Button clicked")
    except:
        print("No more Buttons")
        break
browser.quit()

控制台输出：

Button clicked
Button clicked
Button clicked
Button clicked
Button clicked
Button clicked
Button clicked
Button clicked
Button clicked
Button clicked
No more Buttons

Answer 3

这样试试：

while not re.search(r"Showing (\d+) of  ", browser.page_source):
  browser.execute_script("document.querySelector('[data-event-label=\"Show 10 more products\"]').click()")
  time.sleep(1)

这避免了最终会让您发疯的 selenium 错误。

在带有 Selenium 的网页中单击 "Show more deals"

Click on "Show more deals" in webpage with Selenium

python

selenium

lazy-loading

web-scraping

webdriverwait