如何向下滚动网页直到单击特定按钮？

Question

我正在使用 Python 和 Selenium，但我被卡住抓取试图向下滚动的页面，直到 selenium 找到经典的“显示更多”按钮（看起来被隐藏）。

到目前为止我所做的如下：

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By


from selenium.webdriver.common.keys import Keys


element = browser.find_element_by_xpath('//*[@id="ember371"]/button') # you can use ANY way to locate element
coordinates = element.location_once_scrolled_into_view # returns dict of X, Y coordinates
browser.execute_script('window.scrollTo({}, {});'.format(coordinates['x'], coordinates['y']))

然而，无论我尝试做什么 Python 都会抛出几种错误，例如下面的错误。

NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//*[@id="ember371"]/button"}
  (Session info: chrome=80.0.3987.122)

难道是因为按钮被隐藏了？

下面是我要查找并想点击的按钮的 HTML 代码。

<button class="pv-profile-section__card-action-bar pv-skills-section__additional-skills artdeco-container-card-action-bar artdeco-button artdeco-button--tertiary artdeco-button--3 artdeco-button--fluid" aria-controls="skill-categories-expanded" aria-expanded="false" data-control-name="skill_details" data-ember-action="" data-ember-action-2182="2182">
                    <span aria-hidden="true">
                    Show more
                  </span>
                  <span class="visually-hidden">
                    BLA BLA BLA
                  </span>
                  <li-icon aria-hidden="true" type="chevron-down-icon" class="pv-skills-section__chevron-icon" size="small"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 16 16" data-supported-dps="16x16" fill="currentColor" width="16" height="16" focusable="false">
  <path d="M8 9l5.93-4L15 6.54l-6.15 4.2a1.5 1.5 0 01-1.69 0L1 6.54 2.07 5z"></path>
</svg></li-icon>

</button>

############################################# ####################################

它适用于以下内容：

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC


# scroll down smoothly
scheight = .1
while scheight < 1.0:
    browser.execute_script("window.scrollTo(0, document.body.scrollHeight*%s);" % scheight)
    scheight += .1
    time.sleep(1)
try:
    browser.execute_script("arguments[0].scrollIntoView(true);", WebDriverWait(browser, 10).until(EC.visibility_of_element_located((By.XPATH, "//button[@aria-controls='skill-categories-expanded' and @data-control-name='skill_details']/span[normalize-space()='Show more']"))))
    browser.execute_script("arguments[0].click();", WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[@aria-controls='skill-categories-expanded' and @data-control-name='skill_details']/span[normalize-space()='Show more']"))))
except:
    print('NO')

Answer 1

你可以在这里尝试使用while循环，检查按钮是否存在作为条件，并将browser.execute_script放在正文中。通过这种方式，您将首先检查按钮是否存在，如果不存在则滚动 200 像素左右（只要确保您没有滚动到按钮旁边）直到按钮出现。

Answer 2

您可以先尝试使用 isEnabled() 方法查找 "show more" 按钮是否启用。此方法 returns 如果网页上启用了指定的 Web 元素，则为“true”值；否则 returns 如果网页上禁用了该 Web 元素，则为“false”值。

如果 isEnabled() 方法 returns False，您可以尝试向下滚动页面直到元素可见，然后再次使用 isEnabled() 方法。

另外，尝试用 XPath 替换 id。可能有用。

Answer 3

元素 显示更多 linkedin website is EmberJS enabled element. So to scrollIntoView the element you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following :

使用CSS_SELECTOR:

driver.execute_script("arguments[0].scrollIntoView(true);", WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "button[aria-controls='skill-categories-expanded'][data-control-name='skill_details']>span"))))
driver.execute_script("arguments[0].click();", WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button[aria-controls='skill-categories-expanded'][data-control-name='skill_details']>span"))))

使用XPATH:

driver.execute_script("arguments[0].scrollIntoView(true);", WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.XPATH, "//button[@aria-controls='skill-categories-expanded' and @data-control-name='skill_details']/span[normalize-space()='Show more']"))))
driver.execute_script("arguments[0].click();", WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[@aria-controls='skill-categories-expanded' and @data-control-name='skill_details']/span[normalize-space()='Show more']"))))

注意：您必须添加以下导入：

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

如何向下滚动网页直到单击特定按钮？

How to scroll a web page down until specific button to click?

python

python-3.x

selenium

linkedin