如何在 Headless 模式下将 Chrome 浏览器配置为 运行

Question

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait

from time import sleep

# Gumtree credentials
username = "my username"
password = "my password"

# Removes SSL Issues With Chrome
options = webdriver.ChromeOptions()
options.add_argument('--ignore-certificate-errors')
options.add_argument('--ignore-ssl-errors')
options.add_argument('--ignore-certificate-errors-spki-list')
options.add_argument('log-level=3') 
options.add_argument('--disable-notifications')

# Initiate Headless Chrome
#options.add_argument('--headless')
#options.add_argument('--disable-gpu')
#options.headless = True

# Initiate Chrome Driver
url = 'https://my.gumtree.com/login'
driver = webdriver.Chrome(executable_path="C:\webdrivers\chromedriver.exe",options=options)
driver.get(url)

# Find Username/Email Field and Send to Input Field
driver.find_element_by_id("email").send_keys(username)
# Find Password Field and Send to Input Field
driver.find_element_by_id("fld-password").send_keys(password)

# Initiate Consent Button
consent_button_xpath = '//*[@id="login-form"]/div/form/fieldset/div[3]/button'
consent = WebDriverWait(driver, 40).until(EC.element_to_be_clickable((By.XPATH, consent_button_xpath)))
consent = driver.find_element_by_xpath(consent_button_xpath)
consent.click()

# Print Username to Test Successful Login
login_username = driver.find_element_by_xpath('//*[@id="is-being-refined"]/div[3]/div/header/div[1]/div/nav/div/ul/li[5]/a/div').text
print(login_username)

# close the driver
driver.close()

以上脚本使用 Selenium 成功地自动登录网站并打印用户名。

问题

我正在尝试使用 Headless Chrome 在静默模式下执行上述操作，但没有任何乐趣。
我研究了很多文章，只是添加开关 options.add_argument('--headless') 导致脚本失败。

如有任何帮助，我们将不胜感激。

错误信息

[0830/063818.313:ERROR:ssl_client_socket_impl.cc(981)] handshake failed; returned -1, SSL error code 1, net_error -101
[0830/063818.313:ERROR:ssl_client_socket_impl.cc(981)] handshake failed; returned -1, SSL error code 1, net_error -101
Traceback (most recent call last):
  File "e:\Python Projects\Gumtree\test5.py", line 39, in <module>
    consent.click()
  File "C:\Python39\lib\site-packages\selenium\webdriver\remote\webelement.py", line 80, in click
    self._execute(Command.CLICK_ELEMENT)
  File "C:\Python39\lib\site-packages\selenium\webdriver\remote\webelement.py", line 633, in _execute
    return self._parent.execute(command, params)
  File "C:\Python39\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "C:\Python39\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.ElementClickInterceptedException: Message: element click intercepted: Element <button class="btn-primary btn-full-width" type="submit" data-analytics="gaEvent:...Attempt,userData:{lip:Email}">Login</button> is not clickable at point (386, 564). Other element would receive the click: <button id="onetrust-pc-btn-handler" tabindex="0" class="cookie-setting-link">...</button>
  (Session info: headless chrome=92.0.4515.159)

Answer 1

我建议不要使用鼠标点击提交按钮，而是使用键盘：

from selenium.webdriver.common.keys import Keys
consent.send_keys(Keys.ENTER)

Answer 2

这个错误：

selenium.common.exceptions.ElementClickInterceptedException: Message: element click intercepted: Element Login is not clickable at point (386, 564). Other element would receive the click: ... (Session info: headless chrome=92.0.4515.159)

与不切换到无头模式的 selenium 没有任何关系。

你写作的那一刻

options.add_argument("--headless")

并在 webdriver 对象创建调用中传递此选项引用，Selenium 会将其视为无头模式。

1. 此外，我建议您在需要时使用 explicit waits/Implicit waits。

2. Headless 从来都不稳定，到处都是问题。如果你只是想做 web scraping 然后考虑不同的工具，如 BS4 或请求模块。

3. 你还应该使用 relative xpath 而不是 absolute xpath.