硒网页抓取。 ec2 实例无法定位元素，即使它在我的本地机器上工作

Question

我正在尝试从网页中抓取数据。下面的代码适用于我的本地 window 机器，但不适用于 ec2 linux 实例。

from selenium import webdriver  
from selenium.webdriver.common.keys import Keys  

option = webdriver.firefox.options.Options()
option.add_argument("--headless")
driver = webdriver.Firefox(executable_path=dir_firefox_driver, options=option)
driver.get("https://www.mwave.me/en/mcountdown")
time.sleep(3)
driver.set_window_size(600, 800)
temp = driver.find_element_by_css_selector(".chart_view_more button")

看到文章说等待元素出现，所以我尝试了下面的代码

driver = webdriver.Firefox(executable_path=dir_firefox_driver, options=option)
driver.get("https://www.mwave.me/en/mcountdown")
time.sleep(3)
driver.set_window_size(600, 800)
try:
    element = WebDriverWait(driver, 20).until(
        EC.presence_of_element_located((By.CSS_SELECTOR, ".chart_view_more button"))
    )
except:
    print("there is no element")
    quit()
temp = driver.find_element_by_css_selector(".chart_view_more button")

同样不行。我在我的本地机器和 ec2 实例中找不到区别。有人可以给我任何建议吗？

Answer 1

您可以尝试解决此问题的一些方法：-

而不是

driver.set_window_size(600, 800)

使用：

driver.maximize_window()

也可以使用下面的代码：

driver = webdriver.Firefox(executable_path=dir_firefox_driver, options=option)
driver.get("https://www.mwave.me/en/mcountdown")
time.sleep(3)
driver.maximize_window()
try:
    element = WebDriverWait(driver, 20).until(
        EC.visibility_of_element_located((By.CSS_SELECTOR, ".chart_view_more button"))
    )
except:
    print("there is no element")
    quit()
temp = driver.find_element_by_css_selector(".chart_view_more button")

硒网页抓取。 ec2 实例无法定位元素，即使它在我的本地机器上工作

Selenium web scraping. ec2 instance can't locate element even though it works on my local machine

selenium

amazon-ec2

selenium-firefoxdriver

selenium-webdriver