Selenium WebDriver/ChromeDriver 使用 Selenium 进行 WebScraping 时出错

Question

我正在尝试从以下网站抓取 table - https://www.cmegroup.com/markets/energy/crude-oil/brent-crude-oil.settlements.html#tradeDate=03%2F31%2F2022

尝试了以下代码块，但似乎return 出现错误消息。

我是网络抓取的新手，非常感谢对以下问题的任何帮助。

代码：

import time
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
import pandas as pd
from bs4 import BeautifulSoup

driver = webdriver.Chrome(ChromeDriverManager().install())
driver.maximize_window()
driver.get(url)
time.sleep(5)

soup = BeautifulSoup(driver.page_source, 'lxml')

price = pd.read_html(str(soup))[0]
print(price)

输出：

===== WebDriver manager ======
Could not get version for google-chrome.Is google-chrome installed?
Get LATEST chromedriver version for None google-chrome
There is no [linux64] chromedriver for browser None in cache
Trying to download new driver from https://chromedriver.storage.googleapis.com/100.0.4896.60/chromedriver_linux64.zip
Driver has been saved in cache [/root/.wdm/drivers/chromedriver/linux64/100.0.4896.60]
/usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py:1: DeprecationWarning: executable_path has been deprecated, please pass in a Service object
  """Entry point for launching an IPython kernel.
---------------------------------------------------------------------------
WebDriverException                        Traceback (most recent call last)
<ipython-input-4-c026f0c15647> in <module>()
----> 1 driver = webdriver.Chrome(ChromeDriverManager().install())
3 frames
/usr/local/lib/python3.7/dist-packages/selenium/webdriver/common/service.py in assert_process_still_running(self)
    110             raise WebDriverException(
    111                 'Service %s unexpectedly exited. Status code was: %s'
--> 112                 % (self.path, return_code)
    113             )
    114 
WebDriverException: Message: Service /root/.wdm/drivers/chromedriver/linux64/100.0.4896.60/chromedriver unexpectedly exited. Status code was: -6

Answer 1

您必须按照此处的说明为您的浏览器安装浏览器驱动程序https://www.selenium.dev/documentation/webdriver/getting_started/install_drivers/

Selenium WebDriver/ChromeDriver 使用 Selenium 进行 WebScraping 时出错

Selenium WebDriver/ChromeDriver Error During WebScraping with Selenium

web-scraping

selenium-chromedriver

selenium-webdriver