JS/AJAX 使用 Selenium(Python) 访问 URL 时未加载内容

Question

我正在尝试抓取这个 URL：https://www.wheel-size.com/size/acura/mdx/2001/

我要抓取的值是动态加载的，例如 Center Bore 如果您在普通浏览器中打开 link，内容加载得很好，但如果我使用 Selenium(chromedriver)，它只会继续加载，并且永远不会显示值。

知道如何抓取它吗？下面是它的外观图片。普通浏览器打开link也能看到加载1-2秒。

Answer 1

提取所需的文本，例如64.1 mm, 5x114.3等元素是 Google Tag Manager enabled elements you need to induce WebDriverWait for the and you can use the following :

options = Options()
options.add_argument("start-maximized")
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
options.add_argument('--disable-blink-features=AutomationControlled')
s = Service('C:\BrowserDrivers\chromedriver.exe')
driver = webdriver.Chrome(service=s, options=options)
driver.get('https://www.wheel-size.com/size/acura/mdx/2001/')
print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//span[contains(., 'Center Bore')]//following::span[1]"))).text)
print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//span[contains(., 'PCD')]//following::span[1]"))).text)

控制台输出：

64.1 mm
5x114.3

注意：您必须添加以下导入：

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

You can find a relevant discussion in

JS/AJAX 使用 Selenium(Python) 访问 URL 时未加载内容

JS/AJAX Content Not loading when URL is accessed using Selenium(Python)

javascript

python

ajax

selenium

web-scraping