selenium.common.exceptions.TimeoutException: 消息

Question

低于 pst_hldr.

行的异常

又发现错误如下：

  File "/home/PycharmProjects/reditt/redit1.py", line 44, in get_links
    pst_hldr = wait.until(cond.visibility_of_element_located((By.XPATH, ".//*[@class='QBfRw7Rj8UkxybFpX-USO']")))
  File "/usr/local/lib/python3.9/dist-packages/selenium/webdriver/support/wait.py", line 80, in until
    raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message:

代码：

def get_links(*keywords):
    urllist = []
    keystring = ''
    for kw in keywords:
        keystring += kw + "%20"
        keystring = keystring.strip()
    driver = webdriver.Chrome("chromedriver")

    driver.get('https://www.reddit.com/search/?q=' + keystring)
    driver.maximize_window()

    wait = WebDriverWait(driver, 10)
    tbody = wait.until(cond.presence_of_element_located(
        (By.XPATH, "//*[@class='_3ozFtOe6WpJEMUtxDOIvtU']//*[@class='q4a8asWOWdfdniAbgNhMh']")))
    tb_bar = tbody.find_element_by_xpath("//*[@class='_3ozFtOe6WpJEMUtxDOIvtU']//*["
                                         "@class='q4a8asWOWdfdniAbgNhMh']//*[@class='M7VDHU4AdgCc6tHaZ-UUy']")
    driver.execute_script("arguments[0].click();", tb_bar)
    print("end of bar")
    k = 0
    for i in range(200):
        newht = i * 500
        driver.execute_script("window.scrollTo(0, " + str(newht) + ");")
        time.sleep(0.1)

    wait = WebDriverWait(driver, 10)

    pst_hldr = wait.until(cond.visibility_of_element_located((By.XPATH, ".//*[@class='QBfRw7Rj8UkxybFpX-USO']")))
    pst_tiles = pst_hldr.find_elements_by_xpath(
        ".//*[@class='_1poyrkZ7g36PawDueRza-J']//*[@class='_2XDITKxlj4y3M99thqyCsO']//*["
        "@class='_1Y6dfr4zLlrygH-FLmr8x-']")
    for tl in pst_tiles:
        ttl = tl.find_element_by_xpath(".//*[@class='y8HYJ-y_lTUHkQIc1mdCq _2INHSNB8V5eaWp4P0rY_mE']//a")
        href = ttl.get_attribute('href')
        print(href)
    driver.close()

get_links('america', 'coronavirus', 'cases')

Answer 1

这是预期的行为。您正在使用 WebDriverWait，它等待固定时间（您给定的时间）让元素可见。如果在该时间内未找到该元素，则抛出此异常。它是一种告诉你的方式：“嘿，该元素在你设置的时间内没有出现”。

在此处阅读有关此异常的更多信息：https://www.educative.io/edpresso/timeoutexception-in-selenium

Answer 2

那些类本质上看起来是动态的，请尝试使用以下代码：

你可以使用下面的 css_selector :

div[data-testid='search-results-subnav']+div

在代码中，

pst_hldr = wait.until(cond.visibility_of_element_located((By.CSS_SELECTOR, "div[data-testid='search-results-subnav']+div")))

不过，这个例外

raise TimeoutException(message, screen, stacktrace) selenium.common.exceptions.TimeoutException: Message:

很常见，当您使用 WebDriverWait.

显式等待时

所以尝试使用

driver.find_element_by_css("div[data-testid='search-results-subnav']+div")

看看你得到的确切错误是什么。

更新 1：

div[class*='ListingLayout-outerContainer'] div:nth-of-type(2) div:nth-of-type(3)

在代码中：-

pst_hldr = wait.until(cond.visibility_of_element_located((By.CSS_SELECTOR, "div[class*='ListingLayout-outerContainer'] div:nth-of-type(2) div:nth-of-type(3)")))
pst_tiles = wait.until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "a[data-click-id='body'] span")))
for title in pst_tiles:
    print(title.text)

更新 2：

driver.implicitly_wait(30)
driver.maximize_window()
driver.get("https://www.reddit.com/search/?q=america%20coronavirus%20cases%20")
wait = WebDriverWait(driver, 20)
pst_hldr = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "div[class*='ListingLayout-outerContainer'] div:nth-of-type(2) div:nth-of-type(3)")))
pst_tiles = wait.until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "a[data-click-id='body']")))
for title in pst_tiles:
    print(title.get_attribute('href'))

输出：

Timeline suggests a link between Alzheimers, Prion Disease & Covid-19 Vaccines +2,089 New Cases = 1,243,932 Total Cases in PA; +16 New Deaths = 27,941 Total Deaths in PA Take me out of Latam ✈ is that a good bet? r/CoronavirusDownunder random daily discussion thread - 13 August, 2021 The New York parade reappears Asian-Americans gather downstairs to protest Yan Limeng spread rumors about the origin of the virus Florida becomes epicentre of America’s pandemic as coronavirus cases surge 50 per cent. 1 in 5 coronavirus cases nationally is found in Florida Pre-market brief +1,811 New Cases = 1,241,843 Total Cases in PA; +11 New Deaths = 27,925 Total Deaths in PA Florida becomes epicentre of America’s pandemic as coronavirus cases surge 50 per cent What A Day: Reconcilable BIFerences by Sarah Lazarus & Crooked Media (08/11/21) r/CoronavirusDownunder random daily discussion thread - 12 August, 2021 Hungarian nationalism is not the answer - Slow Boring More evidence suggests COVID-19 was in US by Christmas 2019 3 Major Reasons Why I am All in GME Pre-market brief Know anyone who is PRO MASK / VAX / LOCKDOWN? +2,076 New Cases = 1,240,032 Total Cases in PA; +11 New Deaths = 27,914 Total Deaths in PA What A Day: Cuom Over by Sarah Lazarus & Crooked Media (08/10/21) r/CoronavirusDownunder random daily discussion thread - 11 August, 2021 Democrats and their accomplices in the media are working overtime in Texas and Florida to create panic in an effort to pressure governors to give them back the power to reinstate mask mandates and lockdowns in their communities. More evidence suggests COVID-19 was in US by Christmas 2019 +1,280 New Cases = 1,237,956 Total Cases in PA; +1 New Deaths = 27,903 Total Deaths in PA Even if we don't compare the value of animal lives to human lives, the Holocaust comparison is still valid. AMC Talks Bitcoin, GameStop With Its Reddit Followers -- 2nd Update (https://markets.qtrade.ca/news/story?t=iKXJizM-dSY,RB0cWxp8F57oYAx8Odv4T-UoxUxTcyQA&article=651d9d865e21f8f0#651d9d865e21f8f0)

Process finished with exit code 0

selenium.common.exceptions.TimeoutException: 消息

selenium.common.exceptions.TimeoutException: Message

python

selenium

web-scraping

selenium-chromedriver

selenium-webdriver