Selenium 在 for 循环中返回上一页

Question

我正在制作一个抓取器来抓取博彩网站的数据，这是一段示例代码，可以抓取赛事、球队和赔率。这个想法是遍历所有可用于这项运动的比赛。我能够找到 link 并单击第一个，然后我也能够抓取数据并将其 return 放入列表中（稍后将其放入数据库）。 我的问题是我无法 return 到上一页以单击下一页 link，我在尝试执行此操作时遇到错误。这是我的代码：

driver = webdriver.Chrome(Path)
driver.get("https://www.neds.com.au/sports/table-tennis/")
a = []

links = driver.find_elements_by_class_name("matches-filter__link")

for l in links:
    
    l.click()
    tt_matches = driver.find_elements_by_class_name("sport-event-card")
    for match in tt_matches:
        Match = match.find_element_by_css_selector(".sports-event-title__name-text").text
        a.append(Match)
        Teams = match.find_elements_by_css_selector(".price-button")
        for team in Teams:
            team_name = team.find_element_by_css_selector(".price-button-name").text
            team_odd = team.find_element_by_css_selector(".price-button-odds-price span").text
            a.append(team_name)
            a.append(team_odd)
    driver.back()
    time.sleep(2)

driver.quit()

它 return 是这个错误：

raise exception_class(message, screen, stacktrace)

StaleElementReferenceException: stale element reference: element is not attached to the page document (Session info: chrome=84.0.4147.105)

我该如何解决这个问题，我认为问题出在 driver.back() 上，我也试过 l.back() 但它仍然不起作用。

Answer 1

在这种情况下，您需要收集 URL，然后导航到它们。您可以使用以下代码：

driver.get("https://www.neds.com.au/sports/table-tennis/")
time.sleep(5)
a = []

links = driver.find_elements_by_class_name("matches-filter__link")
urls = [l.get_attribute('href') for l in links]
for u in urls:
    print(u)
    driver.get(u)
    tt_matches = driver.find_elements_by_class_name("sport-event-card")
    for match in tt_matches:
        Match = match.find_element_by_css_selector(".sports-event-title__name-text").text
        a.append(Match)
        Teams = match.find_elements_by_css_selector(".price-button")
        for team in Teams:
            team_name = team.find_element_by_css_selector(".price-button-name").text
            team_odd = team.find_element_by_css_selector(".price-button-odds-price span").text
            a.append(team_name)
            a.append(team_odd)
    time.sleep(2)

print(a)
driver.quit()

Selenium 在 for 循环中返回上一页

Selenium returning to previous page in a for loop

selenium

beautifulsoup

web-scraping

staleelementreferenceexception