在主页中获取 link 和子 link 的标题

Question

我想在网站首页里面搞一个link, 我试图通过selenium获取它但只获取主页link。请帮我把所有 link 都放进去。

代码试验：

from selenium import webdriver
import time
driver = webdriver.Edge()
driver.get('https://m.cafe.naver.com/ca-fe/minivelobike')

time.sleep(7)
elems = driver.find_elements_by_xpath("//a[@href]")
for elem in elems:
    print(elem.get_attribute("href"))
driver.close()

link 的快照：

Answer 1

我认为这应该可行

from selenium import webdriver
from selenium.webdriver.common.by import By
import time
driver = webdriver.Edge()
driver.get('https://m.cafe.naver.com/ca-fe/minivelobike')
time.sleep(7)
wrapper = driver.find_element(By.ID, "ct")
main_ul = wrapper.find_element(By.TAG_NAME, "ul")
for li in main_ul.find_elements(By.TAG_NAME, "li"):
    try:
        anchor_tag = li.find_element(By.TAG_NAME, "a")
        href = anchor_tag.get_attribute("href")
        print(href)
    except:
        print("Anchor tag doesnt exist")

Answer 2

打印文章的href属性值需要归纳WebDriverWait for the and you can use either of the following :

使用CSS_SELECTOR:

driver.get("https://m.cafe.naver.com/ca-fe/minivelobike")
print([my_elem.get_attribute("href") for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "a.txt_area[href]")))])

使用 XPATH:

driver.get("https://m.cafe.naver.com/ca-fe/minivelobike")
print([my_elem.get_attribute("href") for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//a[@class='txt_area' and @href]")))])

控制台输出：

['https://m.cafe.naver.com/ArticleRead.nhn?clubid=11853711&articleid=1074719&boardtype=L', 'https://m.cafe.naver.com/ArticleRead.nhn?clubid=11853711&articleid=1074718&boardtype=L', 'https://m.cafe.naver.com/ArticleRead.nhn?clubid=11853711&articleid=1074717&boardtype=L', 'https://m.cafe.naver.com/ArticleRead.nhn?clubid=11853711&articleid=1074716&boardtype=L', 'https://m.cafe.naver.com/ArticleRead.nhn?clubid=11853711&articleid=1074715&boardtype=L', 'https://m.cafe.naver.com/ArticleRead.nhn?clubid=11853711&articleid=1074714&boardtype=L', 'https://m.cafe.naver.com/ArticleRead.nhn?clubid=11853711&articleid=1074713&boardtype=L', 'https://m.cafe.naver.com/ArticleRead.nhn?clubid=11853711&articleid=1074712&boardtype=L', 'https://m.cafe.naver.com/ArticleRead.nhn?clubid=11853711&articleid=1074711&boardtype=L', 'https://m.cafe.naver.com/ArticleRead.nhn?clubid=11853711&articleid=1074710&boardtype=L', 'https://m.cafe.naver.com/ArticleRead.nhn?clubid=11853711&articleid=1074709&boardtype=L', 'https://m.cafe.naver.com/ArticleRead.nhn?clubid=11853711&articleid=1074708&boardtype=L', 'https://m.cafe.naver.com/ArticleRead.nhn?clubid=11853711&articleid=1074707&boardtype=L', 'https://m.cafe.naver.com/ArticleRead.nhn?clubid=11853711&articleid=1074704&boardtype=L', 'https://m.cafe.naver.com/ArticleRead.nhn?clubid=11853711&articleid=1074703&boardtype=L', 'https://m.cafe.naver.com/ArticleRead.nhn?clubid=11853711&articleid=1074702&boardtype=L', 'https://m.cafe.naver.com/ArticleRead.nhn?clubid=11853711&articleid=1074701&boardtype=L', 'https://m.cafe.naver.com/ArticleRead.nhn?clubid=11853711&articleid=1074700&boardtype=L', 'https://m.cafe.naver.com/ArticleRead.nhn?clubid=11853711&articleid=1074699&boardtype=L', 'https://m.cafe.naver.com/ArticleRead.nhn?clubid=11853711&articleid=1074698&boardtype=L', 'https://m.cafe.naver.com/ArticleRead.nhn?clubid=11853711&articleid=1074697&boardtype=L', 'https://m.cafe.naver.com/ArticleRead.nhn?clubid=11853711&articleid=1074696&boardtype=L', 'https://m.cafe.naver.com/ArticleRead.nhn?clubid=11853711&articleid=1074695&boardtype=L', 'https://m.cafe.naver.com/ArticleRead.nhn?clubid=11853711&articleid=1074694&boardtype=L', 'https://m.cafe.naver.com/ArticleRead.nhn?clubid=11853711&articleid=1074693&boardtype=L', 'https://m.cafe.naver.com/ArticleRead.nhn?clubid=11853711&articleid=1074692&boardtype=L', 'https://m.cafe.naver.com/ArticleRead.nhn?clubid=11853711&articleid=1074691&boardtype=L', 'https://m.cafe.naver.com/ArticleRead.nhn?clubid=11853711&articleid=1074690&boardtype=L', 'https://m.cafe.naver.com/ArticleRead.nhn?clubid=11853711&articleid=1074689&boardtype=L', 'https://m.cafe.naver.com/ArticleRead.nhn?clubid=11853711&articleid=1074688&boardtype=L', 'https://m.cafe.naver.com/ArticleRead.nhn?clubid=11853711&articleid=1074687&boardtype=L', 'https://m.cafe.naver.com/ArticleRead.nhn?clubid=11853711&articleid=1074686&boardtype=L', 'https://m.cafe.naver.com/ArticleRead.nhn?clubid=11853711&articleid=1074685&boardtype=L', 'https://m.cafe.naver.com/ArticleRead.nhn?clubid=11853711&articleid=1074682&boardtype=L', 'https://m.cafe.naver.com/ArticleRead.nhn?clubid=11853711&articleid=1074681&boardtype=L', 'https://m.cafe.naver.com/ArticleRead.nhn?clubid=11853711&articleid=1074680&boardtype=L', 'https://m.cafe.naver.com/ArticleRead.nhn?clubid=11853711&articleid=1074679&boardtype=L', 'https://m.cafe.naver.com/ArticleRead.nhn?clubid=11853711&articleid=1074678&boardtype=L', 'https://m.cafe.naver.com/ArticleRead.nhn?clubid=11853711&articleid=1074677&boardtype=L', 'https://m.cafe.naver.com/ArticleRead.nhn?clubid=11853711&articleid=1074676&boardtype=L', 'https://m.cafe.naver.com/ArticleRead.nhn?clubid=11853711&articleid=1074675&boardtype=L', 'https://m.cafe.naver.com/ArticleRead.nhn?clubid=11853711&articleid=1074674&boardtype=L', 'https://m.cafe.naver.com/ArticleRead.nhn?clubid=11853711&articleid=1074673&boardtype=L', 'https://m.cafe.naver.com/ArticleRead.nhn?clubid=11853711&articleid=1074672&boardtype=L', 'https://m.cafe.naver.com/ArticleRead.nhn?clubid=11853711&articleid=1074671&boardtype=L', 'https://m.cafe.naver.com/ArticleRead.nhn?clubid=11853711&articleid=1074670&boardtype=L', 'https://m.cafe.naver.com/ArticleRead.nhn?clubid=11853711&articleid=1074669&boardtype=L', 'https://m.cafe.naver.com/ArticleRead.nhn?clubid=11853711&articleid=1074668&boardtype=L', 'https://m.cafe.naver.com/ArticleRead.nhn?clubid=11853711&articleid=1074667&boardtype=L', 'https://m.cafe.naver.com/ArticleRead.nhn?clubid=11853711&articleid=1074666&boardtype=L']

注意：您必须添加以下导入：

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

在主页中获取 link 和子 link 的标题

Get link and title of sub link in a homepage

selenium

xpath

css-selectors

selenium-webdriver

webdriverwait