NoSuchElementException 即使使用 find_element(By.XPATH, value='xpath_value')
NoSuchElementException even when using find_element(By.XPATH, value='xpath_value')
目标：从 https://www.boxofficemojo.com/chart/top_lifetime_gross/?ref_=bo_cso_ac 获取标题并将其转换为列表。
代码：
driver = webdriver.Chrome()
driver.get('https://www.boxofficemojo.com/chart/top_lifetime_gross/?ref_=bo_cso_ac')
xpath = '//td[@class="a-text-left mojo-text-type-title"]/a[@class="a-link-normal"]'
movie_names = driver.find_element(By.XPATH, value=xpath)
错误：
NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//td[@class="a-text-left mojo-text-type-title"]/a[@class="a-link-normal"]"}
感谢您的努力并提前提供帮助。
您可以使用 css attribute = value 选择器，如下所示。这是一种快速的匹配方法，可能比长 xpath 更稳定。这针对包含 title.
的标题 hrefs
from selenium import webdriver
d = webdriver.Chrome()
d.get('https://www.boxofficemojo.com/chart/top_lifetime_gross/?ref_=bo_cso_ac')
titles = [i.text for i in d.find_elements_by_css_selector('[href*=title]')]
print(titles)
考虑到弃用警告，这里是按版本
from selenium import webdriver
from selenium.webdriver.common.by import By
d = webdriver.Chrome()
d.get('https://www.boxofficemojo.com/chart/top_lifetime_gross/?ref_=bo_cso_ac')
titles = [i.text for i in d.find_elements(by=By.CSS_SELECTOR, value='[href*=title]')]
print(titles)
标题是 <a> 标签的 innerText，class a-link-normal 和 href 值以 /title 开头。
解决方案
要提取 标题 你需要诱导 WebDriverWait for visibility_of_all_elements_located() and you can use the following Locator Strategy:
使用CSS_SELECTOR:
driver.get("https://www.boxofficemojo.com/chart/top_lifetime_gross/?ref_=bo_cso_ac")
print([my_elem.text for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "a.a-link-normal[href^='/title']")))])
控制台输出：
['Star Wars: Episode VII - The Force Awakens', 'Avengers: Endgame', 'Spider-Man: No Way Home', 'Avatar', 'Black Panther', 'Avengers: Infinity War', 'Titanic', 'Jurassic World', 'The Avengers', 'Star Wars: Episode VIII - The Last Jedi', 'Incredibles 2', 'The Lion King', 'The Dark Knight', 'Rogue One: A Star Wars Story', 'Star Wars: Episode IX - The Rise of Skywalker', 'Beauty and the Beast', 'Finding Dory', 'Frozen II', 'Star Wars: Episode I - The Phantom Menace', 'Star Wars', 'Avengers: Age of Ultron', 'The Dark Knight Rises', 'Shrek 2', 'E.T. the Extra-Terrestrial', 'Toy Story 4', 'Captain Marvel', 'The Hunger Games: Catching Fire', "Pirates of the Caribbean: Dead Man's Chest", 'The Lion King', 'Jurassic World: Fallen Kingdom', 'Toy Story 3', 'Wonder Woman', 'Iron Man 3', 'Captain America: Civil War', 'The Hunger Games', 'Spider-Man', 'Jumanji: Welcome to the Jungle', 'Jurassic Park', 'Transformers: Revenge of the Fallen', 'Frozen', 'Spider-Man: Far from Home', 'Guardians of the Galaxy Vol. 2', 'Harry Potter and the Deathly Hallows: Part 2', 'Finding Nemo', 'Star Wars: Episode III - Revenge of the Sith', 'The Lord of the Rings: The Return of the King', 'Spider-Man 2', 'The Passion of the Christ', 'The Secret Life of Pets', 'Despicable Me 2', 'The Batman', 'The Jungle Book', 'Deadpool', 'Inside Out', 'Aladdin', 'Furious 7', 'Transformers: Dark of the Moon', 'American Sniper', 'The Lord of the Rings: The Two Towers', 'Zootopia', 'The Hunger Games: Mockingjay - Part 1', 'Spider-Man 3', 'Minions', 'Joker', 'Aquaman', 'Spider-Man: Homecoming', 'Alice in Wonderland', 'Guardians of the Galaxy', 'Forrest Gump', 'Batman v Superman: Dawn of Justice', 'It', 'Suicide Squad', 'Deadpool 2', 'Shrek the Third', 'Jumanji: The Next Level', 'Transformers', 'Iron Man', "Harry Potter and the Sorcerer's Stone", 'Indiana Jones and the Kingdom of the Crystal Skull', 'The Lord of the Rings: The Fellowship of the Ring', 'Thor: Ragnarok', 'Iron Man 2', 'Star Wars: Episode II - Attack of the Clones', "Pirates of the Caribbean: At World's End", 'Star Wars: Episode VI - Return of the Jedi', 'Independence Day', 'Pirates of the Caribbean: The Curse of the Black Pearl', 'Skyfall', 'The Hobbit: An Unexpected Journey', 'Harry Potter and the Half-Blood Prince', 'The Twilight Saga: Eclipse', 'The Twilight Saga: New Moon', 'Harry Potter and the Deathly Hallows: Part 1', 'The Sixth Sense', 'Up', 'Star Wars: Episode V - The Empire Strikes Back', 'Inception', 'Harry Potter and the Order of the Phoenix', 'The Twilight Saga: Breaking Dawn - Part 2', 'The Chronicles of Narnia: The Lion, the Witch and the Wardrobe', 'Man of Steel', 'Monsters, Inc.', 'Harry Potter and the Goblet of Fire', 'Home Alone', 'The Hunger Games: Mockingjay - Part 2', 'The Matrix Reloaded', 'The Twilight Saga: Breaking Dawn - Part 1', 'Meet the Fockers', 'The Hangover', 'Gravity', 'The Grinch', 'Sing', 'Monsters University', 'Shrek', 'Despicable Me 3', 'Harry Potter and the Chamber of Secrets', 'The Amazing Spider-Man', 'The Incredibles', 'Jaws', 'How the Grinch Stole Christmas', 'Captain America: The Winter Soldier', 'The Hobbit: The Desolation of Smaug', 'The Lego Movie', 'Star Trek', 'I Am Legend', 'The Blind Side', 'The Hobbit: The Battle of the Five Armies', 'The Hangover Part II', 'Despicable Me', 'Batman', 'Night at the Museum', 'Men in Black', 'Harry Potter and the Prisoner of Azkaban', 'Moana', 'Indiana Jones and the Raiders of the Lost Ark', 'Toy Story 2', 'Transformers: Age of Extinction', 'Cars', 'Ghostbusters', 'Bruce Almighty', 'Twister', 'My Big Fat Greek Wedding', 'Maleficent', 'Pirates of the Caribbean: On Stranger Tides', 'Shrek Forever After', 'Fast & Furious 6', 'Brave', 'Oz the Great and Powerful', 'Beverly Hills Cop', 'X-Men: The Last Stand', 'War of the Worlds', 'Fantastic Beasts and Where to Find Them', 'X-Men: Days of Future Past', 'Cast Away', 'The Exorcist', 'Doctor Strange', 'The Lost World: Jurassic Park', 'Justice League', 'Star Trek Into Darkness', 'The Martian', 'Signs', 'Hancock', 'The Bourne Ultimatum', 'Logan', 'Rush Hour 2', 'The Fate of the Furious', 'Shang-Chi and the Legend of the Ten Rings', 'WALL·E', 'Toy Story', 'Big Hero 6', 'Mission: Impossible - Fallout', 'National Treasure: Book of Secrets', 'Alvin and the Chipmunks: The Squeakquel', 'Mrs. Doubtfire', 'Beauty and the Beast', 'Ted', 'King Kong', 'Ghost', 'How to Train Your Dragon', 'The Da Vinci Code', 'Aladdin', 'Alvin and the Chipmunks', 'Saving Private Ryan', 'Bohemian Rhapsody', 'Ant-Man and the Wasp', "Madagascar 3: Europe's Most Wanted", 'Kung Fu Panda', 'Mission: Impossible II', 'A Star Is Born', 'X2: X-Men United', 'The Lorax', 'Solo: A Star Wars Story', 'Venom: Let There Be Carnage', 'Venom', 'Austin Powers in Goldmember', 'Back to the Future', 'It Chapter Two', '300', 'Coco', 'Fast Five']
注意：您必须添加以下导入：
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
由于 table 数据不是动态的，您也可以使用 bs4 和请求提取数据。
from bs4 import BeautifulSoup
import requests
url= 'https://www.boxofficemojo.com/chart/top_lifetime_gross/?ref_=bo_cso_ac'
req = requests.get(url)
soup = BeautifulSoup(req.text,'html.parser')
for t in soup.select("a.a-link-normal[href^='/title']"):
print(t.text)
输出：
Star Wars: Episode VII - The Force Awakens
Avengers: Endgame
Spider-Man: No Way Home
Avatar
Black Panther
Avengers: Infinity War
Titanic
Jurassic World
The Avengers
Star Wars: Episode VIII - The Last Jedi
Incredibles 2
The Lion King
The Dark Knight
Rogue One: A Star Wars Story
Star Wars: Episode IX - The Rise of Skywalker
Beauty and the Beast
Finding Dory
Frozen II
Star Wars: Episode I - The Phantom Menace
Star Wars
Avengers: Age of Ultron
The Dark Knight Rises
Shrek 2
E.T. the Extra-Terrestrial
Toy Story 4
Captain Marvel
The Hunger Games: Catching Fire
Pirates of the Caribbean: Dead Man's Chest
The Lion King
Jurassic World: Fallen Kingdom
Toy Story 3
Wonder Woman
Iron Man 3
Captain America: Civil War
The Hunger Games
Spider-Man
Jumanji: Welcome to the Jungle
Jurassic Park
Transformers: Revenge of the Fallen
Frozen
...
目标：从 https://www.boxofficemojo.com/chart/top_lifetime_gross/?ref_=bo_cso_ac 获取标题并将其转换为列表。
代码：
driver = webdriver.Chrome()
driver.get('https://www.boxofficemojo.com/chart/top_lifetime_gross/?ref_=bo_cso_ac')
xpath = '//td[@class="a-text-left mojo-text-type-title"]/a[@class="a-link-normal"]'
movie_names = driver.find_element(By.XPATH, value=xpath)
错误：
NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//td[@class="a-text-left mojo-text-type-title"]/a[@class="a-link-normal"]"}
感谢您的努力并提前提供帮助。
您可以使用 css attribute = value 选择器，如下所示。这是一种快速的匹配方法，可能比长 xpath 更稳定。这针对包含 title.
from selenium import webdriver
d = webdriver.Chrome()
d.get('https://www.boxofficemojo.com/chart/top_lifetime_gross/?ref_=bo_cso_ac')
titles = [i.text for i in d.find_elements_by_css_selector('[href*=title]')]
print(titles)
考虑到弃用警告，这里是按版本
from selenium import webdriver
from selenium.webdriver.common.by import By
d = webdriver.Chrome()
d.get('https://www.boxofficemojo.com/chart/top_lifetime_gross/?ref_=bo_cso_ac')
titles = [i.text for i in d.find_elements(by=By.CSS_SELECTOR, value='[href*=title]')]
print(titles)
标题是 <a> 标签的 innerText，class a-link-normal 和 href 值以 /title 开头。
解决方案
要提取 标题 你需要诱导 WebDriverWait for visibility_of_all_elements_located() and you can use the following Locator Strategy:
使用CSS_SELECTOR:
driver.get("https://www.boxofficemojo.com/chart/top_lifetime_gross/?ref_=bo_cso_ac") print([my_elem.text for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "a.a-link-normal[href^='/title']")))])控制台输出：
['Star Wars: Episode VII - The Force Awakens', 'Avengers: Endgame', 'Spider-Man: No Way Home', 'Avatar', 'Black Panther', 'Avengers: Infinity War', 'Titanic', 'Jurassic World', 'The Avengers', 'Star Wars: Episode VIII - The Last Jedi', 'Incredibles 2', 'The Lion King', 'The Dark Knight', 'Rogue One: A Star Wars Story', 'Star Wars: Episode IX - The Rise of Skywalker', 'Beauty and the Beast', 'Finding Dory', 'Frozen II', 'Star Wars: Episode I - The Phantom Menace', 'Star Wars', 'Avengers: Age of Ultron', 'The Dark Knight Rises', 'Shrek 2', 'E.T. the Extra-Terrestrial', 'Toy Story 4', 'Captain Marvel', 'The Hunger Games: Catching Fire', "Pirates of the Caribbean: Dead Man's Chest", 'The Lion King', 'Jurassic World: Fallen Kingdom', 'Toy Story 3', 'Wonder Woman', 'Iron Man 3', 'Captain America: Civil War', 'The Hunger Games', 'Spider-Man', 'Jumanji: Welcome to the Jungle', 'Jurassic Park', 'Transformers: Revenge of the Fallen', 'Frozen', 'Spider-Man: Far from Home', 'Guardians of the Galaxy Vol. 2', 'Harry Potter and the Deathly Hallows: Part 2', 'Finding Nemo', 'Star Wars: Episode III - Revenge of the Sith', 'The Lord of the Rings: The Return of the King', 'Spider-Man 2', 'The Passion of the Christ', 'The Secret Life of Pets', 'Despicable Me 2', 'The Batman', 'The Jungle Book', 'Deadpool', 'Inside Out', 'Aladdin', 'Furious 7', 'Transformers: Dark of the Moon', 'American Sniper', 'The Lord of the Rings: The Two Towers', 'Zootopia', 'The Hunger Games: Mockingjay - Part 1', 'Spider-Man 3', 'Minions', 'Joker', 'Aquaman', 'Spider-Man: Homecoming', 'Alice in Wonderland', 'Guardians of the Galaxy', 'Forrest Gump', 'Batman v Superman: Dawn of Justice', 'It', 'Suicide Squad', 'Deadpool 2', 'Shrek the Third', 'Jumanji: The Next Level', 'Transformers', 'Iron Man', "Harry Potter and the Sorcerer's Stone", 'Indiana Jones and the Kingdom of the Crystal Skull', 'The Lord of the Rings: The Fellowship of the Ring', 'Thor: Ragnarok', 'Iron Man 2', 'Star Wars: Episode II - Attack of the Clones', "Pirates of the Caribbean: At World's End", 'Star Wars: Episode VI - Return of the Jedi', 'Independence Day', 'Pirates of the Caribbean: The Curse of the Black Pearl', 'Skyfall', 'The Hobbit: An Unexpected Journey', 'Harry Potter and the Half-Blood Prince', 'The Twilight Saga: Eclipse', 'The Twilight Saga: New Moon', 'Harry Potter and the Deathly Hallows: Part 1', 'The Sixth Sense', 'Up', 'Star Wars: Episode V - The Empire Strikes Back', 'Inception', 'Harry Potter and the Order of the Phoenix', 'The Twilight Saga: Breaking Dawn - Part 2', 'The Chronicles of Narnia: The Lion, the Witch and the Wardrobe', 'Man of Steel', 'Monsters, Inc.', 'Harry Potter and the Goblet of Fire', 'Home Alone', 'The Hunger Games: Mockingjay - Part 2', 'The Matrix Reloaded', 'The Twilight Saga: Breaking Dawn - Part 1', 'Meet the Fockers', 'The Hangover', 'Gravity', 'The Grinch', 'Sing', 'Monsters University', 'Shrek', 'Despicable Me 3', 'Harry Potter and the Chamber of Secrets', 'The Amazing Spider-Man', 'The Incredibles', 'Jaws', 'How the Grinch Stole Christmas', 'Captain America: The Winter Soldier', 'The Hobbit: The Desolation of Smaug', 'The Lego Movie', 'Star Trek', 'I Am Legend', 'The Blind Side', 'The Hobbit: The Battle of the Five Armies', 'The Hangover Part II', 'Despicable Me', 'Batman', 'Night at the Museum', 'Men in Black', 'Harry Potter and the Prisoner of Azkaban', 'Moana', 'Indiana Jones and the Raiders of the Lost Ark', 'Toy Story 2', 'Transformers: Age of Extinction', 'Cars', 'Ghostbusters', 'Bruce Almighty', 'Twister', 'My Big Fat Greek Wedding', 'Maleficent', 'Pirates of the Caribbean: On Stranger Tides', 'Shrek Forever After', 'Fast & Furious 6', 'Brave', 'Oz the Great and Powerful', 'Beverly Hills Cop', 'X-Men: The Last Stand', 'War of the Worlds', 'Fantastic Beasts and Where to Find Them', 'X-Men: Days of Future Past', 'Cast Away', 'The Exorcist', 'Doctor Strange', 'The Lost World: Jurassic Park', 'Justice League', 'Star Trek Into Darkness', 'The Martian', 'Signs', 'Hancock', 'The Bourne Ultimatum', 'Logan', 'Rush Hour 2', 'The Fate of the Furious', 'Shang-Chi and the Legend of the Ten Rings', 'WALL·E', 'Toy Story', 'Big Hero 6', 'Mission: Impossible - Fallout', 'National Treasure: Book of Secrets', 'Alvin and the Chipmunks: The Squeakquel', 'Mrs. Doubtfire', 'Beauty and the Beast', 'Ted', 'King Kong', 'Ghost', 'How to Train Your Dragon', 'The Da Vinci Code', 'Aladdin', 'Alvin and the Chipmunks', 'Saving Private Ryan', 'Bohemian Rhapsody', 'Ant-Man and the Wasp', "Madagascar 3: Europe's Most Wanted", 'Kung Fu Panda', 'Mission: Impossible II', 'A Star Is Born', 'X2: X-Men United', 'The Lorax', 'Solo: A Star Wars Story', 'Venom: Let There Be Carnage', 'Venom', 'Austin Powers in Goldmember', 'Back to the Future', 'It Chapter Two', '300', 'Coco', 'Fast Five']注意：您必须添加以下导入：
from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.common.by import By from selenium.webdriver.support import expected_conditions as EC
由于 table 数据不是动态的，您也可以使用 bs4 和请求提取数据。
from bs4 import BeautifulSoup
import requests
url= 'https://www.boxofficemojo.com/chart/top_lifetime_gross/?ref_=bo_cso_ac'
req = requests.get(url)
soup = BeautifulSoup(req.text,'html.parser')
for t in soup.select("a.a-link-normal[href^='/title']"):
print(t.text)
输出：
Star Wars: Episode VII - The Force Awakens
Avengers: Endgame
Spider-Man: No Way Home
Avatar
Black Panther
Avengers: Infinity War
Titanic
Jurassic World
The Avengers
Star Wars: Episode VIII - The Last Jedi
Incredibles 2
The Lion King
The Dark Knight
Rogue One: A Star Wars Story
Star Wars: Episode IX - The Rise of Skywalker
Beauty and the Beast
Finding Dory
Frozen II
Star Wars: Episode I - The Phantom Menace
Star Wars
Avengers: Age of Ultron
The Dark Knight Rises
Shrek 2
E.T. the Extra-Terrestrial
Toy Story 4
Captain Marvel
The Hunger Games: Catching Fire
Pirates of the Caribbean: Dead Man's Chest
The Lion King
Jurassic World: Fallen Kingdom
Toy Story 3
Wonder Woman
Iron Man 3
Captain America: Civil War
The Hunger Games
Spider-Man
Jumanji: Welcome to the Jungle
Jurassic Park
Transformers: Revenge of the Fallen
Frozen
...