为什么我不能用 selenium python 抓取特定的亚马逊音乐页面?
Why can't i scrape the particular Amazon music page with selenium python?
https://www.amazon.com/Prettymuch-EP-PRETTYMUCH/dp/B07CF6YXDP
上面提到的纯link,而不是用stack overflow标签点击后直接跳转到的link
这是url。
def get_soup(url):
headers = {'User-Agent':
'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36',
}
r = requests.get(url, headers=headers)
r.raise_for_status()
return BeautifulSoup(r.text, 'lxml')
url = input("Please enter an Amazon music url:")
soup = get_soup(url)
我通过它请求时出错,为什么会这样?
Please enter an Amazon music url:https://www.amazon.com/Prettymuch-EP-
PRETTYMUCH/dp/B07CF6YXDP
Traceback (most recent call last):
File "D:/Pycharm (4)/selemin.py", line 4, in <module>
import amazon
File "D:\Pycharm (4)\amazon.py", line 63, in <module>
soup = get_soup(url)
File "D:\Pycharm (4)\amazon.py", line 12, in get_soup
r.raise_for_status()
File "C:\Users\HP\AppData\Local\Programs\Python\Python37-32\lib\site-
packages\requests\models.py", line 940, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url:
https://www.amazon.com/Prettymuch-EP-PRETTYMUCH/dp/B07CF6YXDP%20
查看错误告诉您的内容 - 这是不同的 url。特别是,它会抛出一个错误,因为它以 %20
结尾。这意味着在您输入的末尾有一个 space。我建议处理您的输入以避免这种情况,例如
new_url = url.strip()
https://www.amazon.com/Prettymuch-EP-PRETTYMUCH/dp/B07CF6YXDP
上面提到的纯link,而不是用stack overflow标签点击后直接跳转到的link
这是url。
def get_soup(url):
headers = {'User-Agent':
'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36',
}
r = requests.get(url, headers=headers)
r.raise_for_status()
return BeautifulSoup(r.text, 'lxml')
url = input("Please enter an Amazon music url:")
soup = get_soup(url)
我通过它请求时出错,为什么会这样?
Please enter an Amazon music url:https://www.amazon.com/Prettymuch-EP-
PRETTYMUCH/dp/B07CF6YXDP
Traceback (most recent call last):
File "D:/Pycharm (4)/selemin.py", line 4, in <module>
import amazon
File "D:\Pycharm (4)\amazon.py", line 63, in <module>
soup = get_soup(url)
File "D:\Pycharm (4)\amazon.py", line 12, in get_soup
r.raise_for_status()
File "C:\Users\HP\AppData\Local\Programs\Python\Python37-32\lib\site-
packages\requests\models.py", line 940, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url:
https://www.amazon.com/Prettymuch-EP-PRETTYMUCH/dp/B07CF6YXDP%20
查看错误告诉您的内容 - 这是不同的 url。特别是,它会抛出一个错误,因为它以 %20
结尾。这意味着在您输入的末尾有一个 space。我建议处理您的输入以避免这种情况,例如
new_url = url.strip()