无法从亚马逊抓取产品价格

Question

def scrapedPage(URL, user_agent):
    if (not os.environ.get('PYTHONHTTPSVERIFY', '') and getattr(ssl, '_create_unverified_context', None)):
        ssl._create_default_https_context = ssl._create_unverified_context

    header = {"User-Agent": user_agent}

    proxy = {'https': 'using-proxy'}

    page = requests.get(URL, headers=header, proxies=proxy)


    return page

我无法获取 12,300 卢比的产品价格

user_agent = random.choice(user_agent_list)

pricelink='https://www.amazon.in/gp/offer-listing/B07NXTH1BD/ref=dp_olp_afts?ie=UTF8&condition=all'

new_page = scrapedPage(pricelink, user_agent)

new_soup = BeautifulSoup(new_page.content, 'html.parser')

print(new_soup.prettify())


find_price = new_soup.find_all('span',attrs={'style':'text-decoration: inherit; white-space: nowrap'})

我尝试了很多方法，如 findAll()、select()、find() 等，但我无法以同样的方式获得价格。请帮帮我。

正在返回空列表

Answer 1

男性在您的代码中进行了更改：

new_soup = BeautifulSoup(new_page.content, 'lxml') # html.parser > lxml

find_price = new_soup.select('span.olpOfferPrice > span') # change selector
print(find_price[0].text)

输出

Rs. 12,300.00

无法从亚马逊抓取产品价格

Unable to Scrape Product Price from Amazon

python-3.x

web-scraping

beautifulsoup

scrapy

scrape