如何在美汤和python中指定css选择器？

Question

我正在尝试从 link 中抓取卡片的标题：https://www.axisbank.com/retail/cards/credit-card

使用下面的代码

from urllib.request import urlopen
from bs4 import BeautifulSoup
import json, requests, re

axis_url = ["https://www.axisbank.com/retail/cards/credit-card"]

html = requests.get(axis_url[0])
soup = BeautifulSoup(html.content, 'lxml')

print(soup.select("#ulCreditCard h3"))

输出如下：

[]

我主要关心的是如何在 #ulCreditCard h3 中指定使用选择器小工具获得的 css 选择器标签 soup.select() 函数。

Answer 1

如前所述，您要查找的信息是通过Javascript获得的。它使用略有不同的 URL 来获取包含所有卡片详细信息的 JSON 数据。如果你改用它，你可以很容易地列出所有的卡名而不需要使用 BeautifulSoup。例如：

import requests
import json

axis_url = "https://www.axisbank.com/AjaxService/GetCreditCardsProducts"
data = {"strcategory" : "[]", "strrewardtypes" :"[]"}
r = requests.post(axis_url, data=data)

for entry in json.loads(r.json()[0]):
    print(entry['Name'])

会给你以下卡片：

Axis Bank ACE Credit Card
Axis Bank AURA Credit Card
Privilege Easy Credit Card
Axis Bank Reserve Credit Card
Axis Bank Freecharge Plus Credit Card
IndianOil Axis Bank Credit Card
Axis Bank Magnus Credit Card
Flipkart Axis Bank Credit Card
Axis Bank Freecharge Credit Card
Axis Bank MY Zone Credit Card
Axis Bank Neo Credit Card
Axis Bank Vistara Credit Card
Axis Bank Vistara Signature Credit Card
Axis Bank Vistara Infinite Credit Card
Axis Bank Privilege Credit Card
Miles and More Axis Bank Credit Card
Axis Bank Select Credit Card
Axis Bank Pride Platinum Credit Card
Axis Bank Pride Signature Credit Card
Axis Bank MY Zone Easy Credit Card
Axis Bank Insta Easy Credit Card
Axis Bank Signature Credit Card with Lifestyle Benefits
Platinum Credit Card
Titanium Smart Traveler Credit Card
Axis Bank My Wings Credit Card

如何在美汤和python中指定css选择器？

How to specify css selector in beautiful soup and python?

python

beautifulsoup

web-scraping

python-requests

scrapinghub