python 网页抓取 soup.find 给空

Question

为什么 a 和 prices 是空的，即使 soup 有数据也不从 soup 中提取值

'''

import requests
from bs4 import BeautifulSoup
import csv
import re

r = requests.get('https://www.daraz.pk/catalog/?q=iphone&_keyori=ss&from=input&spm=a2a0e.home.search.go.35e34937c3qzgp')
soup = BeautifulSoup(r.text, 'html.parser')
products=[] #List to store name of the product
prices=[] #List to store price of the product
ratings=[] #List to store rating of the product

for item in soup.find_all(class_='c3gUW0'):
    prices.append(item.text)

a=soup.find(class_='c3gUW0')
a**strong text**

'''

Answer 1

该网站是动态加载的，因此 requests 模块不支持它。但是，数据以 JSON 格式嵌入网站，您可以使用内置的 re (regex) module, and convert it to a Python dictionary (dict) using the built-in json 模块查找数据。

例如打印名称和价格：

import re
import json
import requests
from bs4 import BeautifulSoup

URL = "https://www.daraz.pk/catalog/?q=iphone&_keyori=ss&from=input&spm=a2a0e.home.search.go.35e34937c3qzgp"
soup = BeautifulSoup(requests.get(URL).content, "html.parser")

data = json.loads(re.search(r"window.pageData=({.*})", str(soup)).group(1))
for d in data["mods"]["listItems"]:
    print("Description:", d["name"])
    print("Price:", d["priceShow"])
    print('-' * 80)

输出（截断）：

Description: Apple iPhone 12 - 64GB - PTA Approved
Price: Rs. 215,000
--------------------------------------------------------------------------------
Description: iPhone 12 - 128GB - PTA Approved
Price: Rs. 230,000
--------------------------------------------------------------------------------
Description: iPhone 8 Plus - 5.5" Display - 3GB RAM + 64/256GB ROM - Phone Only
Price: Rs. 66,999
--------------------------------------------------------------------------------

python 网页抓取 soup.find 给空

python web scraping soup.find give empty

python

web

beautifulsoup

screen-scraping