我可以从此 XHR 请求中提取 JSON 数据吗？

Question

我是一个初学者，试图在此站点的部分进行网络抓取 https://www.csgoroll.com/en/withdraw/csgo/p2p I'm trying to extract the prices and name of each item in the request called "graphql?operationName=TradeList&variables=" soley using the requests library in Python but i'm unsure of how to do it. I've done some research on this and it lead me to use an app called Postman and if I copy the cURL to it, it doesn't return the JSON data. Further research it looks like it uses graphql but when I connect to https://api.csgoroll.com/ 我认为它用于查询数据的游乐场说无法访问服务器。所以我想知道是否可以仅使用 Python 中的请求库来提取数据，如果可以的话如何？

Answer 1

它需要 header User-Agent 来自真正的网络浏览器或至少 'Mozilla/5.0'.

requests 默认使用 python/3.x 这样的东西，所以服务器要求解析 Captcha.

import requests

url = 'https://api.csgoroll.com/graphql'

headers = {
    'User-Agent': 'Mozilla/5.0',
#    'Accept': 'application/json, text/plain, */*',
}

params = {
    'operationName': 'TradeList',
    'variables': '{"first":50,"orderBy":"TOTAL_VALUE_DESC","status":"LISTED","steamAppName":"CSGO"}',
    'extensions': '{"persistedQuery":{"version":1,"sha256Hash":"87239fc5fa143cf0437964a20937aa558145cc8385eae48ca8734cb16abfd266"}}',
}

r = requests.get(url, headers=headers, params=params)
#print(r.text)

data = r.json()
trades = data['data']['trades']['edges']

for trade in trades:
    #print(item)
    for item in trade['node']['tradeItems']:
        print(item['marketName'])

结果：

★ Sport Gloves | Pandora's Box (Factory New)
★ StatTrak™ M9 Bayonet | Crimson Web (Factory New)
★ StatTrak™ Karambit | Crimson Web (Factory New)
★ Butterfly Knife | Sapphire (Factory New)
★ Butterfly Knife | Sapphire (Factory New)
Sticker | HellRaisers (Holo) | Katowice 2014
AWP | Dragon Lore (Factory New)
★ StatTrak™ Karambit | Emerald (Factory New)
★ Butterfly Knife | Ruby (Factory New)
★ M9 Bayonet | Emerald (Minimal Wear)
StatTrak™ M4A4 | Howl (Field-Tested)
★ Paracord Knife | Crimson Web (Factory New)
★ Paracord Knife | Crimson Web (Factory New)
★ Moto Gloves | Smoke Out (Factory New)
M4A1-S | Welcome to the Jungle (Factory New)
AWP | Dragon Lore (Battle-Scarred)
AWP | Desert Hydra (Factory New)
★ Sport Gloves | Slingshot (Minimal Wear)
★ Sport Gloves | Slingshot (Minimal Wear)
★ Classic Knife | Crimson Web (Factory New)
Sticker | compLexity Gaming (Holo) | Katowice 2014
AK-47 | Wild Lotus (Battle-Scarred)
AK-47 | Wild Lotus (Battle-Scarred)
StatTrak™ P90 | Emerald Dragon (Factory New)
StatTrak™ P90 | Emerald Dragon (Factory New)
★ StatTrak™ Skeleton Knife | Case Hardened (Minimal Wear)
★ Hand Wraps | Constrictor (Factory New)
Sticker | ESL Wolf (Foil) | Katowice 2014
Sticker | ESL Skull (Foil) | Katowice 2014
★ StatTrak™ Gut Knife | Ultraviolet (Battle-Scarred)
MP9 | Dark Age (Minimal Wear)
Souvenir P90 | Fallout Warning (Factory New)
Souvenir P250 | Gunsmoke (Factory New)
Sticker | Aerial (Gold) | Katowice 2019
StatTrak™ M4A4 | In Living Color (Battle-Scarred)
StatTrak™ M4A4 | In Living Color (Battle-Scarred)
StatTrak™ M4A4 | In Living Color (Battle-Scarred)
Sticker | TYLOO (Gold) | 2020 RMR
Sticker | yuurih (Gold) | Berlin 2019
Sticker | TYLOO (Gold) | 2020 RMR
Sticker | Natus Vincere (Gold) | 2020 RMR
Sticker | TYLOO (Gold) | 2020 RMR
Sticker | TYLOO (Gold) | 2020 RMR
M4A4 | Neo-Noir (Field-Tested)
M4A4 | Neo-Noir (Field-Tested)
StatTrak™ AWP | Atheris (Field-Tested)
CZ75-Auto | Tuxedo (Minimal Wear)
Sealed Graffiti | G2 Esports | Atlanta 2017
StatTrak™ Desert Eagle | Bronze Deco (Minimal Wear)
Sealed Graffiti | Astralis | Atlanta 2017

我可以从此 XHR 请求中提取 JSON 数据吗？

Can I extract JSON data from this XHR request?

python

ajax

xmlhttprequest

postman

graphql