Python: 如何发送多个 HTTP 请求并接收响应？

Question

如何以最快的方式发送大约 1000 个请求？我知道您可以使用 grequests:

发送多个请求

urls = [
    'sample.url/1',
    'sample.url/2',
    ...
]
request = (grequests.get(u) for u in urls)
print grequests.map(request)

但是return不是内容。我需要的是获取 json 数据，例如：

request = (grequests.get(u) for u in urls)
content = grequests.json(request)

Answer 1

返回的项目不是内容，但确实包含内容。您可以像这样获取所有内容：

result = grequests.map(request)
content = '\n'.join(r.content for r in result) # raw content
text = '\n'.join(r.text for r in result)       # decoded content

您可以像这样解析 json：

result = grequests.map(request)
json = [r.json() for r in result]

示例程序：

import grequests
import pprint

urls = [
    'http://httpbin.org/user-agent',
    'http://httpbin.org/headers',
    'http://httpbin.org/ip',
]

requests = (grequests.get(u) for u in urls)
responses = grequests.map(requests)

json = [response.json() for response in responses]
pprint.pprint(json)

text = '\n'.join(response.text for response in responses)
print(text)

Python: 如何发送多个 HTTP 请求并接收响应？

Python: How can i send multiple HTTP requests and receive the response?

python

json

http

python-requests

grequests