收到响应后 aiohttp 检查请求 headers

Question

这是问题所在：我有一个授权列表 header 需要检查。为此，我使用 aiohhtp

def make_tasks(session, proxies, unchecked_headers):
    tasks = list()
    for unchecked_header in unchecked_headers:
        current_proxy = proxies.pop()
        headers['authorization'] = unchecked_header
        t = session.get(url, proxy=current_proxy, headers=headers)
        tasks.append(t)
    return tasks


async def check_headers(proxies, unchecked_headers):
    async with aiohttp.ClientSession() as s:
        tasks = make_tasks(s, proxies, unchecked_headers)
        results = await asyncio.gather(*tasks)
        for result in results:
            ...

现在，根据响应代码，我需要记录一些有关该授权的信息 header。问题是它在响应中没有提到实际的 header。因此，我得到了所有响应，但我不知道哪个响应对应于哪个 header，因为它们是异步的。

我环顾四周，没有找到一种方法来检查最初在 ClientResponse object 上发送了哪些 header。我可以在这里做什么？

Answer 1

您可以使用 asyncio.as_completed + 包装器 session.get 到 return 响应以及您想要的任何其他数据。例如：

import aiohttp
import asyncio


url = "https://httpbin.org/get"

unchecked_headers = [
    "A",
    "B",
    "C",
]
proxies = ["Proxy1", "Proxy2", "Proxy3"]
headers = {}


def make_tasks(session, proxies, unchecked_headers):
    async def _wrapper(t, *args):
        response = await t
        return response, *args

    tasks = list()
    for unchecked_header in unchecked_headers:
        current_proxy = proxies.pop()
        headers["authorization"] = unchecked_header

        # I commented this out because I don't have access to proxy:
        # t = session.get(url, proxy=current_proxy, headers=headers)

        t = session.get(url, headers=headers)
        tasks.append(_wrapper(t, current_proxy, unchecked_header))
    return tasks


async def check_headers(proxies, unchecked_headers):
    async with aiohttp.ClientSession() as s:
        for task in asyncio.as_completed(
            make_tasks(s, proxies, unchecked_headers)
        ):
            response, proxy, header = await task
            print(response.url, proxy, header)


async def main():
    await check_headers(proxies, unchecked_headers)


if __name__ == "__main__":
    asyncio.run(main())

打印：

https://httpbin.org/get Proxy3 A
https://httpbin.org/get Proxy1 C
https://httpbin.org/get Proxy2 B

收到响应后 aiohttp 检查请求 headers

aiohttp check request headers after receiving response

python

python-requests

header

aiohttp