Python

Question

正如标题所说 - 我遇到了异步问题。我想要实现的是在每个功能下写的。可悲的是，在这种代码状态下，我遇到了错误：

TypeError: object StreamReader can't be used in 'await' expression 最后 RuntimeError: Event loop is closed

我在谷歌上搜索了一段时间，并没有真正找到适合我的事情的解决方案。有没有人可以帮助我并弄清楚我做错了什么？我可以在一个异步函数中使用 2x async with .. 吗？

谢谢！

 
def load_file(file_path):
    with open(file_path, "r") as f:
        content = f.readlines()
        content = [a.strip() for a in content]
    return content

### --> Getting list of urls

async def task(session, item, urls):
    async with session.get(item) as resp:
        image_bytes = BytesIO(await resp.content)

### --> Downloading the image and getting image bytes

    async with session.post(
        TORCH_URL, data=image_bytes, headers={"authorization": TOKEN}
    ) as resp:
        response = await resp.json()
        print(response)

### --> Sending the image bytes to an API and getting a little json file as a response 

async def asyncmain(urls, path, content):

    tasks = []
    async with aiohttp.ClientSession() as session:
        tasks = [task(session, url, urls) for url in content]
        await asyncio.gather(*tasks)

### --> Gathering the tasks with .gather()

@click.command()
@click.option("--urls", "-u", is_flag=True, help="Use this if you have urls")
@click.option(
    "--path",
    "-p",
    help="Path to file with variant IDs, can be combined with -u (having urls in file)",
)
def main(urls, path):
    tasks = []
    content = load_file(path)
    asyncio.run(asyncmain(urls, path, content), debug=True)

### --> Fire asyncio.run with some params

if __name__ == "__main__":
    main()

Answer 1

您的问题是下面的 resp.content 变量使用不当。

async def task(session, item, urls):
    async with session.get(item) as resp:
        image_bytes = BytesIO(await resp.content)

请参阅 aiohttp 的 streaming response content 文档。

While methods read(), json() and text() are very convenient you should use them carefully. All these methods load the whole response in memory. For example if you want to download several gigabyte sized files, these methods will load all the data in memory. Instead you can use the content attribute. It is an instance of the aiohttp.StreamReader class. The gzip and deflate transfer-encodings are automatically decoded for you:
async with session.get('https://api.github.com/events') as resp:
    await resp.content.read(10)

你可以

(a) 下载块并将它们存储到磁盘中，或者
(b) 如果二进制文件不大并且可以存储到内存中——这似乎是基于您对 BytesIO 的使用——使用 io.BytesIO(await resp.read())（请参阅 binary response content.)

Python - 迷失在异步中

Python - getting lost around async

async-await

python-asyncio

aiohttp