为什么我不能在 gdal 模块中等待 readasarray 方法？

Question

我正在尝试将多个远程图像读入 python 并将这些图像读取为 numpyarray，我尝试考虑使用异步来提升我的工作流程，但我收到类似 this:type 的错误： object numpy.ndarray can't be used in 'await' expression',我想知道是不是因为readasarray方法不是异步的，所以如果我必须让它异步，我将不得不通过我的方法重写这个方法own.here 是我的一些代码：

async def taskIO_1():

    in_ds = gdal.Open(a[0])
    data1 = await in_ds.GetRasterBand(1).ReadAsArray()

async def taskIO_2():

    in_ds = gdal.Open(a[1])
    data2 = await in_ds.GetRasterBand(1).ReadAsArray()

async def main():

    tasks = [taskIO_1(), taskIO_2()]
    done,pending = await asyncio.wait(tasks)
    for r in done:
        print(r.result())

if __name__ == '__main__':
    start = time.time()
    loop = asyncio.get_event_loop()
    try:
        loop.run_until_complete(main())
    finally:
        loop.close()
    print(float(time.time()-start))

Answer 1

你的想法是正确的：一般来说，库函数以同步（阻塞）方式执行，除非库被明确编写为支持异步执行（例如通过使用非阻塞I/O），例如作为 aiofiles or aiohttp.

要使用您希望异步执行的同步调用，您可以使用 loop.run_in_executor。这只是将计算卸载到一个单独的线程或进程中并将其包装起来，使其表现得像一个协程。示例显示 here:

import asyncio
import concurrent.futures

def blocking_io():
    # File operations (such as logging) can block the
    # event loop: run them in a thread pool.
    with open('/dev/urandom', 'rb') as f:
        return f.read(100)

def cpu_bound():
    # CPU-bound operations will block the event loop:
    # in general it is preferable to run them in a
    # process pool.
    return sum(i * i for i in range(10 ** 7))

async def main():
    loop = asyncio.get_running_loop()

    ## Options:

    # 1. Run in the default loop's executor:
    result = await loop.run_in_executor(
        None, blocking_io)
    print('default thread pool', result)

    # 2. Run in a custom thread pool:
    with concurrent.futures.ThreadPoolExecutor() as pool:
        result = await loop.run_in_executor(
            pool, blocking_io)
        print('custom thread pool', result)

    # 3. Run in a custom process pool:
    with concurrent.futures.ProcessPoolExecutor() as pool:
        result = await loop.run_in_executor(
            pool, cpu_bound)
        print('custom process pool', result)

asyncio.run(main())

但是，如果您的应用程序没有使用任何真正的异步功能，您最好直接使用 concurrent.futures 池并以这种方式实现并发。

为什么我不能在 gdal 模块中等待 readasarray 方法？

why can't i await readasarray method in gdal module?

python

numpy

gdal

python-asyncio