缓存异步函数时忽略参数

Question

所以在aiohttp中建议重用ClientSession来避免开销。

所以说我有这样的东西：

async def get_id_from_url(session, url):
    async with session.get(url) as response:
        return (await response.json())['id'] if response.ok else ''


async def get_ids(urls: List[str]):
    async with aiohttp.ClientSession() as session:
        ids = await asyncio.gather(*[get_id_from_url(session, url) for url in urls])
        print(ids)

我想缓存 get_id_from_url 函数，我看了很多 questions/packages 与缓存异步函数相关的内容：aiocache 还有这个 answer.

但是有一个问题，我不希望缓存依赖于会话参数，我只希望它依赖于 url 参数。

我该怎么做？

Answer 1

经过大量研究后，我找到了 cachetools 库和这个 pull request。

我刚刚从那里得到了解决方案：这是对 cachetools 的 asyncio 支持的实现。并且 cachetools 已经允许您传递一个关键参数，该参数允许您选择函数的哪些参数将参与散列。

import functools
import inspect
import logging

from cachetools.keys import hashkey

logger = logging.getLogger(__name__)


class NullContext(object):
    """A class for noop context managers."""

    def __enter__(self):
        """Return ``self`` upon entering the runtime context."""
        return self

    def __exit__(self, exc_type, exc_value, traceback):
        """Raise any exception triggered within the runtime context."""

    async def __aenter__(self):
        """Return ``self`` upon entering the runtime context."""
        return self

    async def __aexit__(self, exc_type, exc_value, traceback):
        """Raise any exception triggered within the runtime context."""


def aiocached(cache, key=hashkey, lock=None):
    """Decorator to wrap a function or a coroutine with a memoizing callable.

    When ``lock`` is provided for a standard function, it's expected to
    implement ``__enter__`` and ``__exit__`` that will be used to lock
    the cache when it gets updated. If it wraps a coroutine, ``lock``
    must implement ``__aenter__`` and ``__aexit__``.
    """
    lock = lock or NullContext()

    def decorator(func):
        if not inspect.iscoroutinefunction(func):
            raise RuntimeError('Use aiocached only with async functions')

        async def wrapper(*args, **kwargs):
            fk = key(*args, **kwargs)
            async with lock:
                fval = cache.get(fk)
            # cache hit
            if fval is not None:
                return fval
            # cache miss
            fval = await func(*args, **kwargs)
            try:
                async with lock:
                    cache[fk] = fval
            except ValueError:
                logger.debug('Failed to cache {0}'.format(fval))
            return fval
        return functools.wraps(func)(wrapper)
    return decorator

现在您可以像 cached

一样使用 aiocached 装饰器

缓存异步函数时忽略参数

Ignoring a parameter while caching asyncio functions

python

asynchronous

caching

aiohttp