requests_cache 是否会在更新信息时自动更新缓存？

Question

我有一个非常不可靠的 API，我请求使用 Python。我一直在考虑使用 requests_cache 并将 expire_after 设置为 999999999999 就像我看到其他人所做的那样。唯一的问题是，我不知道 API 何时再次工作，是否更新了数据。如果requests_cache 将自动自动更新并删除旧条目。

我已经尝试阅读文档，但我真的无法在任何地方看到它。

Answer 1

requests_cache 不会更新，直到 expire_after 时间过去。在这种情况下，它不会检测到您的 API 已恢复工作状态。

我注意到该项目已经添加了我过去实现的选项；您现在可以在配置缓存时设置 old_data_on_error 选项；请参阅 CachedSession documentation:

old_data_on_error – If True it will return expired cached response if update fails.

如果后端更新失败，它会重用现有的缓存数据，而不是删除该数据。

过去，我创建了自己的 requests_cache 会话设置（加上小补丁），如果后端给出 500 错误或超时（使用短超时），它将重用超过 expire_after 的缓存值) 来处理有问题的 API 层，而不是依赖 expire_after:

import logging

from datetime import (
    datetime,
    timedelta
)
from requests.exceptions import (
    ConnectionError,
    Timeout,
)
from requests_cache.core import (
    dispatch_hook,
    CachedSession,
)

log = logging.getLogger(__name__)
# Stop logging from complaining if no logging has been configured.
log.addHandler(logging.NullHandler())


class FallbackCachedSession(CachedSession):
    """Cached session that'll reuse expired cache data on timeouts

    This allows survival in case the backend is down, living of stale
    data until it comes back.

    """

    def send(self, request, **kwargs):
        # this *bypasses* CachedSession.send; we want to call the method
        # CachedSession.send() would have delegated to!
        session_send = super(CachedSession, self).send
        if (self._is_cache_disabled or
                request.method not in self._cache_allowable_methods):
            response = session_send(request, **kwargs)
            response.from_cache = False
            return response

        cache_key = self.cache.create_key(request)

        def send_request_and_cache_response(stale=None):
            try:
                response = session_send(request, **kwargs)
            except (Timeout, ConnectionError):
                if stale is None:
                    raise
                log.warning('No response received, reusing stale response for '
                            '%s', request.url)
                return stale

            if stale is not None and response.status_code == 500:
                log.warning('Response gave 500 error, reusing stale response '
                            'for %s', request.url)
                return stale

            if response.status_code in self._cache_allowable_codes:
                self.cache.save_response(cache_key, response)
            response.from_cache = False
            return response

        response, timestamp = self.cache.get_response_and_time(cache_key)
        if response is None:
            return send_request_and_cache_response()

        if self._cache_expire_after is not None:
            is_expired = datetime.utcnow() - timestamp > self._cache_expire_after
            if is_expired:
                self.cache.delete(cache_key)
                # try and get a fresh response, but if that fails reuse the
                # stale one
                return send_request_and_cache_response(stale=response)

        # dispatch hook here, because we've removed it before pickling
        response.from_cache = True
        response = dispatch_hook('response', request.hooks, response, **kwargs)
        return response


def basecache_delete(self, key):
    # We don't really delete; we instead set the timestamp to
    # datetime.min. This way we can re-use stale values if the backend
    # fails
    try:
        if key not in self.responses:
            key = self.keys_map[key]
        self.responses[key] = self.responses[key][0], datetime.min
    except KeyError:
        return

from requests_cache.backends.base import BaseCache
BaseCache.delete = basecache_delete

上述 CachedSession 的子类绕过 original send() method 直接转到原始 requests.Session.send() 方法，即使超时已过 return 现有缓存值但是后端失败了。禁用删除以将超时值设置为 0，因此如果新请求失败，我们仍然可以重用旧值。

使用 FallbackCachedSession 而不是常规 CachedSession 对象。

如果您想使用 requests_cache.install_cache()，请确保将 FallbackCachedSession 传递给 session_factory 关键字参数中的该函数：

import requests_cache

requests_cache.install_cache(
    'cache_name', backend='some_backend', expire_after=180,
    session_factory=FallbackCachedSession)

我的方法比 requests_cache 在我把上面的内容组合在一起一段时间后实施的方法更全面；即使您之前明确将其标记为已删除，我的版本也会退回到陈旧的响应。

Answer 2

尝试做类似的事情：

class UnreliableAPIClient:
  def __init__(self):
    self.some_api_method_cached = {} # we will store results here

  def some_api_method(self, param1, param2)
    params_hash = "{0}-{1}".format(param1, param2) # need to identify input
    try:
      result = do_call_some_api_method_with_fail_probability(param1, param2)
      self.some_api_method_cached[params_hash] = result # save result
    except:
      result = self.some_api_method_cached[params_hash] # resort to cached result
      if result is None:
        raise # reraise exception if nothing cached
    return result

当然你可以用它制作简单的装饰器，由你决定 - http://www.artima.com/weblogs/viewpost.jsp?thread=240808

requests_cache 是否会在更新信息时自动更新缓存？

Does requests_cache automatically update cache on update of info?

python

caching

python-requests