清除 Python 中的所有 lru_cache

Question

我在 python 中有函数，这些函数具有 lru_cache 的缓存，例如

 @lru_cache(maxsize=None)
 def my_function():
    ...

虽然我可以单独清除缓存，例如my_function.cache_clear()有什么方法可以一次清除所有函数的缓存？ [我在想，也许有一种方法可以返回所有加载到内存中的函数名称，然后循环遍历它们，清除每个函数的缓存]。

我特别希望将实施作为回退的一部分，以应对我机器上 90% 的内存被使用的情况。

Answer 1

您可以创建一个修改过的装饰器，它也注意到缓存的函数：

cached_functions = []

def clearable_lru_cache(*args, **kwargs):
    def decorator(func):
        func = lru_cache(*args, **kwargs)(func)
        cached_functions.append(func)
        return func

    return decorator

def clear_all_cached_functions():
    for func in cached_functions:
        func.cache_clear()

如果你真的想要，你也可以使用猴子补丁来替换原来的装饰器。

测试：

@clearable_lru_cache()
def foo(x):
    print('foo', x)

@clearable_lru_cache()
def bar(x):
    print('bar', x)

for i in [1, 2]:
    for j in [1, 2]:
        print('Calling functions')
        for k in [1, 2, 3]:
            for f in [foo, bar]:
                f(k)
        print('Functions called - if you saw nothing they were cached')
    print('Clearing cache')
    clear_all_cached_functions()

输出：

Calling functions
foo 1
bar 1
foo 2
bar 2
foo 3
bar 3
Functions called - if you saw nothing they were cached
Calling functions
Functions called - if you saw nothing they were cached
Clearing cache
Calling functions
foo 1
bar 1
foo 2
bar 2
foo 3
bar 3
Functions called - if you saw nothing they were cached
Calling functions
Functions called - if you saw nothing they were cached
Clearing cache

Answer 2

maybe there is a way of returning all function names loaded in memory, and then loop over them clearing the cache from each

是的，这也是可能的：

import functools
import gc

gc.collect()
wrappers = [
    a for a in gc.get_objects() 
    if isinstance(a, functools._lru_cache_wrapper)]

for wrapper in wrappers:
    wrapper.cache_clear()

I'm specifically looking to implement as part of a fall-back, for cases where say 90% of the memory on my machine gets used.

这是您在正常情况下可能不会使用的代码，但对调试很有用。

在我的特殊情况下，我想清除一些 dangling file handles from matplotlib（以便专注于我自己的松散文件句柄），因此无法使用 Alex Hall 的解决方案。

清除 Python 中的所有 lru_cache

Clear all lru_cache in Python

python

caching

lru