python 3.x joblib简单保存功能

Question

我正在尝试创建一个简单的 joblib 函数，它将计算表达式并对结果进行 pickle，同时检查 pickle 文件是否存在。但是当我把这个函数放在其他文件中并在将文件路径添加到 sys.path 后导入函数时。我收到错误。

from pathlib import Path
import joblib as jl    
def saveobj(filename, expression_obj,ignore_file = False):
    fname = Path(filename)
    if fname.exists() and not ignore_file:
        obj = jl.load(filename)
    else:
        obj = eval(expression_obj)
        jl.dump(obj,fname,compress = True)        
    return obj

调用示例：

rf_clf = saveobj(file, "rnd_cv.fit(X_train, np.ravel(y_train))", ignore_file=True)

错误：

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-11-02c2cae43c5d> in <module>
      1 file = Path("rf.pickle")
----> 2 rf_clf = saveobj(file, "rnd_cv.fit(X_train, np.ravel(y_train))", ignore_file=True)

~/Dropbox/myfnlib/util_funs.py in saveobj(filename, expression_obj, ignore_file)
     37         obj = jl.load(filename)
     38     else:
---> 39         obj = eval(expression_obj)
     40         jl.dump(obj,fname,compress = True)
     41     return obj

~/Dropbox/myfnlib/util_funs.py in <module>

NameError: name 'rnd_cv' is not defined

我想，python 需要在本地评估函数，但由于该范围内不存在对象，因此会引发此错误。有没有更好的方法来做到这一点。我需要反复做这个，这就是为什么一个函数。非常感谢您的帮助。

Answer 1

你可以查看eval的文档：

Help on built-in function eval in module builtins:

eval(source, globals=None, locals=None, /)

Evaluate the given source in the context of globals and locals.

The source may be a string representing a Python expression
or a code object as returned by compile().
The globals must be a dictionary and locals can be any mapping,
defaulting to the current globals and locals.
If only globals is given, locals defaults to it.

它有全局变量和局部变量的参数。因此，对于您的情况，您可以：

from pathlib import Path
import joblib as jl    
def saveobj(filename, expression_obj,global,local,ignore_file = False):
    fname = Path(filename)
    if fname.exists() and not ignore_file:
        obj = jl.load(filename)
    else:
        obj = eval(expression_obj, global, local)
        jl.dump(obj,fname,compress = True)        
    return obj

代码可以改成：

rf_clf = saveobj(file, "rnd_cv.fit(X_train, np.ravel(y_train))", globals(), locals(), ignore_file=True)

Answer 2

我正要post回答我自己的问题，突然看到@youkaichao 的回答。非常感谢。给猫剥皮的另一种方法：（尽管仅限于关键字参数）

def saveobj(filename,func, ignore_file = False, **kwargs):
    fname = Path(filename)
    if fname.exists() and not ignore_file:
        obj = jl.load(filename)
    else:
        obj = func(**kwargs)
        jl.dump(obj,fname,compress = True)        
    return obj

更改调用：

file = Path("rf.pickle")
rf_clf = saveobj(file, rnd_cv.fit, ignore_file=False, X=X_train, y= np.ravel(y_train))

虽然，我还是很想知道，哪个更好。

python 3.x joblib简单保存功能

python 3.x joblib simple save function

python

pickle

joblib