动态定义函数的编程概念
Programming concept to dynamically define a function
我在计算模型和参考数据集的不同评估度量时遇到问题 - 数据集用于三维 (x,y,t) 和 netcdf 数据。评估措施必须针对四种不同类型的时间聚合进行处理：无聚合、整个期间的聚合、重采样聚合（每月）和分组聚合（每小时加班、每月加班）。
我选择的工作马是 python-xarray，因为它们在 netcdf 处理方面非常灵活。
从简单的分数BIAS开始，我有以下代码：
def BIAS_temporal(self,varns_result=None,aggregtime=None,dim_time=None):
""" Compute the BIAS/ME for a timeseries -> e.g. R^3 --> R^2:
BIAS(x,y) = SUM_{dim_t} ( MOD(x,y,dim_t) - OBS(x,y,dim_t) ) """
def bias(x):
return ( x[namex] - x[namey] )
def biasmean(x):
return ( x[namex] - x[namey] ).mean(dim=coordtime)
#
endresult = xarray.Dataset()
for varnsproof,varnsref,varnsres in zip(self.varns_proof,self.varns_ref,varns_result):
# rename the data variables and combine both datasets for evaluation
namex=varnsres+"_x"; namey=varnsres+"_y"
self.DSref.rename({varnsref : namey },inplace=True)
self.DSproof.rename({varnsproof : namex },inplace=True)
DScomb = xarray.merge([self.DSref,self.DSproof])
coordtime = self.MetricCalcProg.ListDimsToXarray(dim_time[varnsref]) #extract the name of time coordinate
#
if aggregtime == 'fullperiod':
DSnew = biasmean(DScomb).to_dataset(name = varnsres)
...
elif aggregtime == '-':
DSnew = bias(DScomb).to_dataset(name = varnsres)
elif "overperiod" in aggregtime:
grpby_method=self.MetricCalcProg.ConvertAggregKey2Groupby(aggregtime)
DSnew = DScomb.groupby(coordtime+'.'+grpby_method).apply(biasmean)
DSnew = DSnew.to_dataset(name = varnsres)
...
elif "overperiod" not in aggregtime:
resamplefreq=self.MetricCalcProg.ConvertAggregKey2Resample(aggregtime)
DSnew = DScomb.resample(time=resamplefreq,keep_attrs=True).apply(biasmean)
DSnew = DSnew.to_dataset(name = varnsres)
...
#
self.DSref.rename({namey : varnsref },inplace=True)
self.DSproof.rename({namex : varnsproof },inplace=True)
unitsnew=GeneralUtils.safe_access_bib('',KeyError,dicttest=self.DSref[varnsref].attrs,dictkey='units',errhandle=False)
if unitsnew is None: unitsnew='-'
longnew="temporal BIAS of "+str(GeneralUtils.safe_access_bib('', KeyError, dicttest=self.DSref[varnsref].attrs, \
dictkey='long_name',errhandle=False))
self.Update_Attributes(Datasetobj=DSnew,variable=varnsres,stdname=varnsres,units=unitsnew,longname=longnew)
endresult = xarray.merge([endresult,DSnew])
return endresult
如您所见，我需要定义函数 'bias' 和 'biasmean'，以便在使用聚合方法 'aggregtime' 的情况下始终获得正确的结果。定义另一个指标，如 'LinearCorrelation' 代码几乎相同：
def LinCorr_temporal(self,varns_result=None,aggregtime=None,dim_time=None):
""" Compute the linear correlation for a timeseries -> e.g. R^3 --> R^2:
LinCorr(x,y) = ???"""
def correl2dtm(x):
a = ( x[namex] - x[namex].mean(dim=coordtime) ) * ( x[namey] - x[namey].mean(dim=coordtime) ) \
/ x[namex].std(dim=coordtime) / x[namey].std(dim=coordtime)
return a.mean(dim=coordtime)
#
endresult = xarray.Dataset()
for varnsproof,varnsref,varnsres in zip(self.varns_proof,self.varns_ref,varns_result):
# ensure that only time stamps are considerd being present in both datasets (otherwise LC>1)
self.DSproof = self.DSproof[varnsproof].where(self.DSref[varnsref].notnull()).to_dataset(name=varnsproof) #harmonize the nan's between both datasets (independent calculations would destroy lincor), part A
self.DSref = self.DSref[varnsref].where(self.DSproof[varnsproof].notnull()).to_dataset(name=varnsref) #harmonize the nan's between both datasets, part B
# rename the data variables and combine both datasets for evaluation
namex=varnsres+"_x"; namey=varnsres+"_y"
self.DSref.rename({varnsref : namey },inplace=True)
self.DSproof.rename({varnsproof : namex },inplace=True)
DScomb = xarray.merge([self.DSref,self.DSproof])
coordtime = self.MetricCalcProg.ListDimsToXarray(dim_time[varnsref]) #extract the name of time coordinate
#
if aggregtime == 'fullperiod':
DSnew = correl2dtm(DScomb).to_dataset(name = varnsres)
...
elif aggregtime == '-':
print(' The Linear Corr. computation makes no sense for each single time step ')
exit()
elif "overperiod" in aggregtime:
grpby_method=self.MetricCalcProg.ConvertAggregKey2Groupby(aggregtime)
DSnew = DScomb.groupby(coordtime+'.'+grpby_method).apply(correl2dtm)
DSnew = DSnew.to_dataset(name = varnsres)
...
elif "overperiod" not in aggregtime:
resamplefreq=self.MetricCalcProg.ConvertAggregKey2Resample(aggregtime)
DSnew = DScomb.resample(time=resamplefreq,keep_attrs=True).apply(correl2dtm)
DSnew = DSnew.to_dataset(name = varnsres)
...
#
self.DSref.rename({namey : varnsref },inplace=True)
self.DSproof.rename({namex : varnsproof },inplace=True)
unitsnew='1'
longnew="temporal lin. correl. of "+str(GeneralUtils.safe_access_bib('', KeyError, dicttest=self.DSref[varnsref].attrs, \
dictkey='long_name',errhandle=False))
self.Update_Attributes(Datasetobj=DSnew,variable=varnsres,stdname=varnsres,units=unitsnew,longname=longnew)
endresult = xarray.merge([endresult,DSnew])
return endresult
如您所见，与 'BIAS' 计算相比只有细微差别。函数定义不同。聚合时间“-”的异常仅发生在 'LinCorr_temporal' 内，并且 where-statement 仅选择存在 'notnull' 数据的那些网格点。
关于函数 'def bias' 和 'def correl2dtm' 的定义，您是否有结合此处定义的两种方法的建议？我想有一种编程技术我至今还没有想到，即动态定义函数。
函数是 python 中的对象，因此您可以将它们分配给变量。您可以将它们作为函数参数传递：
def bias(x, namex, namey, coordtime=False):
return (x[namex] - x[namey])
def correl2dtm(x, namex, namey, coordtime):
a = (x[namex] - x[namex].mean(dim=coordtime)) * (x[namey] - x[namey].mean(dim=coordtime)) \
/ x[namex].std(dim=coordtime) / x[namey].std(dim=coordtime)
return a.mean(dim=coordtime)
def temporal(self, func, varns_result=None,aggregtime=None,dim_time=None):
...
DSnew = func(DScomb, namex, namey, coordtime).to_dataset(name = varnsres)
...
# call temporal() with bias function (= old BIAS_temporal)
temporal(bias, ...)
# call temporal() with correl2dtm (= old LinCorr_temporal)
temporal(correl2dtm, ...)
我在计算模型和参考数据集的不同评估度量时遇到问题 - 数据集用于三维 (x,y,t) 和 netcdf 数据。评估措施必须针对四种不同类型的时间聚合进行处理：无聚合、整个期间的聚合、重采样聚合（每月）和分组聚合（每小时加班、每月加班）。
我选择的工作马是 python-xarray，因为它们在 netcdf 处理方面非常灵活。
从简单的分数BIAS开始，我有以下代码：
def BIAS_temporal(self,varns_result=None,aggregtime=None,dim_time=None):
""" Compute the BIAS/ME for a timeseries -> e.g. R^3 --> R^2:
BIAS(x,y) = SUM_{dim_t} ( MOD(x,y,dim_t) - OBS(x,y,dim_t) ) """
def bias(x):
return ( x[namex] - x[namey] )
def biasmean(x):
return ( x[namex] - x[namey] ).mean(dim=coordtime)
#
endresult = xarray.Dataset()
for varnsproof,varnsref,varnsres in zip(self.varns_proof,self.varns_ref,varns_result):
# rename the data variables and combine both datasets for evaluation
namex=varnsres+"_x"; namey=varnsres+"_y"
self.DSref.rename({varnsref : namey },inplace=True)
self.DSproof.rename({varnsproof : namex },inplace=True)
DScomb = xarray.merge([self.DSref,self.DSproof])
coordtime = self.MetricCalcProg.ListDimsToXarray(dim_time[varnsref]) #extract the name of time coordinate
#
if aggregtime == 'fullperiod':
DSnew = biasmean(DScomb).to_dataset(name = varnsres)
...
elif aggregtime == '-':
DSnew = bias(DScomb).to_dataset(name = varnsres)
elif "overperiod" in aggregtime:
grpby_method=self.MetricCalcProg.ConvertAggregKey2Groupby(aggregtime)
DSnew = DScomb.groupby(coordtime+'.'+grpby_method).apply(biasmean)
DSnew = DSnew.to_dataset(name = varnsres)
...
elif "overperiod" not in aggregtime:
resamplefreq=self.MetricCalcProg.ConvertAggregKey2Resample(aggregtime)
DSnew = DScomb.resample(time=resamplefreq,keep_attrs=True).apply(biasmean)
DSnew = DSnew.to_dataset(name = varnsres)
...
#
self.DSref.rename({namey : varnsref },inplace=True)
self.DSproof.rename({namex : varnsproof },inplace=True)
unitsnew=GeneralUtils.safe_access_bib('',KeyError,dicttest=self.DSref[varnsref].attrs,dictkey='units',errhandle=False)
if unitsnew is None: unitsnew='-'
longnew="temporal BIAS of "+str(GeneralUtils.safe_access_bib('', KeyError, dicttest=self.DSref[varnsref].attrs, \
dictkey='long_name',errhandle=False))
self.Update_Attributes(Datasetobj=DSnew,variable=varnsres,stdname=varnsres,units=unitsnew,longname=longnew)
endresult = xarray.merge([endresult,DSnew])
return endresult
如您所见，我需要定义函数 'bias' 和 'biasmean'，以便在使用聚合方法 'aggregtime' 的情况下始终获得正确的结果。定义另一个指标，如 'LinearCorrelation' 代码几乎相同：
def LinCorr_temporal(self,varns_result=None,aggregtime=None,dim_time=None):
""" Compute the linear correlation for a timeseries -> e.g. R^3 --> R^2:
LinCorr(x,y) = ???"""
def correl2dtm(x):
a = ( x[namex] - x[namex].mean(dim=coordtime) ) * ( x[namey] - x[namey].mean(dim=coordtime) ) \
/ x[namex].std(dim=coordtime) / x[namey].std(dim=coordtime)
return a.mean(dim=coordtime)
#
endresult = xarray.Dataset()
for varnsproof,varnsref,varnsres in zip(self.varns_proof,self.varns_ref,varns_result):
# ensure that only time stamps are considerd being present in both datasets (otherwise LC>1)
self.DSproof = self.DSproof[varnsproof].where(self.DSref[varnsref].notnull()).to_dataset(name=varnsproof) #harmonize the nan's between both datasets (independent calculations would destroy lincor), part A
self.DSref = self.DSref[varnsref].where(self.DSproof[varnsproof].notnull()).to_dataset(name=varnsref) #harmonize the nan's between both datasets, part B
# rename the data variables and combine both datasets for evaluation
namex=varnsres+"_x"; namey=varnsres+"_y"
self.DSref.rename({varnsref : namey },inplace=True)
self.DSproof.rename({varnsproof : namex },inplace=True)
DScomb = xarray.merge([self.DSref,self.DSproof])
coordtime = self.MetricCalcProg.ListDimsToXarray(dim_time[varnsref]) #extract the name of time coordinate
#
if aggregtime == 'fullperiod':
DSnew = correl2dtm(DScomb).to_dataset(name = varnsres)
...
elif aggregtime == '-':
print(' The Linear Corr. computation makes no sense for each single time step ')
exit()
elif "overperiod" in aggregtime:
grpby_method=self.MetricCalcProg.ConvertAggregKey2Groupby(aggregtime)
DSnew = DScomb.groupby(coordtime+'.'+grpby_method).apply(correl2dtm)
DSnew = DSnew.to_dataset(name = varnsres)
...
elif "overperiod" not in aggregtime:
resamplefreq=self.MetricCalcProg.ConvertAggregKey2Resample(aggregtime)
DSnew = DScomb.resample(time=resamplefreq,keep_attrs=True).apply(correl2dtm)
DSnew = DSnew.to_dataset(name = varnsres)
...
#
self.DSref.rename({namey : varnsref },inplace=True)
self.DSproof.rename({namex : varnsproof },inplace=True)
unitsnew='1'
longnew="temporal lin. correl. of "+str(GeneralUtils.safe_access_bib('', KeyError, dicttest=self.DSref[varnsref].attrs, \
dictkey='long_name',errhandle=False))
self.Update_Attributes(Datasetobj=DSnew,variable=varnsres,stdname=varnsres,units=unitsnew,longname=longnew)
endresult = xarray.merge([endresult,DSnew])
return endresult
如您所见，与 'BIAS' 计算相比只有细微差别。函数定义不同。聚合时间“-”的异常仅发生在 'LinCorr_temporal' 内，并且 where-statement 仅选择存在 'notnull' 数据的那些网格点。
关于函数 'def bias' 和 'def correl2dtm' 的定义，您是否有结合此处定义的两种方法的建议？我想有一种编程技术我至今还没有想到，即动态定义函数。
函数是 python 中的对象，因此您可以将它们分配给变量。您可以将它们作为函数参数传递：
def bias(x, namex, namey, coordtime=False):
return (x[namex] - x[namey])
def correl2dtm(x, namex, namey, coordtime):
a = (x[namex] - x[namex].mean(dim=coordtime)) * (x[namey] - x[namey].mean(dim=coordtime)) \
/ x[namex].std(dim=coordtime) / x[namey].std(dim=coordtime)
return a.mean(dim=coordtime)
def temporal(self, func, varns_result=None,aggregtime=None,dim_time=None):
...
DSnew = func(DScomb, namex, namey, coordtime).to_dataset(name = varnsres)
...
# call temporal() with bias function (= old BIAS_temporal)
temporal(bias, ...)
# call temporal() with correl2dtm (= old LinCorr_temporal)
temporal(correl2dtm, ...)