使用时间减去 `n` 的元标签问题后的趋势,使用 `n * obj.freq`

trend following using meta label issue with time ubtracting `n`, use `n * obj.freq`

我正在尝试根据资产管理簿中的标签实施趋势。我找到了我想要实现的以下代码,但是我收到了一个错误

TypeError:不再支持 Addition/subtraction 带有时间戳的整数和整数数组。使用 n * obj.freq

而不是 adding/subtracting n
!pip install yfinance
!pip install mplfinance
import yfinance as yf
import mplfinance as mpf
import numpy as np 
import pandas as pd 

# get the data from yfiance 
df=yf.download('BTC-USD',start='2008-01-04',end='2021-06-3',interval='1d')

#code snippet 5.1
# Fit linear regression on close
# Return the t-statistic for a given parameter estimate.
def tValLinR(close):
    #tValue from a linear trend
    x = np.ones((close.shape[0],2))
    x[:,1] = np.arange(close.shape[0])
    ols = sm1.OLS(close, x).fit()
    return ols.tvalues[1]

    #code snippet 5.2
'''
 #search for the maximum absolutet-value. To identify the trend 
  #  - molecule - index of observations we wish to labels. 
   # - close - which is the time series of x_t
   # - span - is the set of values of L (look forward period) that the algorithm will #try (window_size)
#    The L that maximizes |tHat_B_1| (t-value) is choosen - which is the look-forward #period 
#    with the most significant trend. (optimization)
'''
def getBinsFromTrend(molecule, close, span):
    
    #Derive labels from the sign of t-value of trend line
    #output includes:
     # - t1: End time for the identified trend
     # - tVal: t-value associated with the estimated trend coefficient
      #- bin: Sign of the trend (1,0,-1)
    #The t-statistics for each tick has a different look-back window.
      
    #- idx start time in look-forward window
    #- dt1 stop time in look-forward window
    #- df1 is the look-forward window (window-size)
    #- iloc ? 
    
    out = pd.DataFrame(index=molecule, columns=['t1', 'tVal', 'bin', 'windowSize'])
    hrzns = range(*span)
    windowSize = span[1] - span[0]
    maxWindow = span[1]-1
    minWindow = span[0]
    for idx in close.index:
        idx += maxWindow
        if idx >= len(close):
            break
        df_tval = pd.Series(dtype='float64')
        iloc0 = close.index.get_loc(idx)
        if iloc0+max(hrzns) > close.shape[0]:
            continue
        for hrzn in hrzns:
            dt1 = close.index[iloc0-hrzn+1]
            df1 = close.loc[dt1:idx]
            df_tval.loc[dt1] = tValLinR(df1.values) #calculates t-statistics on period
        dt1 = df_tval.replace([-np.inf, np.inf, np.nan], 0).abs().idxmax() #get largest t-statistics calculated over span period

        print(df_tval.index[-1])
        print(dt1)
        print(abs(df_tval.values).argmax() + minWindow)
        out.loc[idx, ['t1', 'tVal', 'bin', 'windowSize']] = df_tval.index[-1], df_tval[dt1], np.sign(df_tval[dt1]), abs(df_tval.values).argmax() + minWindow #prevent leakage
    out['t1'] = pd.to_datetime(out['t1'])
    out['bin'] = pd.to_numeric(out['bin'], downcast='signed')

    #deal with massive t-Value outliers - they dont provide more confidence and they ruin the scatter plot
    tValueVariance = out['tVal'].values.var()
    tMax = 20
    if tValueVariance < tMax:
        tMax = tValueVariance

    out.loc[out['tVal'] > tMax, 'tVal'] = tMax #cutoff tValues > 20
    out.loc[out['tVal'] < (-1)*tMax, 'tVal'] = (-1)*tMax #cutoff tValues < -20
    return out.dropna(subset=['bin'])

if __name__ == '__main__':
    #snippet 5.3
    idx_range_from = 3
    idx_range_to = 10
    df1 = getBinsFromTrend(df.index, df['Close'], [idx_range_from,idx_range_to,1]) #[3,10,1] = range(3,10) This is the issue 
    tValues = df1['tVal'].values #tVal

    doNormalize = False
    #normalise t-values to -1, 1
    if doNormalize:
        np.min(tValues)
        minusArgs = [i for i in range(0, len(tValues)) if tValues[i] < 0]
        tValues[minusArgs] = tValues[minusArgs] / (np.min(tValues)*(-1.0))

        plus_one = [i for i in range(0, len(tValues)) if tValues[i] > 0]
        tValues[plus_one] = tValues[plus_one] / np.max(tValues)

    #+(idx_range_to-idx_range_from+1)
    plt.scatter(df1.index, df0.loc[df1.index].values, c=tValues, cmap='viridis') #df1['tVal'].values, cmap='viridis')
    plt.plot(df0.index, df0.values, color='gray')
    plt.colorbar()
    plt.show()
    plt.savefig('fig5.2.png')
    plt.clf()
    plt.df['Close']()
    plt.scatter(df1.index, df0.loc[df1.index].values, c=df1['bin'].values,  cmap='vipridis')

    #Test methods
    ols_tvalue = tValLinR( np.array([3.0, 3.5, 4.0]) )

您“发现”的代码至少有 几个 个问题(不仅仅是您发布的那个问题)。

在我讨论一些问题之前,让我说以下,我很可能是错误的,但根据我的经验和你问题的措辞方式(以及你是“想要实现“您“找到”的代码”)在我看来,您的编码和调试经验很少。

Whosebug 不是一个让别人调试你的代码的地方。也就是说,我将尝试向您介绍我在尝试弄清楚这段代码的运行过程中采取的一些步骤,然后可能会为您指出一些可以学习相同技能的资源。

第 1 步:

我把你发布的代码,copy/pasted 放到了一个我命名为 so68871906.py 的文件中;然后我注释掉了顶部安装 yfinancemplfinance 的两行,因为我不想每次 运行 代码时都尝试安装它们;相反,我会在 运行 编写代码之前安装它们一次。

然后我 运行 具有以下结果的代码(类似于您发布的内容)...

dino@DINO:~/code/mplfinance/examples/scratch_pad/issues$ python so68871906.py
[*********************100%***********************]  1 of 1 completed
Traceback (most recent call last):
  File "so68871906.py", line 84, in <module>
    df1 = getBinsFromTrend(df.index, df['Close'], [idx_range_from,idx_range_to,1]) #[3,10,1] = range(3,10) This is the issue
  File "so68871906.py", line 50, in getBinsFromTrend
    idx += maxWindow
  File "pandas/_libs/tslibs/timestamps.pyx", line 310, in pandas._libs.tslibs.timestamps._Timestamp.__add__
TypeError: Addition/subtraction of integers and integer-arrays with Timestamp is no longer supported.  Instead of adding/subtracting `n`, use `n * obj.freq`

成功调试的关键,尤其是在 python 中,是要认识到 Traceback 为您提供了许多非常重要的信息。 你只需要非常仔细地阅读它。 在上面的例子中,Traceback 告诉我:

  1. 问题在于这行代码:idx += maxWindow。这行代码添加 idx + maxWindow 并将结果重新分配回 idx
  2. 错误是 TypeError,它告诉我变量的 types 有问题。由于该行代码中有两个变量(idxmaxWindow),人们可能会猜测其中一个或两个变量是错误的 type 或者不兼容代码试图对变量做什么。
  3. 基于错误消息“Addition/subtraction of integers and integer-arrays with Timestamp is no longer supported”,以及我们正在添加 idxmaxWindow 的事实,你可以猜到其中一个变量是integerinteger-array类型,而另一个是Timestamp.
  4. 类型
  5. 您可以通过在错误发生之前添加打印语句来验证类型。代码如下所示:
    maxWindow = span[1]-1
    minWindow = span[0]
    for idx in close.index:
        print('type(idx)=',type(idx))
        print('type(maxWindow)=',type(maxWindow))
        idx += maxWindow

现在输出如下所示:

dino@DINO:~/code/mplfinance/examples/scratch_pad/issues$ python so68871906.py
[*********************100%***********************]  1 of 1 completed
type(idx)= <class 'pandas._libs.tslibs.timestamps.Timestamp'>
type(maxWindow)= <class 'int'>
Traceback (most recent call last):
  File "so68871906.py", line 86, in <module>
    df1 = getBinsFromTrend(df.index, df['Close'], [idx_range_from,idx_range_to,1]) #[3,10,1] = range(3,10) This is the issue
  File "so68871906.py", line 52, in getBinsFromTrend
    idx += maxWindow
  File "pandas/_libs/tslibs/timestamps.pyx", line 310, in pandas._libs.tslibs.timestamps._Timestamp.__add__
TypeError: Addition/subtraction of integers and integer-arrays with Timestamp is no longer supported.  Instead of adding/subtracting `n`, use `n * obj.freq`

请注意 type(maxWindow) 确实是 inttype(idx) Timestamp

TypeError 异常消息进一步声明“使用 n * obj.freq 代替 adding/subtracting n”,从中可以推断出 n 旨在表示整数。错误似乎是建议我们先将整数乘以一些 frequency,然后再将其添加到 Timestamp 变量。这还不是很清楚,所以我 Google 搜索“pandas 将整数添加到时间戳”(因为很明显这就是代码试图做的)。所有最热门的答案都建议使用 pandas.to_timedelta()pandas.Timedelta().

在这一点上,我心想:您不能只向时间戳添加一个整数是有道理的,因为您要添加什么?分钟?秒?天?几周?

但是,您可以添加这些频率之一的整数,实际上 pandas.Timedelta() 构造函数采用 value 参数表示天数、周数等。

来自 yf.download() 的数据是每日 (interval='1d'),这表明该整数应乘以 1 天的 pandas.Timedelta。我不能确定这一点,因为我没有你的教科书,所以我不能 100% 确定代码在那里试图完成什么,但这是一个合理的猜测,所以我将 idx += maxWindow 更改为

idx += (maxWindow*pd.Timedelta('1 day'))

看看会发生什么:

dino@DINO:~/code/mplfinance/examples/scratch_pad/issues$ python so68871906.py
[*********************100%***********************]  1 of 1 completed
Traceback (most recent call last):
  File "so68871906.py", line 84, in <module>
    df1 = getBinsFromTrend(df.index, df['Close'], [idx_range_from,idx_range_to,1]) #[3,10,1] = range(3,10) This is the issue
  File "so68871906.py", line 51, in getBinsFromTrend
    if idx >= len(close):
TypeError: '>=' not supported between instances of 'Timestamp' and 'int'

第 2 步:

代码通过了修改后的代码行(第 50 行),但现在它在下一行代码中失败了,类似 TypeError 因为它不支持比较 ('>=')和整数和时间戳。所以接下来我尝试类似地将第 51 行修改为:

    if idx >= len(close)*pd.Timedelta('1 day'):

结果:

dino@DINO:~/code/mplfinance/examples/scratch_pad/issues$ python so68871906.py
[*********************100%***********************]  1 of 1 completed
Traceback (most recent call last):
  File "so68871906.py", line 84, in <module>
    df1 = getBinsFromTrend(df.index, df['Close'], [idx_range_from,idx_range_to,1]) #[3,10,1] = range(3,10) This is the issue
  File "so68871906.py", line 51, in getBinsFromTrend
    if idx >= len(close)*pd.Timedelta('1 day'):
TypeError: '>=' not supported between instances of 'Timestamp' and 'Timedelta'

如您所见,这也不起作用(无法比较 TimestampTimedelta)。

第 3 步:

更仔细地查看代码,似乎代码试图确定将 maxWindow 添加到 idx 是否已将 idx 移动到 末尾数据.

查看代码中更高的几行,您可以看到变量 idx 来自 close.index 中的 Timestamp 个对象列表,因此正确的比较可能是:

    if idx >= close.index[-1]:

也就是说,将 idx 与最后一个可能的 idx 值进行比较。结果:

dino@DINO:~/code/mplfinance/examples/scratch_pad/issues$ python so68871906.py
[*********************100%***********************]  1 of 1 completed
Traceback (most recent call last):
  File "so68871906.py", line 84, in <module>
    df1 = getBinsFromTrend(df.index, df['Close'], [idx_range_from,idx_range_to,1]) #[3,10,1] = range(3,10) This is the issue
  File "so68871906.py", line 60, in getBinsFromTrend
    df_tval.loc[dt1] = tValLinR(df1.values) #calculates t-statistics on period
  File "so68871906.py", line 18, in tValLinR
    ols = sm1.OLS(close, x).fit()
NameError: name 'sm1' is not defined

第 4 步:

哇!很好,我们克服了第 50 和 51 行的错误。但现在我们在第 18 行出现错误。表明名称 sm1 未定义。这表明要么你没有复制你应该拥有的所有代码,要么可能需要导入其他东西来为 python 解释器定义 sm1


所以你看,这是调试代码的基本过程。同样,至少对于 python,关键是仔细阅读 Traceback。再说一次,Whosebug 是 而不是 的,目的是让其他人调试您的代码。在网上稍微搜索一下“学习 python”和“python 调试技术”之类的内容,将会得到大量有用的信息。

我希望这为您指明了正确的方向。如果出于某种原因我对这个答案有很大的误解,请告诉我,我会删除它。