如何计算回归预测的置信区间？以及如何在 python 中绘制它

Question

图 7.1，统计学习简介

我目前正在学习一本名为 Introduction to Statistical Learning with applications in R 的书，并将解决方案转换为 python 语言。
我无法获得如何获得置信区间并绘制它们，如上图（虚线）所示。我画了线。这是我的代码 - （我正在使用多项式回归与预测变量 - 'age' 和响应 - 'wage'，度数为 4）

poly = PolynomialFeatures(4)
X = poly.fit_transform(data['age'].to_frame())
y = data['wage']
# X.shape

model = sm.OLS(y,X).fit()
print(model.summary())

# So, what we want here is not only the final line, but also the standart error related to the line
# TO find that we need to calcualte the predictions for some values of age
test_ages = np.linspace(data['age'].min(),data['age'].max(),100)

X_test = poly.transform(test_ages.reshape(-1,1))
pred = model.predict(X_test)

plt.figure(figsize = (12,8))
plt.scatter(data['age'],data['wage'],facecolors='none', edgecolors='darkgray')
plt.plot(test_ages,pred)

这里的数据是 R 中可用的 WAGE 数据。这是我得到的结果图 -

Answer 1

以下代码生成 95% 的置信区间

from scipy import stats

confidence = 0.95
squared_errors = (<<predicted values>> - <<true y_test values>>) ** 2
np.sqrt(stats.t.interval(confidence, len(squared_errors) - 1,
                         loc=squared_errors.mean(),
                         scale=stats.sem(squared_errors)))

Answer 2

我使用引导程序来计算置信区间，为此我使用了一个自定义模块 -

import numpy as np
import pandas as pd
from tqdm import tqdm

class Bootstrap_ci:


    def boot(self,X_data,y_data,R,test_data,model):
        predictions = []
        for i in tqdm(range(R)):
            predictions.append(self.alpha(X_data,y_data,self.get_indices(X_data,200),test_data,model))
           
        return np.percentile(predictions,2.5,axis = 0),np.percentile(predictions,97.5,axis = 0)

    def alpha(self,X_data,y_data,index,test_data,model):
        X = X_data.loc[index]
        y = y_data.loc[index]
        
        lr = model
        lr.fit(pd.DataFrame(X),y)
        
        return lr.predict(pd.DataFrame(test_data))


    def get_indices(self,data,num_samples):
        return  np.random.choice(data.index, num_samples, replace=True)

以上模块可以作为-

poly = PolynomialFeatures(4)
X = poly.fit_transform(data['age'].to_frame())
y = data['wage']

X_test = np.linspace(min(data['age']),max(data['age']),100)
X_test_poly = poly.transform(X_test.reshape(-1,1))

from bootstrap import Bootstrap_ci

bootstrap = Bootstrap_ci()

li,ui = bootstrap.boot(pd.DataFrame(X),y,1000,X_test_poly,LinearRegression())

这将为我们提供较低的置信区间和较高的置信区间。绘制图形 -

plt.scatter(data['age'],data['wage'],facecolors='none', edgecolors='darkgray')
plt.plot(X_test,pred,label = 'Fitted Line')
plt.plot(X_test,ui,linestyle = 'dashed',color = 'r',label = 'Confidence Intervals')
plt.plot(X_test,li,linestyle = 'dashed',color = 'r')

结果图是

如何计算回归预测的置信区间？以及如何在 python 中绘制它

How to calculate the confidence intervals for prediction in Regression? and also how to plot it in python

statistics

machine-learning

matplotlib

linear-regression

confidence-interval