多元线性回归成本太高
Multivariate Linear Regression Cost Too High
我正在使用 this link、imports-85.data
中提供的数据集进行价格预测。
使用 horsepower
、curb-weight
、engine-size
和 highway-mpg
,我尝试归一化(由于成本高)和 运行 梯度下降通过实现以下算法:
初始化
data = df[attrs]
m = len(data) # m-training examples
f = len(attrs) # n-features
X = np.hstack((np.ones(shape=(m,1)),np.array(data)))
T = np.zeros(f + 1) # Coefficients of x(0),x(1),...x(n)
norm_price = df.price / 1000
Y = np.array(norm_price)
# Normalization
data['curb-weight'] = (data['curb-weight'] * 0.453592) / 1000 # To kg (e-1000)
data['highway-mpg'] = data['highway-mpg'] * 0.425144 # To km per litre (kml)
data['engine-size'] = data['engine-size'] / 100 # To e-100
data['horsepower'] = data['horsepower'] / 100 # To e-100
col_rename = {
'curb-weight':'curb-weight-kg(e-1000)',
'highway-mpg':'highway-kml',
'engine-size':'engine-size(e-100)',
'horsepower':'horsepower(e-100)'
}
data.rename(columns=col_rename,inplace=True)
成本计算
def calculateCost():
global m,T,X
hypot = (X.dot(T) - Y).transpose().dot(X.dot(T) - Y)
return hypot / (2 * m)
梯度下降
def gradDescent(threshold,iter = 10000,alpha = 3e-8):
global T,X,Y,m
i = 0
cost = calculateCost()
cost_hist = [cost]
while i < iter:
T = T - (alpha / m) * X.transpose().dot(X.dot(T) - Y)
cost = calculateCost()
cost_hist.append(cost)
i += 1
if cost <= threshold:
return cost_hist
我运行 使用此实现的梯度下降:
Batch Gradient Descent
如果没有规范化,成本将为 118634960.460199
。
通过规范化,成本将为 118.634960460199
因此,我有几个问题:
- 我的标准化技术正确吗?
- 标准化后,成本会有所不同。归一化后的成本阈值如何设置?
我认为您在机器学习的上下文中可能存在误解 'normalization'。根据我对您代码的解释,您的 'normalization' 部分正在进行单位转换。在梯度下降之前,通常应用最大最小缩放或标准缩放,请参阅 scikit learn user guide. These techniques create features with a consistent scale range, so that changes in a single feature do not completely dominate the loss function. This question and this blog post 有更长的讨论。
我正在使用 this link、imports-85.data
中提供的数据集进行价格预测。
使用 horsepower
、curb-weight
、engine-size
和 highway-mpg
,我尝试归一化(由于成本高)和 运行 梯度下降通过实现以下算法:
初始化
data = df[attrs]
m = len(data) # m-training examples
f = len(attrs) # n-features
X = np.hstack((np.ones(shape=(m,1)),np.array(data)))
T = np.zeros(f + 1) # Coefficients of x(0),x(1),...x(n)
norm_price = df.price / 1000
Y = np.array(norm_price)
# Normalization
data['curb-weight'] = (data['curb-weight'] * 0.453592) / 1000 # To kg (e-1000)
data['highway-mpg'] = data['highway-mpg'] * 0.425144 # To km per litre (kml)
data['engine-size'] = data['engine-size'] / 100 # To e-100
data['horsepower'] = data['horsepower'] / 100 # To e-100
col_rename = {
'curb-weight':'curb-weight-kg(e-1000)',
'highway-mpg':'highway-kml',
'engine-size':'engine-size(e-100)',
'horsepower':'horsepower(e-100)'
}
data.rename(columns=col_rename,inplace=True)
成本计算
def calculateCost():
global m,T,X
hypot = (X.dot(T) - Y).transpose().dot(X.dot(T) - Y)
return hypot / (2 * m)
梯度下降
def gradDescent(threshold,iter = 10000,alpha = 3e-8):
global T,X,Y,m
i = 0
cost = calculateCost()
cost_hist = [cost]
while i < iter:
T = T - (alpha / m) * X.transpose().dot(X.dot(T) - Y)
cost = calculateCost()
cost_hist.append(cost)
i += 1
if cost <= threshold:
return cost_hist
我运行 使用此实现的梯度下降: Batch Gradient Descent
如果没有规范化,成本将为 118634960.460199
。
通过规范化,成本将为 118.634960460199
因此,我有几个问题:
- 我的标准化技术正确吗?
- 标准化后,成本会有所不同。归一化后的成本阈值如何设置?
我认为您在机器学习的上下文中可能存在误解 'normalization'。根据我对您代码的解释,您的 'normalization' 部分正在进行单位转换。在梯度下降之前,通常应用最大最小缩放或标准缩放,请参阅 scikit learn user guide. These techniques create features with a consistent scale range, so that changes in a single feature do not completely dominate the loss function. This question and this blog post 有更长的讨论。