获取 nnet 的插入符号模型参数

Question

我在为 nnet 提取插入符号的最终模型参数时遇到问题。如果我使用 - 在我看来 - caret::train 和 nnet::nnet 完全相同的参数，我会（有时）得到很大的差异。我是否忘记了参数或者这是由于神经网络的计算算法？我知道我可以对 caret_net 使用预测（在下面的示例中），但我仍然想仅使用 nnet.

重现结果

示例：

library(nnet)
library(caret)

len <- 100
set.seed(4321)
X <- data.frame(x1 = rnorm(len, 40, 25), x2 = rnorm(len, 70, 4), x3 = rnorm(len, 1.6, 0.3))
y <- 20000 + X$x1 * 3 - X$x1*X$x2 * 4 - (X$x3**4) * 7 + rnorm(len, 0, 4)
XY <- cbind(X, y)

# pre-processing
preProcPrms <- preProcess(XY, method = c("center", "scale"))
XY_pre <- predict(preProcPrms, XY)

# caret-nnet
controlList <- trainControl(method = "cv", number = 5)
tuneMatrix <- expand.grid(size = c(1, 2), decay = c(0, 0.1))

caret_net <- train(x = XY_pre[ , colnames(XY_pre) != "y"],
                   y = XY_pre[ , colnames(XY_pre) == "y"],
                   method = "nnet",
                   linout = TRUE,
                   TRACE = FALSE,
                   maxit = 100,
                   tuneGrid = tuneMatrix,
                   trControl = controlList)

# nnet-nnet
nnet_net <- nnet(x = XY_pre[ , colnames(XY_pre) != "y"],
                 y = XY_pre[ , colnames(XY_pre) == "y"],
                 linout = caret_net$finalModel$param$linout,
                 TRACE = caret_net$finalModel$param$TRACE,
                 size = caret_net$bestTune$size,
                 decay = caret_net$bestTune$decay,
                 entropy = caret_net$finalModel$entropy,
                 maxit = 100)

# print
print(caret_net$finalModel)
print(nnet_net)

y_caret <- predict(caret_net$finalModel, XY_pre[ , colnames(XY_pre) != "y"])
y_nnet <- predict(nnet_net, XY_pre[ , colnames(XY_pre) != "y"])

plot(y_caret, y_nnet, main = "Hard to spot, but y_caret <> y_nnet - which prm have I forgotten?")
hist(y_caret - y_nnet)

谢谢和亲切的问候

Answer 1

如评论中所述，差异是由不同的种子引起的。引用@Artem Sokolov：神经网络训练通常从随机状态开始。 caret::train 和 nnet::nnet 从两个不同的状态开始是合理的。因此，它们可能会收敛到两个不同的局部最优值。

要从同一种子开始获得可重现的模型：

controlList <- trainControl(method = "none", seeds = 1)
tuneMatrix <- expand.grid(size = 2, decay = 0)

set.seed(1)
caret_net <- train(x = XY_pre[ , colnames(XY_pre) != "y"],
                   y = XY_pre[ , colnames(XY_pre) == "y"],
                   method = "nnet",
                   linout = TRUE,
                   TRACE = FALSE,
                   maxit = 100,
                   tuneGrid = tuneMatrix,
                   trControl = controlList)

set.seed(1)
nnet_net <- nnet(x = XY_pre[ , colnames(XY_pre) != "y"],
                 y = XY_pre[ , colnames(XY_pre) == "y"],
                 linout = caret_net$finalModel$param$linout,
                 TRACE = caret_net$finalModel$param$TRACE,
                 size = caret_net$bestTune$size,
                 decay = caret_net$bestTune$decay,
                 entropy = caret_net$finalModel$entropy,
                 maxit = 100)

y_caret <- predict(caret_net, XY_pre[ , colnames(XY_pre) != "y"])
y_nnet <- predict(nnet_net, XY_pre[ , colnames(XY_pre) != "y"])


all.equal(as.vector(y_caret[,1]), y_nnet[,1])
#TRUE

除了设置相同的种子之外，关键是要避免在插入符号中重新采样，因为它取决于种子并在模型训练之前。

获取 nnet 的插入符号模型参数

Get caret's model-parameters for nnet

r

nnet

r-caret