如何排除 NA？ （拟合函数）
How to exclude NAs? (fitdist function)
我有 100x2 数据帧 DFN。 运行 fitdist 列 DFN$Lret 给出错误消息 "function mle failed to estimate the parameters, with the error code 100"。我认为原因是最后一行包含 NA。因此我 运行 fitdist 排除了 NA，现在我得到错误 "data must be a numeric vector of length greater than 1"。关于如何解决这个问题的任何想法？非常感谢。
DFN <- structure(list(LRet = c(0.0011, 0, -0.0026, 0, -0.0015, 0.0038, 3e-04, -0.0021, 4e-04, -0.001, 0, 0.0019, -6e-04, -8e-04, -5e-04, -8e-04, 3e-04, -5e-04, -0.0026, 0.0014, 7e-04, 0, -2e-04, 0.0011, -0.0025, 0.0042, 0.0022, -0.0017, -0.0058, 1e-04, 2e-04, 8e-04, -9e-04, -0.0014, -0.0014, -0.001, -0.0032, -0.0015, 6e-04, -8e-04, 0.001, -0.0014, -0.0017, -8e-04, -0.001, 0.0011, 0.0013, -0.001, 5e-04, 9e-04, -8e-04, -0.0025, 0.0027, 6e-04, 2e-04, -6e-04, 9e-04, -3e-04, -7e-04, 3e-04, 0, 2e-04, -6e-04, 1e-04, -1e-04, -7e-04, -8e-04, 7e-04, -1e-04, -7e-04, 7e-04, 8e-04, -8e-04, 8e-04, 0.0058, -1e-04, -5e-04, 0.0027, -0.0012, 7e-04, 7e-04, 0, 3e-04, -1e-04, 2e-04, -2e-04, -0.0013, -1e-04, 1e-04, -0.0011, 0.0013, 2e-04, -3e-04, -7e-04, 0, 0.0015, 1e-04, 3e-04, -0.0012, NA), LRetPct = c("0.11%", "0.00%", "-0.26%", "0.00%", "-0.15%", "0.38%", "0.03%", "-0.21%", "0.04%", "-0.10%", "0.00%", "0.19%", "-0.06%", "-0.08%", "-0.05%", "-0.08%", "0.03%", "-0.05%", "-0.26%", "0.14%", "0.07%", "0.00%", "-0.02%", "0.11%", "-0.25%", "0.42%", "0.22%", "-0.17%", "-0.58%", "0.01%", "0.02%", "0.08%", "-0.09%", "-0.14%", "-0.14%", "-0.10%", "-0.32%", "-0.15%", "0.06%", "-0.08%", "0.10%", "-0.14%", "-0.17%", "-0.08%", "-0.10%", "0.11%", "0.13%", "-0.10%", "0.05%", "0.09%", "-0.08%", "-0.25%", "0.27%", "0.06%", "0.02%", "-0.06%", "0.09%", "-0.03%", "-0.07%", "0.03%", "0.00%", "0.02%", "-0.06%", "0.01%", "-0.01%", "-0.07%", "-0.08%", "0.07%", "-0.01%", "-0.07%", "0.07%", "0.08%", "-0.08%", "0.08%", "0.58%", "-0.01%", "-0.05%", "0.27%", "-0.12%", "0.07%", "0.07%", "0.00%", "0.03%", "-0.01%", "0.02%", "-0.02%", "-0.13%", "-0.01%", "0.01%", "-0.11%", "0.13%", "0.02%", "-0.03%", "-0.07%", "0.00%", "0.15%", "0.01%", "0.03%", "-0.12%", " NA%")), .Names = c("LRet", "LRetPct"), class = "data.frame", row.names = 901:1000)
library(fitdistrplus)
#Following gives error code 100
f1 <- fitdist(DFN$LRet,"norm")
#Following gives error code 100
f1 <- fitdist(DFN$LRet,"norm", na.rm=T)
#Following gives error data must be a numeric vector of length greater than 1"
f1 <- fitdist(na.exclude(DFN$LRet),"norm")
#Same result using na.omit
请注意，如果删除包含 NA 的最后一行，则上述代码可以正常工作。如果可以避免，我宁愿不必删除 运行ning fitdist 之前的最后一行。
EDIT/UPDATE：用 NA 消除最后一行一开始确实解决了问题，但我现在无法始终如一地重现（即在消除后成功 运行 代码几次最后一行，但并非总是如此）。我试图理解为什么。我尝试使用 25x2 数据框、100x2 和 300x2 以及矢量，结果相似。认为数据框或向量的大小可能是问题的一部分，因此进行了不同大小的试验。
调试通过fitdist显示
if (!(is.vector(data) & is.numeric(data) & length(data) > 1))
stop("data must be a numeric vector of length greater than 1")
正在查看?is.vector：
‘is.vector’ returns ‘TRUE’ if ‘x’ is a vector of the specified
mode having no attributes other than names.
na.exclude 及其亲戚（na.omit 等）将有关排除值的信息保存为属性，因此 is.vector() 变为 FALSE ...
c() 的副作用之一是删除名称以外的属性，所以 is.vector(c(na.exclude(DFN$LRet))) 是 TRUE，所以
fitdist(c(na.exclude(DFN$LRet)), "norm")
至少没有得到 "must be a numeric vector" 错误——但我仍然得到 "error 100"。进一步调查...
进一步深入 fitdist 的内部，似乎（正如@42- 所建议的那样）optim() 遇到了麻烦。具体来说，它实际上得到了一个答案，但是当它尝试计算解的 Hessian 矩阵时，它尝试为标准差参数和 barfs 取一个负值。
举个例子，这是可行的：
nn <- c(na.exclude(DFN$LRet))
fn <- function(x) -sum(dnorm(nn,mean=x[1],sd=x[2],log=TRUE))
optim(fn,par=c(mean(nn),sd(nn)),method="Nelder-Mead")
但这失败了：
optim(fn,par=c(mean(nn),sd(nn)),method="Nelder-Mead",hessian=TRUE)
（也发现了代码写得不好的is.vector段，但是并没有解决错误。）fitdist函数似乎对小方差的向量有困难：
var( na.exclude(DFN$LRet))
[1] 2.220427e-06
你可以通过乘以 10 来解决这个问题：
> f1 <- fitdist(10*c(na.exclude(DFN$LRet)),"norm")
> f1
Fitting of the distribution ' norm ' by maximum likelihood
Parameters:
estimate Std. Error
mean -0.0009090909 0.001490034
sd 0.0148256472 0.001032122
标准概率论让您可以修正这些估计：除以 10 得到平均值，除以 100 得到方差（或 10 得到 sd）。修正后的 fitdist- 结果的估计值相当接近样本值：
> all.equal( 0.0148256472/10 , sd(na.exclude(DFN$LRet) ) )
[1] "Mean relative difference: 0.005089095"
我有 100x2 数据帧 DFN。 运行 fitdist 列 DFN$Lret 给出错误消息 "function mle failed to estimate the parameters, with the error code 100"。我认为原因是最后一行包含 NA。因此我 运行 fitdist 排除了 NA，现在我得到错误 "data must be a numeric vector of length greater than 1"。关于如何解决这个问题的任何想法？非常感谢。
DFN <- structure(list(LRet = c(0.0011, 0, -0.0026, 0, -0.0015, 0.0038, 3e-04, -0.0021, 4e-04, -0.001, 0, 0.0019, -6e-04, -8e-04, -5e-04, -8e-04, 3e-04, -5e-04, -0.0026, 0.0014, 7e-04, 0, -2e-04, 0.0011, -0.0025, 0.0042, 0.0022, -0.0017, -0.0058, 1e-04, 2e-04, 8e-04, -9e-04, -0.0014, -0.0014, -0.001, -0.0032, -0.0015, 6e-04, -8e-04, 0.001, -0.0014, -0.0017, -8e-04, -0.001, 0.0011, 0.0013, -0.001, 5e-04, 9e-04, -8e-04, -0.0025, 0.0027, 6e-04, 2e-04, -6e-04, 9e-04, -3e-04, -7e-04, 3e-04, 0, 2e-04, -6e-04, 1e-04, -1e-04, -7e-04, -8e-04, 7e-04, -1e-04, -7e-04, 7e-04, 8e-04, -8e-04, 8e-04, 0.0058, -1e-04, -5e-04, 0.0027, -0.0012, 7e-04, 7e-04, 0, 3e-04, -1e-04, 2e-04, -2e-04, -0.0013, -1e-04, 1e-04, -0.0011, 0.0013, 2e-04, -3e-04, -7e-04, 0, 0.0015, 1e-04, 3e-04, -0.0012, NA), LRetPct = c("0.11%", "0.00%", "-0.26%", "0.00%", "-0.15%", "0.38%", "0.03%", "-0.21%", "0.04%", "-0.10%", "0.00%", "0.19%", "-0.06%", "-0.08%", "-0.05%", "-0.08%", "0.03%", "-0.05%", "-0.26%", "0.14%", "0.07%", "0.00%", "-0.02%", "0.11%", "-0.25%", "0.42%", "0.22%", "-0.17%", "-0.58%", "0.01%", "0.02%", "0.08%", "-0.09%", "-0.14%", "-0.14%", "-0.10%", "-0.32%", "-0.15%", "0.06%", "-0.08%", "0.10%", "-0.14%", "-0.17%", "-0.08%", "-0.10%", "0.11%", "0.13%", "-0.10%", "0.05%", "0.09%", "-0.08%", "-0.25%", "0.27%", "0.06%", "0.02%", "-0.06%", "0.09%", "-0.03%", "-0.07%", "0.03%", "0.00%", "0.02%", "-0.06%", "0.01%", "-0.01%", "-0.07%", "-0.08%", "0.07%", "-0.01%", "-0.07%", "0.07%", "0.08%", "-0.08%", "0.08%", "0.58%", "-0.01%", "-0.05%", "0.27%", "-0.12%", "0.07%", "0.07%", "0.00%", "0.03%", "-0.01%", "0.02%", "-0.02%", "-0.13%", "-0.01%", "0.01%", "-0.11%", "0.13%", "0.02%", "-0.03%", "-0.07%", "0.00%", "0.15%", "0.01%", "0.03%", "-0.12%", " NA%")), .Names = c("LRet", "LRetPct"), class = "data.frame", row.names = 901:1000)
library(fitdistrplus)
#Following gives error code 100
f1 <- fitdist(DFN$LRet,"norm")
#Following gives error code 100
f1 <- fitdist(DFN$LRet,"norm", na.rm=T)
#Following gives error data must be a numeric vector of length greater than 1"
f1 <- fitdist(na.exclude(DFN$LRet),"norm")
#Same result using na.omit
请注意，如果删除包含 NA 的最后一行，则上述代码可以正常工作。如果可以避免，我宁愿不必删除 运行ning fitdist 之前的最后一行。
EDIT/UPDATE：用 NA 消除最后一行一开始确实解决了问题，但我现在无法始终如一地重现（即在消除后成功 运行 代码几次最后一行，但并非总是如此）。我试图理解为什么。我尝试使用 25x2 数据框、100x2 和 300x2 以及矢量，结果相似。认为数据框或向量的大小可能是问题的一部分，因此进行了不同大小的试验。
调试通过fitdist显示
if (!(is.vector(data) & is.numeric(data) & length(data) > 1))
stop("data must be a numeric vector of length greater than 1")
正在查看?is.vector：
‘is.vector’ returns ‘TRUE’ if ‘x’ is a vector of the specified mode having no attributes other than names.
na.exclude 及其亲戚（na.omit 等）将有关排除值的信息保存为属性，因此 is.vector() 变为 FALSE ...
c() 的副作用之一是删除名称以外的属性，所以 is.vector(c(na.exclude(DFN$LRet))) 是 TRUE，所以
fitdist(c(na.exclude(DFN$LRet)), "norm")
至少没有得到 "must be a numeric vector" 错误——但我仍然得到 "error 100"。进一步调查...
进一步深入 fitdist 的内部，似乎（正如@42- 所建议的那样）optim() 遇到了麻烦。具体来说，它实际上得到了一个答案，但是当它尝试计算解的 Hessian 矩阵时，它尝试为标准差参数和 barfs 取一个负值。
举个例子，这是可行的：
nn <- c(na.exclude(DFN$LRet))
fn <- function(x) -sum(dnorm(nn,mean=x[1],sd=x[2],log=TRUE))
optim(fn,par=c(mean(nn),sd(nn)),method="Nelder-Mead")
但这失败了：
optim(fn,par=c(mean(nn),sd(nn)),method="Nelder-Mead",hessian=TRUE)
（也发现了代码写得不好的is.vector段，但是并没有解决错误。）fitdist函数似乎对小方差的向量有困难：
var( na.exclude(DFN$LRet))
[1] 2.220427e-06
你可以通过乘以 10 来解决这个问题：
> f1 <- fitdist(10*c(na.exclude(DFN$LRet)),"norm")
> f1
Fitting of the distribution ' norm ' by maximum likelihood
Parameters:
estimate Std. Error
mean -0.0009090909 0.001490034
sd 0.0148256472 0.001032122
标准概率论让您可以修正这些估计：除以 10 得到平均值，除以 100 得到方差（或 10 得到 sd）。修正后的 fitdist- 结果的估计值相当接近样本值：
> all.equal( 0.0148256472/10 , sd(na.exclude(DFN$LRet) ) )
[1] "Mean relative difference: 0.005089095"