专家组合 - fminunc 优化过早停止

Question

我有一个混合专家代码，它使用少量参数。如下：

global x_au;
global x_vi;
global x_alpha;
global y;
global parameter;

options = optimoptions(@fminunc,'GradObj', 'on', 'Algorithm','quasi-newton','MaxIter', 10000,'Display','iter-detailed'); % change number of iterations
optTheta=[];

x_au=x_au_train;
x_vi=x_vi_train;
x_alpha=x_alpha_train;
y=y_train;
parameter=zeros(8969,1);

%expectation step
fprintf('opt1 begins');
opt_1;
fprintf('opt1 complete');
%maximaization step
[x] = fminunc(@costfunction,parameter(1:4483),options);
parameter(1:4483)=x;
resnorm1=total_error(parameter(1:4483));
k=1;
count = 1;
while(1)
opt_1; 
fprintf('expectation complete');
%maximaization step
[x] = fminunc(@costfunction,parameter(1:4483),options);
parameter(1:4483)=x;
resnorm2=total_error(parameter(1:4483));
fprintf('resnorm1-resnorm2 - %f, resnorm2 - %f, k - %f',resnorm1-resnorm2,0.000001*resnorm2,k);
if((resnorm1-resnorm2)< .000001*resnorm2 & k~=1)  %% to decrease training time 
    break;
end

但是现在，当我不得不在具有大量参数的问题上使用它时，我得到以下日志。

                                                     First-order 
 Iteration  Func-count       f(x)        Step-size       optimality
     0           1      5.31444e+10                      4.75e+14

Optimization stopped because the objective function cannot be decreased in the 
current search direction. Either the predicted change in the objective function,
or the line search interval is less than eps.
                                                   First-order 
 Iteration  Func-count       f(x)        Step-size       optimality
     0           1      5.31444e+10                      4.75e+14

Optimization stopped because the objective function cannot be decreased in the 
current search direction. Either the predicted change in the objective function,
or the line search interval is less than eps.

resnorm1-resnorm2 - 0.000000, resnorm2 - 53144.356560, k - 1.000000 
                                                    First-order 
 Iteration  Func-count       f(x)        Step-size       optimality
     0           1      5.31444e+10                      4.75e+14

Optimization stopped because the objective function cannot be decreased in the 
current search direction. Either the predicted change in the objective function,
or the line search interval is less than eps.

    resnorm1-resnorm2 - 0.000000, resnorm2 - 53144.356560, k - 2.000000
>>

然后该过程以非常糟糕的结果完成。如，可以看出 fminunc 无法正确优化。有人能帮我一下吗？

Answer 1

看来您需要降低训练速度系数，或者对参数向量进行归一化。

Answer 2

我将参数初始化从 zeros 更改为 rand 并结合规范化，我让它工作了。

专家组合 - fminunc 优化过早停止

Mixture of Experts - fminunc optimization stopped prematurely

matlab

regression

machine-learning