准确度值在训练过程中上下波动

Question

训练网络后，我注意到准确率上下波动。最初我认为这是由学习率引起的，但它设置为非常小的值。请检查随附的屏幕截图。 Plot Accuracy Screenshot

我的网络（在 Pytorch 中）如下所示：

class Network(nn.Module):
    def __init__(self):
        super(Network,self).__init__()
    
    self.layer1 = nn.Sequential(
        nn.Conv2d(3,16,kernel_size=3),
        nn.ReLU(),
        nn.MaxPool2d(2)
    )
    
    self.layer2 = nn.Sequential(
        nn.Conv2d(16,32, kernel_size=3),
        nn.ReLU(),
        nn.MaxPool2d(2)
        )
    
    self.layer3 = nn.Sequential(
        nn.Conv2d(32,64, kernel_size=3),
        nn.ReLU(),
        nn.MaxPool2d(2)
    )       
  
    self.fc1 = nn.Linear(17*17*64,512)
    self.fc2 = nn.Linear(512,1)
    self.relu = nn.ReLU()
    self.sigmoid = nn.Sigmoid()
    
    
def forward(self,x):
    out = self.layer1(x)
    out = self.layer2(out)
    out = self.layer3(out)
    out = out.view(out.size(0),-1)
    out = self.relu(self.fc1(out))
    out = self.fc2(out)
    out = torch.sigmoid(out)
    return out

我使用 RMSprop 作为优化器，使用 BCELoss 作为标准。 学习率设置为0.001

训练过程如下：

epochs = 15
itr = 1
p_itr = 100
model.train()
total_loss = 0
loss_list = []
acc_list = []
for epoch in range(epochs):
    for samples, labels in train_loader:
        samples, labels = samples.to(device), labels.to(device)
        optimizer.zero_grad()
        output = model(samples)
        labels = labels.unsqueeze(-1)
        labels = labels.float()
        loss = criterion(output, labels)
        loss.backward()
        optimizer.step()
        total_loss += loss.item()
        scheduler.step()
    
    if itr%p_itr == 0:
        pred = torch.argmax(output, dim=1)
        correct = pred.eq(labels)
        acc = torch.mean(correct.float())
        print('[Epoch {}/{}] Iteration {} -> Train Loss: {:.4f}, Accuracy: {:.3f}'.format(epoch+1, epochs, itr, total_loss/p_itr, acc))
        loss_list.append(total_loss/p_itr)
        acc_list.append(acc)
        total_loss = 0
        
    itr += 1

我的数据集很小——2000 次训练和 1000 次验证（二进制分类 0/1）。我想进行 80/20 拆分，但我被要求保持这样。我在想对于这么小的数据集来说架构可能太复杂了。

在训练过程中有什么可能导致这种跳跃的命中吗？

Answer 1

你这里的代码是错误的：pred = torch.argmax(output, dim=1)

此行用于具有交叉熵损失的多类分类。您的任务是二元分类，因此 pred 值是错误的。更改为：

if itr%p_itr == 0:
  pred = torch.round(output)
  ....

您可以将优化器更改为 Adam、SGD 或 RMSprop 以找到合适的优化器来帮助您更快地覆盖模型。同时更改 forward() 函数：

def forward(self,x):
   out = self.layer1(x)
   out = self.layer2(out)
   out = self.layer3(out)
   out = out.view(out.size(0),-1)
   out = self.relu(self.fc1(out))
   out = self.fc2(out)
   return self.sigmoid(out) #use your forward is ok, but this cleaner

准确度值在训练过程中上下波动

Accuracy value goes up and down on the training process

machine-learning

pytorch

image-classification