张量的 Wav2Vec pytorch 元素 0 不需要 grad 并且没有 grad_fn

Question

我正在为分类问题的拥抱脸重新训练 wav2vec 模型。我有 5 类，输入是张量列表 [1,400]。这是我如何获得模型

num_labels = 5
model_name = "Zaid/wav2vec2-large-xlsr-53-arabic-egyptian"
model_config = AutoConfig.from_pretrained(model_name, num_labels=num_labels)  ##needed for the visualizations
tokenizer = Wav2Vec2CTCTokenizer.from_pretrained(model_name)
model = Wav2Vec2ForCTC.from_pretrained(model_name, config=model_config)

这是模型更新后的设置

# Freeze the pre trained parameters
for param in model.parameters():
    param.requires_grad = False
criterion = nn.MSELoss().to(device)
optimizer = AdamW(model.parameters(), lr=2e-5, eps=1e-6)

# Add three new layers at the end of the network
model.classifier = nn.Sequential(
    nn.Linear(768, 256),
    nn.Dropout(0.25),
    nn.ReLU(),
    nn.Linear(256, 64),
    nn.Dropout(0.25),
    nn.ReLU(),
    nn.Linear(64, 2),
    nn.Dropout(0.25),
    nn.Softmax(dim=1)
)

然后是训练循环

print_every = 300

total_loss = 0
all_losses = []
model.train()
for epoch in range(2):
    print("Epoch number: ", epoch)
    for row in range(16918):
        Input = torch.tensor(trn_ivectors[row]).double()
        label = torch.tensor(trn_labels[row]).long().to(device)
        label = torch.unsqueeze(label,0).to(device)
        #print("Label", label.shape)
        Input = torch.unsqueeze(Input,1).to(device)
        #print(Input.shape)
        optimizer.zero_grad()
        
        #Input.requires_grad = True
        Input = F.softmax(Input[0], dim=-1)
        
        if label == 0:
            label = torch.tensor([1.0, 0.0]).float().to(device)
        elif label == 1:
            label = torch.tensor([0.0, 1.0]).float().to(device)

        # print(overall_output, label)

        loss = criterion(Input, label)
        total_loss += loss.item()

        loss.backward()
        optimizer.step()

        if idx % print_every == 0 and idx > 0:
            average_loss = total_loss / print_every
            print("{}/{}. Average loss: {}".format(idx, len(train_data), average_loss))
            all_losses.append(average_loss)
            total_loss = 0

torch.save(model.state_dict(), "model_after_train.pt")

不幸的是，当我尝试训练程序时，出现以下错误

RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn

如果您能告诉我如何修复此错误，我将不胜感激。我一直在寻找修复它的方法但没有修复它

谢谢

Answer 1

请尝试添加

requires_grad = True

张量的 Wav2Vec pytorch 元素 0 不需要 grad 并且没有 grad_fn

Wav2Vec pytorch element 0 of tensors does not require grad and does not have a grad_fn

python

nlp

deep-learning

pytorch

huggingface-transformers