RuntimeError: expected scalar type Float but found Double error torch.nn.CrossEntropyLoss Pytorch

Question

我正在尝试训练 pytorch 模型。损失函数为：

cn_loss = torch.nn.CrossEntropyLoss(weight=train_label_weight, reduction='mean')

来自训练函数的代码片段：

for sents, targets in batch_iter(df_train, batch_size=train_batch_size, shuffle=True, bert=bert_size):
        train_iter += 1
        optimizer.zero_grad()
        batch_size = len(sents)
        pre_softmax = model(sents)
        float_targets=torch.tensor(targets, dtype=torch.float, device=device)

        loss = cn_loss(pre_softmax, float_targets)
        
        loss.backward()
        optimizer.step()

pre_softmax和float_targets的数据类型都是torch.float32。（在原始代码中，targets 的数据类型已使用 torch.tensor(targets, dtype=torch.long, device=device) 转换为 torch.int64。但是，当我得到

RuntimeError: expected scalar type Float but found Double

错误我将targets的数据类型转换为torch.float32)

即使 cn_loss() 函数中的两个参数都是 torch.float32，当我运行代码时我得到以下错误：

loss = cn_loss(pre_softmax, float_targets) File "C:\Users\user\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\nn\modules\module.py", line 1102, in _call_impl return forward_call(*input, **kwargs) File "C:\Users\user\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\nn\modules\loss.py", line 1152, in forward label_smoothing=self.label_smoothing) File "C:\Users\user\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\nn\functional.py", line 2846, in cross_entropy return torch._C._nn.cross_entropy_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index, label_smoothing) RuntimeError: expected scalar type Float but found Double

我多次检查数据类型并使用 torch.FloatTensor(targets) 更改了 float_targets 的数据类型。但是我得到了同样的错误。

Answer 1

正如@aretor 在评论中指出的那样，train_label_weight 的数据类型是torch.float64。当将其转换为 torch.float32 并再次将目标更改为 torch.long 时，代码运行完美

RuntimeError: expected scalar type Float but found Double error torch.nn.CrossEntropyLoss Pytorch

RuntimeError: expected scalar type Float but found Double error torch.nn.CrossEntropyLoss Pytorch

python

cross-entropy

pytorch