`RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn` for linear regression with gradient descent using torch

Question

我正在尝试使用 pytorch 实现线性回归的简单梯度下降，如文档中 this example 所示：

import torch
from torch.autograd import Variable

learning_rate = 0.01
y = 5
x = torch.tensor([3., 0., 1.])
w = torch.tensor([2., 3., 9.], requires_grad=True)
b = torch.tensor(1., requires_grad=True)

for z in range(100):
    y_pred = b + torch.sum(w * x)
    loss = (y_pred - y).pow(2)
    loss = Variable(loss, requires_grad = True)
    # loss.requires_grad = True
    loss.backward()
    
    with torch.no_grad():
        w = w - learning_rate * w.grad
        b = b - learning_rate * b.grad
        
        w.grad = None
        b.grad = None

当我运行代码时，我收到错误 RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn

我读过here and here可以解决

使用

loss = Variable(loss, requires_grad = True) 结果 TypeError: unsupported operand type(s) for *: 'float' and 'NoneType'
或loss.requires_grad = True结果为RuntimeError: you can only change requires_grad flags of leaf variables.

我该如何解决这个问题？

Answer 1

因为下面的问题更像是一个不完全合适的 hack。这个错误实际上是因为我将 torch 的计算函数与 python 内置函数混合使用（numpy 也应该如此）

在向后调用 .retain_grad() 之前为我解决了这个问题：

learning_rate = 0.01
y = 5
x = torch.tensor([3., 0., 1.])
w = torch.tensor([2., 3., 9.], requires_grad=True)
b = torch.tensor(1., requires_grad=True)

for z in range(100):
    y_pred = b + torch.sum(w * x)
    loss = (y_pred - y).pow(2)
    
    w.retain_grad()
    b.retain_grad()
    loss.backward()
    
    w = w - learning_rate * w.grad
    b = b - learning_rate * b.grad

很好的解释可以看here

`RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn` for linear regression with gradient descent using torch

`RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn` for linear regression with gradient descent using torch

error-handling

linear-regression

gradient-descent

pytorch