Training Yolov5 on RTX 3060 Ti GPU I'm getting error "RuntimeError: Unable to find a valid cuDNN algorithm to run convolution"

Question

使用以下命令在 RTX 3060 Ti GPU 上使用 --img 8088 和批量大小 16 训练 Yolov5

python train.py --img 1088 --batch 16 --epochs 3 --data coco128.yaml --weights yolov5s.pt --设备 0 --workers 0

我收到以下异常“RuntimeError：无法找到一个有效的 cuDNN 算法来运行卷积”，通过将批量大小减少到 8，我能够训练模型

 File "train.py", line 611, in <module>
    main(opt)
  File "train.py", line 509, in main
    train(opt.hyp, opt, device)
  File "train.py", line 311, in train
    pred = model(imgs)  # forward
  File "C:\Program Files\Python38\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "C:\Users\hamza.m\workspace\yolov5\models\yolo.py", line 123, in forward
    return self.forward_once(x, profile, visualize)  # single-scale inference, train
  File "C:\Users\hamza.m\workspace\yolov5\models\yolo.py", line 155, in forward_once
    x = m(x)  # run
  File "C:\Program Files\Python38\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "C:\Users\hamza.m\workspace\yolov5\models\common.py", line 137, in forward
    return self.cv3(torch.cat((self.m(self.cv1(x)), self.cv2(x)), dim=1))
  File "C:\Program Files\Python38\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "C:\Users\hamza.m\workspace\yolov5\models\common.py", line 45, in forward
    return self.act(self.bn(self.conv(x)))
  File "C:\Program Files\Python38\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "C:\Program Files\Python38\lib\site-packages\torch\nn\modules\conv.py", line 423, in forward
    return self._conv_forward(input, self.weight)
  File "C:\Program Files\Python38\lib\site-packages\torch\nn\modules\conv.py", line 419, in _conv_forward
    return F.conv2d(input, weight, self.bias, self.stride,
RuntimeError: Unable to find a valid cuDNN algorithm to run convolution

P.S 还有谁能指导我如何评估哪种 GPU 最适合训练我的模型，请也请赐教

Answer 1

答案在错误日志中

运行时错误：CUDA 内存不足。尝试分配 100.00 MiB（GPU 0；8.00 GiB 总容量；5.48 GiB 已分配；81.94 MiB 可用；PyTorch 总共保留 5.61 GiB）

它正在尝试分配比您的 GPU 上更多的内存。

Answer 2

尝试减少 batch_size，我遇到了同样的问题，当我减少批量大小时，它对我有用！

Training Yolov5 on RTX 3060 Ti GPU I'm getting error "RuntimeError: Unable to find a valid cuDNN algorithm to run convolution"

Training Yolov5 on RTX 3060 Ti GPU I'm getting error "RuntimeError: Unable to find a valid cuDNN algorithm to run convolution"

python

gpu

deep-learning

tensorflow