OSError: [Errno 22] Invalid argument when using torch.load

Question

我正在尝试加载我的数据集，它之前可以正常工作，但突然间这个错误开始出现。

当我尝试像这样加载它时：

  train_set = Database_load(root = "C:\Users\Public\PhysNet\",train = "train.pth")

它给我以下错误：


  File "C:\Users\Public\Lucas\PhysNet\modt.py", line 185, in <module>
    train_set = Database_load(root = "C:\Users\Public\Lucas\PhysNet\",train = "train.pth")

  File "C:\Users\Public\Lucas\PhysNet\database_load.py", line 21, in __init__
    self.data, self.y1, self.y2= torch.load(os.path.join(self.root, self.train))

  File "C:\ProgramData\Anaconda3\envs\project\lib\site-packages\torch\serialization.py", line 386, in load
    return _load(f, map_location, pickle_module, **pickle_load_args)

  File "C:\ProgramData\Anaconda3\envs\project\lib\site-packages\torch\serialization.py", line 580, in _load
    deserialized_objects[key]._set_from_file(f, offset, f_should_read_directly)

OSError: [Errno 22] Invalid argument

我在进入路径时也尝试过以下操作，但没有成功：

  train_set = Database_load(root = r"C:\Users\Public\PhysNet\",train = "train.pth")

  train_set = Database_load(root = r'C:/Users/Public/PhysNet/',train = 'train.pth')

关于如何解决这个问题有什么建议吗？

Answer 1

这是一个已知问题（Issue#26998 and Issue#) caused by PR#20900. The problem happens because you're trying to load a file larger than 2GB, and it is specific to Windows in which "sizeof(long)=4 for both 32-bit and 64-system systems". This issue was fixed by PR#27069，仅适用于 PyTorch 1.3+。因此，要解决此问题，请升级您的 PyTorch 版本。

OSError: [Errno 22] Invalid argument when using torch.load

OSError: [Errno 22] Invalid argument when using torch.load

python

pytorch

oserror