在 PyTorch 中复制子张量

Question

我有一个张量“image_features”，形状为 torch.Size([100, 1024, 14, 14])。我需要将每个子张量 (1024, 14, 14) 复制 10 次，获得形状为 torch.Size([1000, 1024, 14, 14]) 的张量。

基本上，结果张量的前十行应对应原始张量的第一行，结果张量的后十行应对应原始张量的第二行，依此类推。如果可能的话，我不想创建一个副本（每个复制的子张量都可以与它复制的张量共享内存），但如果没有其他办法，创建一个副本也是可以的。

我该怎么做？

非常感谢。

Answer 1

这是一种使用 tensor.repeat() 的方法，涉及数据复制：

# sample tensor for us to work with
In [89]: shp = (100, 1024, 14, 14)
In [90]: t = torch.randn(shp)

# number of desired repetitions
In [91]: reps = 10

# all the magic happens here
# 10 -> we wish to repeat the entries `reps` times along first dimension
# 1 -> we don't want to repeat along the rest of the dimensions
In [92]: rep_tensor = t.repeat(reps, 1, 1, 1).view(-1, *shp[1:])

In [93]: rep_tensor.shape
Out[93]: torch.Size([1000, 1024, 14, 14])

这是一个简单的完整性检查示例：

In [109]: shp = (1, 3, 2)
In [110]: t = torch.randn(shp)

In [111]: t
Out[111]: 
tensor([[[-0.8974,  0.7790],
         [-0.0637, -1.0532],
         [-0.1682, -0.1921]]])

# repeat 3 times along axis 0
In [112]: rep_tensor = t.repeat(3, 1, 1).view(-1, *shp[1:])

In [113]: rep_tensor
Out[113]: 
tensor([[[-0.8974,  0.7790],
         [-0.0637, -1.0532],
         [-0.1682, -0.1921]],

        [[-0.8974,  0.7790],
         [-0.0637, -1.0532],
         [-0.1682, -0.1921]],

        [[-0.8974,  0.7790],
         [-0.0637, -1.0532],
         [-0.1682, -0.1921]]])

Answer 2

另一种可以解决您的问题的方法是：

orig_shape = (100, 1024, 14, 14)
new_shape = (100, 10, 1024, 14, 14)
input = torch.randn(orig_shape) # [100, 1024, 14, 14]
input = input.unsqueeze(1) # [100, 1, 1024, 14, 14]
input = input.expand(*new_shape) # [100, 10, 1024, 14, 14]
input = input.transpose(0, 1).contiguous() # [10, 100, 1024, 14, 14]
input = input.view(-1, *orig_shape[1:]) # [1000, 1024, 14, 14]

我们可以验证一下。

orig_shape = (2, 3, 4)
new_shape = (2, 5, 3, 4)
input = torch.randn(orig_shape)
print(input)
input = input.unsqueeze(1)
input = input.expand(*new_shape)
input = input.transpose(0, 1).contiguous()
input = input.view(-1, *orig_shape[1:])
print(input)

代码片段导致：

tensor([[[-1.1728,  1.0421, -1.0716,  0.6456],
     [-1.2214,  1.1484, -0.1436,  1.2353],
     [-0.4395, -0.9473, -0.1382, -0.9357]],

    [[-0.4735, -1.4329, -0.0025, -0.6384],
     [ 0.5102,  0.7813,  1.2810, -0.6013],
     [ 0.6152,  1.1734, -0.4591, -1.7447]]])

tensor([[[-1.1728,  1.0421, -1.0716,  0.6456],
     [-1.2214,  1.1484, -0.1436,  1.2353],
     [-0.4395, -0.9473, -0.1382, -0.9357]],

    [[-0.4735, -1.4329, -0.0025, -0.6384],
     [ 0.5102,  0.7813,  1.2810, -0.6013],
     [ 0.6152,  1.1734, -0.4591, -1.7447]],

    [[-1.1728,  1.0421, -1.0716,  0.6456],
     [-1.2214,  1.1484, -0.1436,  1.2353],
     [-0.4395, -0.9473, -0.1382, -0.9357]],

    [[-0.4735, -1.4329, -0.0025, -0.6384],
     [ 0.5102,  0.7813,  1.2810, -0.6013],
     [ 0.6152,  1.1734, -0.4591, -1.7447]]])

Answer 3

Pytorch 有一个单行解决方案来实现这一点：

a =  torch.randn(100,1024,14,14) 
b = torch.repeat_interleave(a,10,dim=0)


#a.size() --> torch.Size([100, 1024, 14, 14])
#b.size() --> torch.Size([1000, 1024, 14, 14])

repeat_interleave() 在给定轴上重复给定次数的值 - 在轴 0 上重复 10 次。它具有以下函数定义：

torch.repeat_interleave(input, repeats, dim=None)

在 PyTorch 中复制子张量

Replicate subtensors in PyTorch

python

deep-learning

pytorch

tensor