更改多行数组的numpy矢量化方式（行可以重复）

Question

我运行在为 cs231n assignment1 实现矢量化 svm 梯度时遇到了这个问题。这是一个例子：

ary = np.array([[1,-9,0],
                [1,2,3],
                [0,0,0]])
ary[[0,1]] += np.ones((2,2),dtype='int')

它输出：

array([[ 2, -8,  1],
      [ 2,  3,  4],
      [ 0,  0,  0]])

一切都很好，直到行不是唯一的：

ary[[0,1,1]] += np.ones((3,3),dtype='int')

虽然没有报错，但是输出的结果真的很奇怪：

array([[ 2, -8,  1],
       [ 2,  3,  4],
       [ 0,  0,  0]])

我希望第二行应该是 [3,4,5] 而不是 [2,3,4]，我用来解决这个问题的天真的方法是使用这样的 for 循环：

ary = np.array([[ 2, -8,  1],
                [ 2,  3,  4],
                [ 0,  0,  0]])
# the rows I want to change
rows = [0,1,2,1,0,1]
# the change matrix
change = np.random.randn((6,3))
for i,row in enumerate(rows):
  ary[row] += change[i]

所以我真的不知道如何向量化这个 for 循环，在 NumPy 中有更好的方法吗？为什么这样做是错误的？:

ary[rows] += change

如果有人好奇我为什么要这样做，这是我对 svm_loss_vectorized 函数的实现，我需要根据标签 y:

计算权重的梯度

def svm_loss_vectorized(W, X, y, reg):
    """
    Structured SVM loss function, vectorized implementation.

    Inputs and outputs are the same as svm_loss_naive.
    """
    loss = 0.0
    dW = np.zeros(W.shape) # initialize the gradient as zero

    # transpose X and W
    # D means input dimensions, N means number of train example
    # C means number of classes
    # X.shape will be (D,N)
    # W.shape will be (C,D)
    X = X.T
    W = W.T
    dW = dW.T
    num_train = X.shape[1]
    # transpose W_y shape to (D,N) 
    W_y = W[y].T
    S_y = np.sum(W_y*X ,axis=0)
    margins =  np.dot(W,X) + 1 - S_y
    mask = np.array(margins>0)

    # get the impact of num_train examples made on W's gradient
    # that is,only when the mask is positive 
    # the train example has impact on W's gradient
    dW_j = np.dot(mask, X.T)
    dW +=  dW_j
    mul_mask = np.sum(mask, axis=0, keepdims=True).T

    # dW[y] -= mul_mask * X.T
    dW_y =  mul_mask * X.T
    for i,label in enumerate(y):
      dW[label] -= dW_y[i]

    loss = np.sum(margins*mask) - num_train
    loss /= num_train
    dW /= num_train
    # add regularization term
    loss += reg * np.sum(W*W)
    dW += reg * 2 * W
    dW = dW.T

    return loss, dW

Answer 1

使用内置 np.add.at

此类任务的内置是np.add.at，即

np.add.at(ary, rows, change)

但是，由于我们使用的是 2D 数组，这可能不是性能最高的数组。

杠杆速度快matrix-multiplication

事实证明，对于这种情况，我们也可以利用非常高效的 matrix-multplication，并且给定足够数量的重复行进行求和，这可能非常好。下面是我们如何使用它 -

mask = rows == np.arange(len(ary))[:,None]
ary += mask.dot(change)

基准测试

让我们用 np.add.at 方法来对抗基于 matrix-multiplication 的更大数组的方法 -

In [681]: ary = np.random.rand(1000,1000)

In [682]: rows = np.random.randint(0,len(ary),(10000))

In [683]: change = np.random.rand(10000,1000)

In [684]: %timeit np.add.at(ary, rows, change)
1 loop, best of 3: 604 ms per loop

In [687]: def matmul_addat(ary, rows, change):
     ...:     mask = rows == np.arange(len(ary))[:,None]
     ...:     ary += mask.dot(change)

In [688]: %timeit matmul_addat(ary, rows, change)
10 loops, best of 3: 158 ms per loop

更改多行数组的numpy矢量化方式（行可以重复）

numpy vectorized way to change multiple rows of array(rows can be repeated)

python

arrays

numpy

vectorization

svm