如何将tensorflow中的dropout应用于多维张量？

Question

我有一个名为 X 的 3D 张量，形状为 [2,20,300]，我想仅将 dropout 应用于三维。但是，我希望丢弃的元素对于 20 个实例（第二维）是相同的，但对于第一维则不一定。

以下行为是什么：

tf.nn.dropout(X[0], keep_prob=p)

会不会只作用于我想要的维度？如果是这样，那么对于多个第一维度，我可以遍历它们并应用上面的行。

Answer 1

请参阅 tf.nn.dropout 的文档：

By default, each element is kept or dropped independently. If noise_shape is specified, it must be broadcastable to the shape of x, and only dimensions with noise_shape[i] == shape(x)[i] will make independent decisions

所以很简单：

import tensorflow as tf
import numpy as np

data = np.arange(300).reshape((1, 1, 300))
data = np.tile(data, (2, 20, 1))

data_op = tf.convert_to_tensor(data.astype(np.float32))
data_op = tf.nn.dropout(data_op, 0.5, noise_shape=[2, 1, 300])

with tf.Session() as sess:
    data = sess.run(data_op)

for b in range(2):
    for c in range(20):
        assert np.allclose(data[0, 0, :], data[0, c, :])
        assert np.allclose(data[1, 0, :], data[1, c, :])

print((data[0, 0, :] - data[1, 0, :]).sum())
# output something != 0 with high probability#

如何将tensorflow中的dropout应用于多维张量？

How to apply dropout in tensorflow to multidimensional tensors?

tensorflow

dimensions

tensor