批量归一化中的维度
dimensions in batch normalization
我正在尝试在 Tensorflow 中构建一个通用的批量归一化函数。
我在这个 article 中学习了 batch normalization，我觉得很亲切。
我对 scale 和 beta 变量的维度有疑问：在我的例子中，批量归一化应用于每个激活每个卷积层，因此如果我有卷积层的输出一个大小为：
的三元
[57,57,96]
我需要 scale 和 beta 与卷积层输出具有相同的维度，对吗？
这是我的函数，程序可以运行，但我不知道是否正确
def batch_normalization_layer(batch):
# Calculate batch mean and variance
batch_mean, batch_var = tf.nn.moments(batch, axes=[0, 1, 2])
# Apply the initial batch normalizing transform
scale = tf.Variable(tf.ones([batch.get_shape()[1],batch.get_shape()[2],batch.get_shape()[3]]))
beta = tf.Variable(tf.zeros([batch.get_shape()[1],batch.get_shape()[2],batch.get_shape()[3]]))
normalized_batch = tf.nn.batch_normalization(batch, batch_mean, batch_var, beta, scale, 0.0001)
return normalized_batch
来自 tf.nn.batch_normalization 的文档：
mean, variance, offset and scale are all expected to be of one of two
shapes:
In all generality, they can have the same number of dimensions as the
input x, with identical sizes as x for the dimensions that are not
normalized over (the 'depth' dimension(s)), and dimension 1 for the
others which are being normalized over. mean and variance in this case
would typically be the outputs of tf.nn.moments(..., keep_dims=True)
during training, or running averages thereof during inference.
In the
common case where the 'depth' dimension is the last dimension in the
input tensor x, they may be one dimensional tensors of the same size
as the 'depth' dimension. This is the case for example for the common
[batch, depth] layout of fully-connected layers, and [batch, height,
width, depth] for convolutions. mean and variance in this case would
typically be the outputs of tf.nn.moments(..., keep_dims=False) during
training, or running averages thereof during inference.
使用您的值（scale=1.0 和 offset=0），您也可以只提供值 None。
我正在尝试在 Tensorflow 中构建一个通用的批量归一化函数。
我在这个 article 中学习了 batch normalization，我觉得很亲切。
我对 scale 和 beta 变量的维度有疑问：在我的例子中，批量归一化应用于每个激活每个卷积层，因此如果我有卷积层的输出一个大小为：
的三元[57,57,96]
我需要 scale 和 beta 与卷积层输出具有相同的维度，对吗？
这是我的函数，程序可以运行，但我不知道是否正确
def batch_normalization_layer(batch):
# Calculate batch mean and variance
batch_mean, batch_var = tf.nn.moments(batch, axes=[0, 1, 2])
# Apply the initial batch normalizing transform
scale = tf.Variable(tf.ones([batch.get_shape()[1],batch.get_shape()[2],batch.get_shape()[3]]))
beta = tf.Variable(tf.zeros([batch.get_shape()[1],batch.get_shape()[2],batch.get_shape()[3]]))
normalized_batch = tf.nn.batch_normalization(batch, batch_mean, batch_var, beta, scale, 0.0001)
return normalized_batch
来自 tf.nn.batch_normalization 的文档：
mean, variance, offset and scale are all expected to be of one of two shapes:
In all generality, they can have the same number of dimensions as the input x, with identical sizes as x for the dimensions that are not normalized over (the 'depth' dimension(s)), and dimension 1 for the others which are being normalized over. mean and variance in this case would typically be the outputs of tf.nn.moments(..., keep_dims=True) during training, or running averages thereof during inference.
In the common case where the 'depth' dimension is the last dimension in the input tensor x, they may be one dimensional tensors of the same size as the 'depth' dimension. This is the case for example for the common [batch, depth] layout of fully-connected layers, and [batch, height, width, depth] for convolutions. mean and variance in this case would typically be the outputs of tf.nn.moments(..., keep_dims=False) during training, or running averages thereof during inference.
使用您的值（scale=1.0 和 offset=0），您也可以只提供值 None。