如何将层添加到与检查点中的变量相关的预训练模型？

Question

我不习惯使用 TensorFlow 或 nn，所以如果我第一次不明白你们所说的，请原谅我。

我目前正在尝试在我从互联网上获得的 yolo v1 代码中的每个卷积层之后添加一个批量归一化层。

下面的代码是我使用的批量归一化函数。

def batchnorm(self, inp):
    with tf.variable_scope("batchnorm"):
        channels = inp.get_shape()[3]
        offset = tf.get_variable("offset",
                             channels,
                             dtype=tf.float32,
                             initializer=tf.zeros_initializer())
        scale = tf.get_variable("scale",
                            channels,
                            dtype=tf.float32,
                            initializer=tf.random_normal_initializer(1.0, 0.02))

        mean, variance = tf.nn.moments(inp, axes=[0, 1, 2], keep_dims=False)
        variance_epsilon = 1e-5
        normalized = tf.nn.batch_normalization(inp, mean, variance,
                                           offset, scale, variance_epsilon)
    return normalized

下面的代码是我使用的yolov1代码的结构

 if self.verbose:
        print('Building Yolo Graph....')
    # Reset default graph
    tf.reset_default_graph()
    # Input placeholder
    self.x = tf.placeholder('float32', [None, 448, 448, 3])
    self.label_batch = tf.placeholder('float32', [None, 73])

    # conv1, pool1
    self.conv1 = self.conv_layer(1, self.x, 64, 7, 2)
    self.pool1 = self.maxpool_layer(2, self.conv1, 2, 2)
    # size reduced to 64x112x112
    # conv2, pool2
    self.conv2 = self.conv_layer(3, self.pool1, 192, 3, 1)
    self.pool2 = self.maxpool_layer(4, self.conv2, 2, 2)
    # size reduced to 192x56x56
    # conv3, conv4, conv5, conv6, pool3
    self.conv3 = self.conv_layer(5, self.pool2, 128, 1, 1)
    self.conv4 = self.conv_layer(6, self.conv3, 256, 3, 1)
    self.conv5 = self.conv_layer(7, self.conv4, 256, 1, 1)
    self.conv6 = self.conv_layer(8, self.conv5, 512, 3, 1)
    self.pool3 = self.maxpool_layer(9, self.conv6, 2, 2)
    # size reduced to 512x28x28
    # conv7 - conv16, pool4
    self.conv7 = self.conv_layer(10, self.pool3, 256, 1, 1)
    self.conv8 = self.conv_layer(11, self.conv7, 512, 3, 1)
    self.conv9 = self.conv_layer(12, self.conv8, 256, 1, 1)
    self.conv10 = self.conv_layer(13, self.conv9, 512, 3, 1)
    self.conv11 = self.conv_layer(14, self.conv10, 256, 1, 1)
    self.conv12 = self.conv_layer(15, self.conv11, 512, 3, 1)
    self.conv13 = self.conv_layer(16, self.conv12, 256, 1, 1)
    self.conv14 = self.conv_layer(17, self.conv13, 512, 3, 1)
    self.conv15 = self.conv_layer(18, self.conv14, 512, 1, 1)
    self.conv16 = self.conv_layer(19, self.conv15, 1024, 3, 1)
    self.pool4 = self.maxpool_layer(20, self.conv16, 2, 2)
    # size reduced to 1024x14x14
    # conv17 - conv24
    self.conv17 = self.conv_layer(21, self.pool4, 512, 1, 1)
    self.conv18 = self.conv_layer(22, self.conv17, 1024, 3, 1)
    self.conv19 = self.conv_layer(23, self.conv18, 512, 1, 1)
    self.conv20 = self.conv_layer(24, self.conv19, 1024, 3, 1)
    self.conv21 = self.conv_layer(25, self.conv20, 1024, 3, 1)
    self.conv22 = self.conv_layer(26, self.conv21, 1024, 3, 2)
    self.conv23 = self.conv_layer(27, self.conv22, 1024, 3, 1)
    self.conv24 = self.conv_layer(28, self.conv23, 1024, 3, 1)

    # size reduced to 1024x7x7
    # fc1, fc2, fc3
    self.fc1 = self.fc_layer(29, self.conv24, 512,
                             flatten=True, linear=False)
    self.fc2 = self.fc_layer(
        30, self.fc1, 4096, flatten=False, linear=False)
    self.fc3 = self.fc_layer(
        31, self.fc2, 1470, flatten=False, linear=True)

    varlist = self.print_tensors_in_checkpoint_file(file_name=self.weightFile, all_tensors=True, tensor_name=None)
    variables = tf.get_collection(tf.GraphKeys.GLOBAL_VARIABLES)
    self.saver = tf.train.Saver(variables[:len(varlist)])

    self.loss = self.calculate_loss_function(self.fc3 , self.label_batch)

    self.sess = tf.Session()

    self.saver.restore(self.sess, self.weightFile)

    self.only_restore_conv20 = False
    if self.only_restore_conv20:
        after_20_initializer = [var.initializer for var in tf.global_variables()[3:]]
        self.sess.run(after_20_initializer)

    #exerpath = 'C:/Users/dml/PycharmProjects/YOLOv1-master/exer_ckpt/exer.ckpt'

    self.training = tf.train.MomentumOptimizer(momentum=0.5, learning_rate=1e-4).minimize(self.loss)

    Momentum_initializers = [var.initializer for var in tf.global_variables() if 'Momentum' in var.name]

    self.sess.run(Momentum_initializers)

最后我在将 batchnorm 层放在 conv1 层之后得到的错误

self.conv1 = self.conv_layer(1, self.x, 64, 7, 2)
    self.bn1 = self.batchnorm(self.conv1)
    self.pool1 = self.maxpool_layer(2, self.bn1, 2, 2)

是

NotFoundError: Key batchnorm/offset not found in checkpoint
 [[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, ..., DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]

经过几天的挣扎，我发现它与恢复检查点文件中的权重有关。而且因为我的 batchnorm 变量不在检查点文件中。但是我找不到如何让我的代码工作。

Answer 1

你是对的，问题是当你加载一个检查点时，TensorFlow 想要恢复所有变量的值。如果在检查点文件中找不到某些变量，则会引发错误。

我猜你的检查点文件不包含新规范化层中的变量。如果是这样，这个检查点可能就没用了。在新的网络结构中使用预训练变量值可能会产生非常糟糕的结果（在每个转换层之后使用规范化层）。

如果您仍想尝试使用检查点文件中的预训练权重，您将需要自己从检查点加载变量值。假设变量名和形状没有改变，你应该可以在这个 gist 中使用 optimistic_restore 函数的一个版本。此要点显示了在创建检查点后添加图层的示例 - 您的具体情况。

如何将层添加到与检查点中的变量相关的预训练模型？

How to add layers to a pre-trained model related to variables in the checkpoint?

checkpoint

conv-neural-network

tensorflow

yolo