如何合并经过训练的模型和未经训练的模型？

Question

我很难做到以下几点：

我有一个模型，其中包含 125,089,410 个我已经训练过的可训练参数；
模型输出两个张量，形状为 (None, 96);
我想通过冻结之前模型的层来构建一个新模型，然后输出单个 (None, 96) 张量。

重要提示：我无意在我训练的原始模型中添加更多层。

这是我一直在尝试的：

def get_output_model (prev_model):

    # Freezing prev model
    for l in prev_model.layers:
        l.trainable = False

    # Compiling so it won't complain about parameters number
    prev_model.compile(loss='binary_crossentropy')

    # Sanity check
    print (prev_model.summary())

    # Loss function
    def loss_fn(y_true, y_pred):
        pass # This doesn't matter here

    # Building model
    out_model_in = prev_model.output
    out_model = tf.keras.layers.Dense(2 * MAX_LEN, activation='linear')(out_model_in)
    out_model = tf.keras.layers.Activation('relu')(out_model)
    out_model = tf.keras.layers.BatchNormalization()(out_model)
    out_model = tf.keras.layers.Dense(2 * MAX_LEN, activation='linear')(out_model)
    out_model = tf.keras.layers.Activation('relu')(out_model)
    out_model = tf.keras.layers.BatchNormalization()(out_model)
    out_model = tf.keras.layers.Dense(2 * MAX_LEN, activation='linear')(out_model)
    out_model = tf.keras.layers.Activation('relu')(out_model)
    out_model = tf.keras.layers.BatchNormalization()(out_model)
    out_model = tf.keras.layers.Dense(MAX_LEN, activation='linear')(out_model)
    out_model = tf.keras.layers.Activation('softmax')(out_model)
    model = tf.keras.models.Model(inputs=[out_model_in], outputs=[out_model])
    model.compile(loss=loss_fn, optimizer='nadam', metrics=['accuracy', f1_m])

但它给了我：

ValueError: Layer dense_49 expects 1 inputs, but it received 2 input tensors. Inputs received: [<tf.Tensor 'activation_57/Identity:0' shape=(None, 96) dtype=float32>, <tf.Tensor 'activation_58/Identity:0' shape=(None, 96) dtype=float32>]

我知道这个错误是预料之中的，但我不知道如何解决这个问题。

Answer 1

好吧，我找到了一种方法，它对我有用：

def get_output_model (prev_model):

    # Freezing prev model
    for l in prev_model.layers:
        l.trainable = False

    # Compiling so it won't complain about parameters number
    prev_model.compile(loss='binary_crossentropy')

    # Sanity check
    print (prev_model.summary())

    # Loss function
    def loss_fn(y_true, y_pred):
        pass # This doesn't matter here

    # Building model
    x1 = prev_model.output[0]
    x2 = prev_model.output[1]
    out_model = tf.keras.layers.Add()([x1, x2])
    out_model = tf.keras.layers.Dense(2 * MAX_LEN, activation='linear')(out_model)
    out_model = tf.keras.layers.Activation('relu')(out_model)
    out_model = tf.keras.layers.BatchNormalization()(out_model)
    out_model = tf.keras.layers.Dense(2 * MAX_LEN, activation='linear')(out_model)
    out_model = tf.keras.layers.Activation('relu')(out_model)
    out_model = tf.keras.layers.BatchNormalization()(out_model)
    out_model = tf.keras.layers.Dense(2 * MAX_LEN, activation='linear')(out_model)
    out_model = tf.keras.layers.Activation('relu')(out_model)
    out_model = tf.keras.layers.BatchNormalization()(out_model)
    out_model = tf.keras.layers.Dense(MAX_LEN, activation='linear')(out_model)
    out_model = tf.keras.layers.Activation('softmax')(out_model)
    model = tf.keras.models.Model(inputs=[prev_model.input], outputs=[out_model])
    model.compile(loss=loss_fn, optimizer='nadam', metrics=['accuracy', f1_m])
    return model

如何合并经过训练的模型和未经训练的模型？

How to merge a trained model and an untrained one?

python

keras

tensorflow

tf.keras

tensorflow2.0