来自 VGG19 的 Keras 自定义损失

Question

我想在 Keras 中使用使用 VGG19 网络的自定义损失函数。

# imports
from keras.applications import VGG19
from keras.layers import Input, UpSampling2D, Conv2D, Lambda
from keras.models import Model
import keras.backend as K

我使用以下方法构建 vgg 模型：

def build_vgg_loss():
    img = Input(shape=(300, 300, 3))
    vgg = VGG19(weights="imagenet", include_top=False)
    vgg.outputs = [
        # STOPS WORKING IF INCLUDING ALL, COMPILES WITH JUST 1 OUTPUT!!!
        vgg.get_layer('block2]1_conv1').output
        #         vgg.get_layer('block2_conv1').output,
        #         vgg.get_layer('block3_conv1').output
    ]
    model = Model(inputs=img, outputs=vgg(img))
    return model

我使用以下方法构建损失函数：

def build_loss_fn():

    vgg_model = build_vgg_loss()

    def loss(Ics, Is):

        loss = \
            K.sqrt(
                K.sum(
                    K.square(
                        vgg_model(Is)[0] - vgg_model(Ics)[0]
                    ), axis=-1
                )
            )

        return loss

    def full_loss(y_true, y_pred):
        Ics = K.expand_dims(y_pred[0], 0)
        Is = K.expand_dims(y_pred[1], 0)

        return loss(Ics, Is)

    return full_loss

编译虚拟模型

input1 = Input(shape=(300,300,3))
input2 = Input(shape=(300,300,3))

dummy_model = Model(inputs=[input1, input2], outputs=[input1, input2])
dummy_model.compile(
            loss=build_loss_fn(),
            optimizer=Adam(0.0001, 0.9),
)

然而，当我用这个自定义损失编译我的模型时，我得到了以下回溯，我不知道出了什么问题。

~/script.py in loss(Ics, Is)
    107                     K.sum(
    108                         K.square(
--> 109                             vgg_model(Is)[0] - vgg_model(Ics)[0]
    110                         ), axis=-1
    111                     )

/anaconda3/envs/ml36/lib/python3.6/site-packages/keras/engine/base_layer.py in __call__(self, inputs, **kwargs)
    472             if all([s is not None
    473                     for s in to_list(input_shape)]):
--> 474                 output_shape = self.compute_output_shape(input_shape)
    475             else:
    476                 if isinstance(input_shape, list):

/anaconda3/envs/ml36/lib/python3.6/site-packages/keras/engine/network.py in compute_output_shape(self, input_shape)
    655 
    656             for i, key in enumerate(output_shape_keys):
--> 657                 assert key in layers_to_output_shapes
    658                 output_shapes.append(layers_to_output_shapes[key])
    659             # Store in cache.

AssertionError:

我的猜测是 VGG 模型实例化不正确。如果我将 include_top 参数更改为 True 然后我得到错误

ValueError: The shape of the input to "Flatten" is not fully defined (got (None, None, 512). Make sure to pass a complete "input_shape" or "batch_input_shape" argument to the first layer in your model.

Answer 1

将 vgg_loss 模型实例化更改为以下内容：

def build_vgg_loss(self):
    vgg = VGG19(weights="imagenet", include_top=False, input_shape=(h,w,c))
    vgg.outputs = [
        vgg.get_layer('block1_conv1').output
    ]

    model = Model(inputs=vgg.inputs, outputs=vgg.outputs)
    model.trainable = False

    return model

来自 VGG19 的 Keras 自定义损失

Keras custom loss from VGG19

python

tensorflow

keras

vgg-net