连接 2 个 Keras 层的输出

Question

我正在尝试使用 Keras 实现联合模型，这是模型的架构。

但是，我很难将子网络和主网络的输入串联起来。以下是我的代码：

import tensorflow as tf
from tensorflow.keras import layers
from tensorflow.keras.datasets import mnist
from tensorflow.keras.models import Model, Sequential
from tensorflow.keras.layers import Conv2D, MaxPooling2D, Dense, Flatten, Reshape, Concatenate


def Autoencoder():
  input = layers.Input(shape=(256, 256, 5))
  layers.Flatten()

  x = layers.Conv2D(32, (3, 3), activation="relu", padding="same")(input)
  x = layers.Conv2D(32, (3, 3), activation="relu", padding="same")(x)
  x = layers.MaxPooling2D((2, 2), padding="same")(x)

  x = layers.Conv2D(64, (3, 3), activation="relu", padding="same")(x)
  x = layers.Conv2D(64, (3, 3), activation="relu", padding="same")(x)
  x = layers.MaxPooling2D((2, 2), padding="same")(x)

  x = layers.Conv2D(128, (3, 3), activation="relu", padding="same")(x)
  x = layers.Conv2D(128, (3, 3), activation="relu", padding="same")(x)
  x = layers.MaxPooling2D((2, 2), padding="same", name='last_layer')(x)
  autoencoder = Model(input, x)
  return autoencoder.get_layer('last_layer')



def Subnetwork():
  input = layers.Input(shape=(12,1))

  x = layers.Flatten()(input)
  x = layers.Dense(4096, activation="relu")(x)
  x = layers.Reshape((32, 32, 4), name='last_layer')(x)
  subnetwork = Model(input, x)
  return subnetwork.get_layer('last_layer')

def Joint():
  layer_autoencoder = Autoencoder()
  layer_subnetwork = Subnetwork()
  merged= Concatenate([layer_autoencoder, layer_subnetwork])
  model = Model(inputs=[layer_autoencoder, layer_subnetwork], outputs=merged)
  return model

Model = Joint()
Model.summary()

错误消息如下所示：

ValueError: Found unexpected instance while processing input tensors for keras functional model. Expecting KerasTensor which is from tf.keras.Input() or output from keras layer call(). Got: <keras.layers.pooling.MaxPooling2D object at 0x7fbfd8634990>

有谁知道导致错误的原因以及正确的解决方案是什么？

Answer 1

您还应该在 Joint 模型中重新定义输入层

def Joint():

  input_autoencoder = layers.Input(shape=(256, 256, 5))  ### define input layer
  layer_autoencoder = Autoencoder()(input_autoencoder)  ### pass input to Autoencoder
  input_subnetwork = layers.Input(shape=(12, 1))  ### define input layer
  layer_subnetwork = Subnetwork()(input_subnetwork)  ### pass input to Subnetwork
  
  merged= Concatenate()([layer_autoencoder, layer_subnetwork])  ### it's Concatenate()([...]) not Concatenate([...])
  model = Model([input_autoencoder, input_subnetwork], merged)  ### use correct inputs
  return model

另请注意 Autoencoder 和 Subnetwork。他们必须 return 个 TF 模型实例。所以他们变成了：

def Autoencoder():
  ...
  autoencoder = Model(input, x)
  return autoencoder

def Subnetwork():
  ...
  subnetwork = Model(input, x)
  return subnetwork

连接 2 个 Keras 层的输出

Joining the output of 2 Keras layers

python

machine-learning

deep-learning

keras

tensorflow