Keras: Dense vs. Embedding - ValueError: Input 0 is incompatible with layer repeat_vector_9: expected ndim=2, found ndim=3

Question

我有以下工作正常的网络：

left = Sequential()
left.add(Dense(EMBED_DIM,input_shape=(ENCODE_DIM,)))
left.add(RepeatVector(look_back))

但是，我需要用嵌入层替换密集层：

left = Sequential()
left.add(Embedding(ENCODE_DIM, EMBED_DIM, input_length=1))
left.add(RepeatVector(look_back))

然后我在使用Embedding层时出现如下错误：

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-119-5a5f11c97e39> in <module>()
     29 left.add(Embedding(ENCODE_DIM, EMBED_DIM, input_length=1))
---> 30 left.add(RepeatVector(look_back))
     31 
     32 leftOutput = left.output

/usr/local/lib/python3.4/dist-packages/keras/models.py in add(self, layer)
    467                           output_shapes=[self.outputs[0]._keras_shape])
    468         else:
--> 469             output_tensor = layer(self.outputs[0])
    470             if isinstance(output_tensor, list):
    471                 raise TypeError('All layers in a Sequential model '

/usr/local/lib/python3.4/dist-packages/keras/engine/topology.py in __call__(self, inputs, **kwargs)
    550                 # Raise exceptions in case the input is not compatible
    551                 # with the input_spec specified in the layer constructor.
--> 552                 self.assert_input_compatibility(inputs)
    553 
    554                 # Collect input shapes to build layer.

/usr/local/lib/python3.4/dist-packages/keras/engine/topology.py in assert_input_compatibility(self, inputs)
    449                                      self.name + ': expected ndim=' +
    450                                      str(spec.ndim) + ', found ndim=' +
--> 451                                      str(K.ndim(x)))
    452             if spec.max_ndim is not None:
    453                 ndim = K.ndim(x)

ValueError: Input 0 is incompatible with layer repeat_vector_9: expected ndim=2, found ndim=3

用 Embedding 层替换 Dense 层时，我需要做哪些额外更改？谢谢！

Answer 1

Dense层的输出形状是(None, EMBED_DIM)。然而，Embedding层的输出形状是(None, input_length, EMBED_DIM)。对于 input_length=1，它将是 (None, 1, EMBED_DIM)。您可以在 Embedding 层之后添加一个 Flatten 层以删除轴 1。

您可以打印出输出形状来调试您的模型。例如，

EMBED_DIM = 128
left = Sequential()
left.add(Dense(EMBED_DIM, input_shape=(ENCODE_DIM,)))
print(left.output_shape)
(None, 128)

left = Sequential()
left.add(Embedding(ENCODE_DIM, EMBED_DIM, input_length=1))
print(left.output_shape)
(None, 1, 128)

left.add(Flatten())
print(left.output_shape)
(None, 128)

Keras: Dense vs. Embedding - ValueError: Input 0 is incompatible with layer repeat_vector_9: expected ndim=2, found ndim=3

Keras: Dense vs. Embedding - ValueError: Input 0 is incompatible with layer repeat_vector_9: expected ndim=2, found ndim=3

embedding

keras

word-embedding

keras-layer

keras-2