在 tensorflow-hub 预训练模型后添加 LSTM 层

Question

我正在使用 Tensorflow-hub 预训练的 Word2vec 模型进行文本分类。我正在寻求为 keras 模型添加一个 LSTM 层。为此，我使用了以下代码：

model = tf.keras.models.Sequential()
model.add(hub.KerasLayer(hub.load('https://tfhub.dev/google/Wiki-words-250/2'), 
                        input_shape=[], 
                        dtype=tf.string, 
                        trainable=True))

添加 LSTM 层后：

model.add(tf.keras.layers.LSTM(32))

它显示了以下错误：

~\anaconda3\lib\site-packages\tensorflow\python\keras\engine\input_spec.py in assert_input_compatibility(input_spec, inputs, layer_name)
    174       ndim = x.shape.ndims
    175       if ndim != spec.ndim:
--> 176         raise ValueError('Input ' + str(input_index) + ' of layer ' +
    177                          layer_name + ' is incompatible with the layer: '
    178                          'expected ndim=' + str(spec.ndim) + ', found ndim=' +

ValueError: Input 0 of layer lstm_0 is incompatible with the layer: expected ndim=3, found ndim=2. Full shape received: [None, 250]

任何帮助都是可观的。

Answer 1

您可以重塑 hub.KerasLayer 的输出：

model.add(hub.KerasLayer(hub.load('https://tfhub.dev/google/Wiki-words-250/2'), 
                        input_shape=[], 
                        dtype=tf.string, 
                        trainable=True))

model.add(tf.keras.layers.Reshape((250, 1)))
model.add(tf.keras.layers.LSTM(32))

model.summary()

Layer (type)                 Output Shape              Param #   
=================================================================
keras_layer_4 (KerasLayer)   (None, 250)               252343750 
_________________________________________________________________
reshape_2 (Reshape)          (None, 250, 1)            0         
_________________________________________________________________
lstm_2 (LSTM)                (None, 32)                4352      
=================================================================
Total params: 252,348,102
Trainable params: 252,348,102
Non-trainable params: 0

在 tensorflow-hub 预训练模型后添加 LSTM 层

Add LSTM layers after tensorflow-hub pretrained model

python

lstm

keras

tensorflow-hub