Tensorflow InvalidArgumentError: 2 root error(s) found. indices[28,0] = 11292 is not in [0, 11272)

Question

我使用 Keras Sequential API 和 Glove pretraining embeddings

创建了一个模型

def create_model(
        input_length=20,
        output_length=20):

    encoder_input = tf.keras.Input(shape=(input_length,))
    decoder_input = tf.keras.Input(shape=(output_length,))

    encoder = tf.keras.layers.Embedding(original_embedding_matrix.shape[0], original_embedding_dim, weights=[original_embedding_matrix], mask_zero=True)(encoder_input)
    encoder, h_encoder, u_encoder = tf.keras.layers.LSTM(64, return_state=True)(encoder)

    decoder = tf.keras.layers.Embedding(clone_embedding_matrix.shape[0], clone_embedding_dim, weights=[clone_embedding_matrix], mask_zero=True)(decoder_input)
    decoder = tf.keras.layers.LSTM(64, return_sequences=True)(decoder, initial_state=[h_encoder, u_encoder])
    decoder = tf.keras.layers.TimeDistributed(tf.keras.layers.Dense(clone_vocab_size+1))(decoder)

    model = tf.keras.Model(inputs=[encoder_input, decoder_input], outputs=[decoder])
    model.compile(optimizer='adam', loss=tf.keras.losses.MeanSquaredError(), metrics=['accuracy'])

    return model

model = create_model()

这是我的 encoder/decoder 形状：

training_encoder_input.shape --> (2500, 20) 
training_decoder_input.shape --> (2500, 20) 
training_decoder_output.shape ---> (2500, 20, 11272) 
clone_vocab_size ---> 11271

model.summary() 的输出：

Model: "functional_1"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_1 (InputLayer)            [(None, 20)]         0                                            
__________________________________________________________________________________________________
input_2 (InputLayer)            [(None, 20)]         0                                            
__________________________________________________________________________________________________
embedding (Embedding)           (None, 20, 50)       564800      input_1[0][0]                    
__________________________________________________________________________________________________
embedding_1 (Embedding)         (None, 20, 50)       563600      input_2[0][0]                    
__________________________________________________________________________________________________
lstm (LSTM)                     [(None, 64), (None,  29440       embedding[0][0]                  
__________________________________________________________________________________________________
lstm_1 (LSTM)                   (None, 20, 64)       29440       embedding_1[0][0]                
                                                                 lstm[0][1]                       
                                                                 lstm[0][2]                       
__________________________________________________________________________________________________
time_distributed (TimeDistribut (None, 20, 11272)    732680      lstm_1[0][0]                     
==================================================================================================
Total params: 1,919,960
Trainable params: 1,919,960
Non-trainable params: 0
__________________________________________________________________________________________________

但是当我尝试训练模型时：

model.fit(x=[training_encoder_input, training_decoder_input],
          y=training_decoder_output,
          verbose=2,
          batch_size=128,
          epochs=10)

我收到这个错误：

InvalidArgumentError: 2 root error(s) found.
  (0) Invalid argument:  indices[28,0] = 11292 is not in [0, 11272)
     [[node functional_1/embedding_1/embedding_lookup (defined at <ipython-input-11-967d0351a90e>:31) ]]
  (1) Invalid argument:  indices[28,0] = 11292 is not in [0, 11272)
     [[node functional_1/embedding_1/embedding_lookup (defined at <ipython-input-11-967d0351a90e>:31) ]]
     [[broadcast_weights_1/assert_broadcastable/AssertGuard/else/_13/broadcast_weights_1/assert_broadcastable/AssertGuard/Assert/data_7/_78]]
0 successful operations.
0 derived errors ignored. [Op:__inference_train_function_13975]

Errors may have originated from an input operation.
Input Source operations connected to node functional_1/embedding_1/embedding_lookup:
 functional_1/embedding_1/embedding_lookup/8859 (defined at /usr/lib/python3.6/contextlib.py:81)

Input Source operations connected to node functional_1/embedding_1/embedding_lookup:
 functional_1/embedding_1/embedding_lookup/8859 (defined at /usr/lib/python3.6/contextlib.py:81)

Function call stack:
train_function -> train_function

已经有人问过 this question 但 none 的回复对我有用，可能错误在损失函数内或嵌入层的词汇表内，但我无法弄清楚究竟是什么问题。

Answer 1

解决方法其实很简单，在错误中：

(0) Invalid argument:  indices[28,0] = 11292 is not in [0, 11272)

11292 是一个输入元素（映射到我的 Tokenizer 词典中的一个词）
11272 是我的词汇量

如果我的分词器的长度只是 11272，为什么我有一个数字 11292 的词？

我有两个分词器，一个用于输入，另一个用于输出，所以一个解决方案是利用较小的 ans 在模型中使用它。

您还可以限制 Tensorflow 分词器中使用的单词数：

tokenizer = Tokenizer(num_words=20000)

它会取20000个重复次数最多的单词。

Tensorflow InvalidArgumentError: 2 root error(s) found. indices[28,0] = 11292 is not in [0, 11272)

Tensorflow InvalidArgumentError: 2 root error(s) found. indices[28,0] = 11292 is not in [0, 11272)

python

keras

tensorflow

loss-function