tensorflow InvalidArgumentError indices[0,x = y is not in [0, z) ]
tensorflow InvalidArgumentError indices[0,x = y is not in [0, z)
我正在使用 tensorflow.keras 来判断一封电子邮件是关于使用电子邮件发件人、主题和内容的。
# Use tokenizers to change email data to model input series
subject_sequence = subject_tk.texts_to_sequences(subject_series)
subject_sequence = sequence.pad_sequences(subject_sequence, maxlen = subject_length)
sender_sequence = subject_tk.texts_to_sequences(sender_series)
sender_sequence = sequence.pad_sequences(sender_sequence, maxlen = sender_length)
body_sequence = body_tk.texts_to_sequences(body_series)
body_sequence = sequence.pad_sequences(body_sequence, maxlen = body_length)
# Run learning model on input series and make predication
predication = email_classification_model.predict([subject_sequence, sender_sequence , body_sequence])
print(predication)
但是，我注意到有时，比如 10%，模型会失败并出现以下错误：
File "mailMonitory.py", line 102, in OnItemAdd
predication = email_classification_model.predict([subject_sequence, sender_sequence , body_sequence])
....
tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
tensorflow.python.framework.errors_impl.InvalidArgumentError: indices[0,253] = 3686 is not in [0, 1897)
[[node model/embedding_1/embedding_lookup (defined at mailMonitory.py:102) ]] [Op:__inference_predict_function_4617]
print(sender_sequence)
[[ 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
693 3686 139 169]]
根据我的测试，问题始终是我的分词器将发件人电子邮件转换为包含模型超出范围的数字的系列。
为什么会这样？我的分词器没有包含足够的数据还是我的模式有问题？我该如何解决这个问题？
当您向 Embedding 层输入超出定义的 input_dim 大小时的整数值时，通常会出现此错误。例如，第一个序列有效，因为所有整数值都小于 input_dim。第二个序列抛出异常，因为几乎所有值都在可能的整数范围之外：
import tensorflow as tf
input = tf.keras.layers.Input(shape=(5,))
output = tf.keras.layers.Embedding(input_dim=10, output_dim=5)(input)
model = tf.keras.models.Model(input,output)
print(model(tf.constant([1, 5, 2, 6, 8])))
print(model(tf.constant([1, 12, 18, 19, 10, 4000])))
tf.Tensor(
[[-0.03517901 0.01769676 0.01823583 0.01846877 -0.01214858]
[-0.04662237 -0.01376029 0.04361605 0.0426343 -0.01796628]
[ 0.020581 0.02564194 0.00014243 0.03558977 0.01154976]
[-0.01251727 0.00095896 0.00218729 -0.01606169 0.02248188]
[ 0.03368715 0.01532438 -0.01821761 0.00139984 0.00360139]], shape=(5, 5), dtype=float32)
---------------------------------------------------------------------------
InvalidArgumentError Traceback (most recent call last)
<ipython-input-3-cd27383a1b70> in <module>()
6
7 print(model(tf.constant([1, 5, 2, 6, 8])))
----> 8 print(model(tf.constant([1, 12, 18, 19, 10, 4000])))
...
InvalidArgumentError: Exception encountered when calling layer "embedding_1" (type Embedding).
indices[1] = 12 is not in [0, 10) [Op:ResourceGather]
Call arguments received:
• inputs=tf.Tensor(shape=(6,), dtype=float32)
因此，解决方案是确保您为 input_dim 参数使用正确的尺寸。
我正在使用 tensorflow.keras 来判断一封电子邮件是关于使用电子邮件发件人、主题和内容的。
# Use tokenizers to change email data to model input series
subject_sequence = subject_tk.texts_to_sequences(subject_series)
subject_sequence = sequence.pad_sequences(subject_sequence, maxlen = subject_length)
sender_sequence = subject_tk.texts_to_sequences(sender_series)
sender_sequence = sequence.pad_sequences(sender_sequence, maxlen = sender_length)
body_sequence = body_tk.texts_to_sequences(body_series)
body_sequence = sequence.pad_sequences(body_sequence, maxlen = body_length)
# Run learning model on input series and make predication
predication = email_classification_model.predict([subject_sequence, sender_sequence , body_sequence])
print(predication)
但是，我注意到有时，比如 10%，模型会失败并出现以下错误：
File "mailMonitory.py", line 102, in OnItemAdd
predication = email_classification_model.predict([subject_sequence, sender_sequence , body_sequence])
....
tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
tensorflow.python.framework.errors_impl.InvalidArgumentError: indices[0,253] = 3686 is not in [0, 1897)
[[node model/embedding_1/embedding_lookup (defined at mailMonitory.py:102) ]] [Op:__inference_predict_function_4617]
print(sender_sequence)
[[ 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
693 3686 139 169]]
根据我的测试，问题始终是我的分词器将发件人电子邮件转换为包含模型超出范围的数字的系列。 为什么会这样？我的分词器没有包含足够的数据还是我的模式有问题？我该如何解决这个问题？
当您向 Embedding 层输入超出定义的 input_dim 大小时的整数值时，通常会出现此错误。例如，第一个序列有效，因为所有整数值都小于 input_dim。第二个序列抛出异常，因为几乎所有值都在可能的整数范围之外：
import tensorflow as tf
input = tf.keras.layers.Input(shape=(5,))
output = tf.keras.layers.Embedding(input_dim=10, output_dim=5)(input)
model = tf.keras.models.Model(input,output)
print(model(tf.constant([1, 5, 2, 6, 8])))
print(model(tf.constant([1, 12, 18, 19, 10, 4000])))
tf.Tensor(
[[-0.03517901 0.01769676 0.01823583 0.01846877 -0.01214858]
[-0.04662237 -0.01376029 0.04361605 0.0426343 -0.01796628]
[ 0.020581 0.02564194 0.00014243 0.03558977 0.01154976]
[-0.01251727 0.00095896 0.00218729 -0.01606169 0.02248188]
[ 0.03368715 0.01532438 -0.01821761 0.00139984 0.00360139]], shape=(5, 5), dtype=float32)
---------------------------------------------------------------------------
InvalidArgumentError Traceback (most recent call last)
<ipython-input-3-cd27383a1b70> in <module>()
6
7 print(model(tf.constant([1, 5, 2, 6, 8])))
----> 8 print(model(tf.constant([1, 12, 18, 19, 10, 4000])))
...
InvalidArgumentError: Exception encountered when calling layer "embedding_1" (type Embedding).
indices[1] = 12 is not in [0, 10) [Op:ResourceGather]
Call arguments received:
• inputs=tf.Tensor(shape=(6,), dtype=float32)
因此，解决方案是确保您为 input_dim 参数使用正确的尺寸。