TF2.0: Transformer model ValueError: Shapes (40759, 128) and (40765, 128) are incompatible

Question

Transformer 模型具有以下参数。我使用 h5py 保存并重新加载了模型。我只为少数数据集收到此错误。

h5f = h5py.File(path + '.model.weights.h5', 'w')

# Weights reloaded

variables = []
    h5f = h5py.File(path + '.model.weights.h5', 'r')
    for idx in sorted([int(i) for i in h5f]):
        variables.append(np.array(h5f[str(idx)]))
    h5f.close()
    for idx, t in enumerate(this.model.trainable_variables):
        t.assign(variables[idx])

训练模型的超参数是：

BUFFER_SIZE = 20000
BATCH_SIZE = 64
MAX_LENGTH = 40
num_layers = 4
d_model = 128
dff = 512
num_heads = 8
input_vocab_size = tokenizer_pt.vocab_size + 2
target_vocab_size = tokenizer_en.vocab_size + 2
dropout_rate = 0.1

transformer = Transformer(num_layers, d_model, num_heads, dff,
                          input_vocab_size, target_vocab_size, 
                          pe_input=input_vocab_size, 
                          pe_target=target_vocab_size,
                          rate=dropout_rate)

重新加载模型后，出现以下错误。我可以保存所有参数，但加载失败并出现不兼容错误。这些张量形状表示什么？

raise ValueError("Shapes %s and %s are incompatible" % (self, other))
ValueError: Shapes (40759, 128) and (40765, 128) are incompatible

回溯：

File "/Users/Models/Model.py", line 400, in load
    t.assign(modelTrainables[idx])
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tensorflow_core/python/ops/resource_variable_ops.py", line 600, in assign
    self._shape.assert_is_compatible_with(value_tensor.shape)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tensorflow_core/python/framework/tensor_shape.py", line 700, in assert_is_compatible_with
    raise ValueError("Shapes %s and %s are incompatible" % (self, other))
ValueError: Shapes (40759, 128) and (40765, 128) are incompatible

Answer 1

问题在于加载分词器。使用保存功能时会保存一些标点符号，但在加载时，其中一些无法加载（不知道根本原因）并导致形状不匹配。在我清理文本以删除标点符号和多余空格后，它工作正常。

TF2.0: Transformer model ValueError: Shapes (40759, 128) and (40765, 128) are incompatible

TF2.0: Transformer model ValueError: Shapes (40759, 128) and (40765, 128) are incompatible

python

nlp

tensorflow2.0