使用嵌入层序列化 keras 模型

Question

我已经用这样的预训练词嵌入训练了一个模型：

embedding_matrix = np.zeros((vocab_size, 100))
for word, i in text_tokenizer.word_index.items():
    embedding_vector = embeddings_index.get(word)
    if embedding_vector is not None:
        embedding_matrix[i] = embedding_vector

embedding_layer = Embedding(vocab_size,
                        100,
                        embeddings_initializer=Constant(embedding_matrix),
                        input_length=50,
                        trainable=False)

架构如下：

sequence_input = Input(shape=(50,), dtype='int32')
embedded_sequences = embedding_layer(sequence_input)
text_cnn = Conv1D(filters=5, kernel_size=5, padding='same',     activation='relu')(embedded_sequences)
text_lstm = LSTM(500, return_sequences=True)(embedded_sequences)


char_in = Input(shape=(50, 18, ))
char_cnn = Conv1D(filters=5, kernel_size=5, padding='same', activation='relu')(char_in)
char_cnn = GaussianNoise(0.40)(char_cnn)
char_lstm = LSTM(500, return_sequences=True)(char_in)



merged = concatenate([char_lstm, text_lstm]) 

merged_d1 = Dense(800, activation='relu')(merged)
merged_d1 = Dropout(0.5)(merged_d1)

text_class = Dense(len(y_unique), activation='softmax')(merged_d1)
model = Model([sequence_input,char_in], text_class)

当我将模型转换为 json 时，出现此错误：

ValueError: can only convert an array of size 1 to a Python scalar

同样，如果我使用model.save()功能，它似乎正确保存，但当我加载它时，我得到Type Error: Expected Float32。

我的问题是：在尝试序列化此模型时是否遗漏了什么？我是否需要某种 Lambda 层或类似的东西？

如有任何帮助，我们将不胜感激！

Answer 1

您可以在 Embedding 层中使用 weights 参数来提供初始权重。

embedding_layer = Embedding(vocab_size,
                            100,
                            weights=[embedding_matrix],
                            input_length=50,
                            trainable=False)

在模型 saving/loading 之后权重应该保持不可训练：

model.save('1.h5')
m = load_model('1.h5')
m.summary()

__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to
==================================================================================================
input_3 (InputLayer)            (None, 50)           0
__________________________________________________________________________________________________
input_4 (InputLayer)            (None, 50, 18)       0
__________________________________________________________________________________________________
embedding_1 (Embedding)         (None, 50, 100)      1000000     input_3[0][0]
__________________________________________________________________________________________________
lstm_4 (LSTM)                   (None, 50, 500)      1038000     input_4[0][0]
__________________________________________________________________________________________________
lstm_3 (LSTM)                   (None, 50, 500)      1202000     embedding_1[0][0]
__________________________________________________________________________________________________
concatenate_2 (Concatenate)     (None, 50, 1000)     0           lstm_4[0][0]
                                                                 lstm_3[0][0]
__________________________________________________________________________________________________
dense_2 (Dense)                 (None, 50, 800)      800800      concatenate_2[0][0]
__________________________________________________________________________________________________
dropout_2 (Dropout)             (None, 50, 800)      0           dense_2[0][0]
__________________________________________________________________________________________________
dense_3 (Dense)                 (None, 50, 15)       12015       dropout_2[0][0]
==================================================================================================
Total params: 4,052,815
Trainable params: 3,052,815
Non-trainable params: 1,000,000
__________________________________________________________________________________________________

Answer 2

希望您在编译后保存模型。喜欢：

    model.compile(loss='categorical_crossentropy', optimizer='rmsprop', metrics=['accuracy'])

要保存模型，您可以这样做：

    from keras.models import load_model

    model.save('model.h5')
    model = load_model('model_detect1.h5')
    model_json = model.to_json()
    with open("model.json", "w") as json_file:
        json_file.write(model_json)

要加载模型，

    from keras.models import model_from_json

    json_file = open('model.json', 'r')
    model_json = json_file.read()
    model = model_from_json(model_json)
    model.load_weights("model.h5")

Answer 3

我尝试了多种方法。问题是当我们在嵌入层工作时，pickle 不起作用，并且无法保存数据。所以当你有像这样的层时你可以做什么：-

## Creating model
embedding_vector_features=100
model=Sequential()
model.add(Embedding(voc_size,embedding_vector_features,input_length=sent_length))
model.add(LSTM(100))
model.add(Dense(1,activation='sigmoid'))
model.compile(loss='binary_crossentropy',optimizer='adam',metrics=['accuracy'])
print(model.summary())

然后，你可以使用 h5 extension to d=save file，然后将其转换为json，model converetd to model2 here

from tensorflow.keras.models import load_model
model.save('model.h5')
model = load_model('model.h5')
model_json = model.to_json()
with open("model.json", "w") as json_file:
json_file.write(model_json)

加载数据：-

from tensorflow.keras.models import model_from_json
json_file = open('model.json', 'r')
model_json = json_file.read()
model2 = model_from_json(model_json)
model2.load_weights("model.h5")

使用嵌入层序列化 keras 模型

Seralizing a keras model with an embedding layer

tensorflow

keras

word-embedding