将包含通用句子编码器的模型保存为其嵌入

Question

我正在尝试保存一个模型，该模型使用来自 tf-hub 的 USE 作为其嵌入层，并在其上堆叠了一些 FFN。该模型似乎工作正常，但我在保存和加载模型时遇到问题。

disable_eager_execution()

embed = hub.Module(module_url)

def UniversalEmbedding(x):
    return embed(tf.squeeze(tf.cast(x, tf.string)))

input_text = Input(shape=[], dtype=tf.string)
response_text = Input(shape=[], dtype=tf.string)

text_embedding = Lambda(UniversalEmbedding, output_shape=(512, ))(input_text)

response_embedding = Lambda(UniversalEmbedding, output_shape=(512, ))(response_text)

response_embedding = Dense(512, activation='relu')(response_embedding)
response_embedding = Dense(512, activation='relu')(response_embedding)

score = Dot(axes=1, normalize=True)([text_embedding, response_embedding])

pred = Dense(2, activation='softmax')(score)

text_encoder = Model(inputs=[input_text], outputs=text_embedding)
response_encoder = Model(inputs=[response_text], outputs=response_embedding)
model = Model(inputs=[input_text, response_text], outputs=pred)

上面的代码是我构建模型的方式（它是一个双编码器模型，使用 USE 作为其编码器）。

我不得不禁用急切执行，因为 USE 似乎还不能在急切执行环境中工作。如果没有，并且如果有解决方法，我也非常感谢对此的任何帮助:)

模型通过以下代码训练并保存：

with tf.compat.v1.Session() as session:
    K.set_session(session)
    session.run(tf.compat.v1.global_variables_initializer())  
    session.run(tf.compat.v1.tables_initializer())
    history = model.fit_generator(generator=train_neg_sample_generator, 
                                  validation_data=val_neg_sample_generator, epochs=20,
                                  callbacks=[checkpointer, earlystopper], verbose=0)

并且当检查点中的权重（保存在 hdf5 文件中）加载到上面代码中定义的模型时，模型加载没有错误。所以下面的代码工作正常，只是因为架构 'model' 已经在上面定义了。

with tf.compat.v1.Session() as session:
    K.set_session(session)
    session.run(tf.compat.v1.global_variables_initializer())
    session.run(tf.compat.v1.tables_initializer())
    model.load_weights('./saved_models/weights.03-0.29.hdf5') 
    tf.keras.models.save_model(model, 'test_model2.hdf5') 
    predicts = model.predict([["how are you?", "how are you?", 'hi', 'my two favorites in one pic!'], ["i'm fine", "what the heck", 'hi', 'same!']])
    print(predicts)
    print(np.argmax(predicts, axis=1))

然后我尝试了两件事。首先，我尝试将架构保存为 json 格式，加载模型架构，然后加载权重，但没有成功。然后我尝试通过keras.models.save_model保存整个模型，但它也没有用。

在这两种情况下，他们都返回了 AttributeError: 模块 'tensorflow' 没有属性 'placeholder'

如何 save/load 整个模型（如果不是一次加载，单独加载架构/权重也可以）？

这是整个错误日志

AttributeError                            Traceback (most recent call last)
<ipython-input-31-47468f2533ad> in <module>()
      1 from keras.models import load_model
      2 
----> 3 model2 = load_model('testest.h5')

13 frames
/usr/local/lib/python3.6/dist-packages/keras/engine/saving.py in load_wrapper(*args, **kwargs)
    456                 os.remove(tmp_filepath)
    457             return res
--> 458         return load_function(*args, **kwargs)
    459 
    460     return load_wrapper

/usr/local/lib/python3.6/dist-packages/keras/engine/saving.py in load_model(filepath, custom_objects, compile)
    548     if H5Dict.is_supported_type(filepath):
    549         with H5Dict(filepath, mode='r') as h5dict:
--> 550             model = _deserialize_model(h5dict, custom_objects, compile)
    551     elif hasattr(filepath, 'write') and callable(filepath.write):
    552         def load_function(h5file):

/usr/local/lib/python3.6/dist-packages/keras/engine/saving.py in _deserialize_model(h5dict, custom_objects, compile)
    241         raise ValueError('No model found in config.')
    242     model_config = json.loads(model_config.decode('utf-8'))
--> 243     model = model_from_config(model_config, custom_objects=custom_objects)
    244     model_weights_group = h5dict['model_weights']
    245 

/usr/local/lib/python3.6/dist-packages/keras/engine/saving.py in model_from_config(config, custom_objects)
    591                         '`Sequential.from_config(config)`?')
    592     from ..layers import deserialize
--> 593     return deserialize(config, custom_objects=custom_objects)
    594 
    595 

/usr/local/lib/python3.6/dist-packages/keras/layers/__init__.py in deserialize(config, custom_objects)
    166                                     module_objects=globs,
    167                                     custom_objects=custom_objects,
--> 168                                     printable_module_name='layer')

/usr/local/lib/python3.6/dist-packages/keras/utils/generic_utils.py in deserialize_keras_object(identifier, module_objects, custom_objects, printable_module_name)
    145                     config['config'],
    146                     custom_objects=dict(list(_GLOBAL_CUSTOM_OBJECTS.items()) +
--> 147                                         list(custom_objects.items())))
    148             with CustomObjectScope(custom_objects):
    149                 return cls.from_config(config['config'])

/usr/local/lib/python3.6/dist-packages/keras/engine/network.py in from_config(cls, config, custom_objects)
   1041         # First, we create all layers and enqueue nodes to be processed
   1042         for layer_data in config['layers']:
-> 1043             process_layer(layer_data)
   1044 
   1045         # Then we process nodes in order of layer depth.

/usr/local/lib/python3.6/dist-packages/keras/engine/network.py in process_layer(layer_data)
   1027 
   1028             layer = deserialize_layer(layer_data,
-> 1029                                       custom_objects=custom_objects)
   1030             created_layers[layer_name] = layer
   1031 

/usr/local/lib/python3.6/dist-packages/keras/layers/__init__.py in deserialize(config, custom_objects)
    166                                     module_objects=globs,
    167                                     custom_objects=custom_objects,
--> 168                                     printable_module_name='layer')

/usr/local/lib/python3.6/dist-packages/keras/utils/generic_utils.py in deserialize_keras_object(identifier, module_objects, custom_objects, printable_module_name)
    147                                         list(custom_objects.items())))
    148             with CustomObjectScope(custom_objects):
--> 149                 return cls.from_config(config['config'])
    150         else:
    151             # Then `cls` may be a function returning a class.

/usr/local/lib/python3.6/dist-packages/keras/engine/base_layer.py in from_config(cls, config)
   1101             A layer instance.
   1102         """
-> 1103         return cls(**config)
   1104 
   1105     def count_params(self):

/usr/local/lib/python3.6/dist-packages/keras/legacy/interfaces.py in wrapper(*args, **kwargs)
     89                 warnings.warn('Update your `' + object_name + '` call to the ' +
     90                               'Keras 2 API: ' + signature, stacklevel=2)
---> 91             return func(*args, **kwargs)
     92         wrapper._original_function = func
     93         return wrapper

/usr/local/lib/python3.6/dist-packages/keras/engine/input_layer.py in __init__(self, input_shape, batch_size, batch_input_shape, dtype, input_tensor, sparse, name)
     85                                          dtype=dtype,
     86                                          sparse=self.sparse,
---> 87                                          name=self.name)
     88         else:
     89             self.is_placeholder = False

/usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py in placeholder(shape, ndim, dtype, sparse, name)
    539         x = tf.sparse_placeholder(dtype, shape=shape, name=name)
    540     else:
--> 541         x = tf.placeholder(dtype, shape=shape, name=name)
    542     x._keras_shape = shape
    543     x._uses_learning_phase = False

AttributeError: module 'tensorflow' has no attribute 'placeholder'

Answer 1

提供整个错误日志，而不仅仅是其中的一部分。

如果真的因为保存出错那么model.save('model.h5')呢？不使用 tf.keras.models 中的模块，而是从 Model class 本身调用方法。

但是为什么要这些代码？

with tf.compat.v1.Session() as session:
    K.set_session(session)
    session.run(tf.compat.v1.global_variables_initializer())
    session.run(tf.compat.v1.tables_initializer())

我相信你可以马上调用 model.fit，你的 tf 是版本 2 对吗？为什么叫compat.v1？

Tensorflow 2 没有 Placeholder 所以我认为可能与此有关。

Answer 2

在 tensorflow 版本 1.15 上运行良好期待tf-hub与tensorflow 2.0和keras完全兼容...

将包含通用句子编码器的模型保存为其嵌入

Saving A Model That Contains Universal Sentence Encoder as its Embedding

tensorflow

tensorflow2.0

tf.keras

tensorflow-hub