NodeDef 提到 attr \'batch_dims\' not in Op in tensorflow serving

Question

我正在使用 keras 编写一个 ner 模型，并将模型部署到 tensorflow-serving。然后使用 http 请求来获取预测结果。

这是我的代码：

EMBEDDING_OUT_DIM = 128
TIME_STAMPS = 100
HIDDEN_UNITS = 200
DROPOUT_RATE = 0.3

def crf_model(VOCAB_SIZE, TAGS_NUMS):
    model = Sequential()
    model.add(Embedding(VOCAB_SIZE, output_dim=EMBEDDING_OUT_DIM, mask_zero=True, input_length=TIME_STAMPS))
    model.add(Bidirectional(LSTM(HIDDEN_UNITS, return_sequences=True)))
    model.add(Dropout(DROPOUT_RATE))
    model.add(Bidirectional(LSTM(HIDDEN_UNITS, return_sequences=True)))
    model.add(Dropout(DROPOUT_RATE))
    model.add(TimeDistributed(Dense(TAGS_NUMS)))
    crf = CRF(TAGS_NUMS, sparse_target=True)
    model.add(crf)
    model.summary()
    model.compile('rmsprop', loss=crf.loss_function, metrics=[crf.accuracy])
    return model

def save_model_to_serving(model, export_version, export_path='ner_serving'):
    print(model.input, model.output)
    signature = tf.saved_model.signature_def_utils.predict_signature_def( 
        inputs={'input_ids': model.input}, outputs={'outputs': model.output})
    export_path = os.path.join(
        tf.compat.as_bytes(export_path),
        tf.compat.as_bytes(str(export_version)))
    builder = tf.saved_model.builder.SavedModelBuilder(export_path)
    legacy_init_op = tf.group(tf.tables_initializer(), name='legacy_init_op')
    builder.add_meta_graph_and_variables(
        sess=K.get_session(),                                                                                                                    
        tags=[tf.saved_model.tag_constants.SERVING],                                                                                             
        signature_def_map={                                                                                                                      
            'ner': signature,                                                                                                                     
        },
        legacy_init_op=legacy_init_op)
    builder.save()

http请求如下：

test_data = dict()
test_data['inputs'] = [0] * 100


data = json.dumps({"signature_name": "ner", "instances": [test_data]})
print(data)

headers = {"content-type": "application/json"}
json_response = requests.post('http://localhost:9001/v1/models/ner-serving:predict', data=data, headers=headers)

print(json_response.text)

predictions = json.loads(json_response.text)

print(predictions)

tensorflow-serving returns NodeDef mentions attr \'batch_dims\' not in Op

Answer 1

确保您用于训练和模型保存的 tensorflow 版本与您用于服务的 tensorflow/serving 容器相同。

当我使用 tensorflow 1.14 进行训练但使用 tensorflow/serving v1.13 的旧容器映像进行服务时，我收到了相同的错误消息。一旦我在两个阶段都使用了 v1.14，它就起作用了。

NodeDef 提到 attr \'batch_dims\' not in Op in tensorflow serving

NodeDef mentions attr \'batch_dims\' not in Op in tensorflow serving

python

keras

tensorflow-serving