TF Server 提供的导出的 Keras 分类模型给出： Expects arg[0] to be float but string is provided

Question

我已经在 Keras 中训练了一个分类模型（根据本文，Keras 和 TF 的最新版本）在输入和输出方面与 CIFAR10 相似。为了提供这个模型，我使用以下代码将它导出到分类模型（查看类型）：

def keras_model_to_tf_serve(saved_keras_model,
                        local_version_dir,
                        type='classification',
                        save_model_version=1):

sess = tf.Session()
K.set_session(sess)
K.set_learning_phase(0)

old_model = load_model(saved_keras_model)
config = old_model.get_config()
weights = old_model.get_weights()

new_model = Sequential.from_config(config)
new_model.set_weights(weights)

classification_inputs = utils.build_tensor_info(new_model.input)
classification_outputs_classes = utils.build_tensor_info(new_model.output)

# The classification signature
classification_signature = tf.saved_model.signature_def_utils.build_signature_def(
    inputs={signature_constants.CLASSIFY_INPUTS: classification_inputs},
    outputs={
        signature_constants.CLASSIFY_OUTPUT_CLASSES:
            classification_outputs_classes
    },
    method_name=signature_constants.CLASSIFY_METHOD_NAME)
#print(classification_signature)
# The prediction signature
tensor_info_x = utils.build_tensor_info(new_model.input)
tensor_info_y = utils.build_tensor_info(new_model.output)
prediction_signature = tf.saved_model.signature_def_utils.build_signature_def(
    inputs={'inputs': tensor_info_x},
    outputs={'outputs': tensor_info_y},
    method_name=signature_constants.PREDICT_METHOD_NAME)
legacy_init_op = tf.group(tf.tables_initializer(), name='legacy_init_op')

print(prediction_signature)
save_model_dir = os.path.join(local_version_dir,str(save_model_version))
if os.path.exists(save_model_dir) and os.path.isdir(save_model_dir):
    shutil.rmtree(save_model_dir)

builder = saved_model_builder.SavedModelBuilder(save_model_dir)
with K.get_session() as sess:
    if type == 'classification':
        builder.add_meta_graph_and_variables(
            sess, [tag_constants.SERVING],
            signature_def_map={
                signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY:
                    classification_signature,
            },
            clear_devices=True, legacy_init_op=legacy_init_op)
    elif type == 'prediction':
        builder.add_meta_graph_and_variables(
            sess, [tag_constants.SERVING],
            signature_def_map={
                # Uncomment the first two lines below and comment out the subsequent four to reset.
                # signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY:
                #    classification_signature,
                'predict_results':
                    prediction_signature,
            },
            clear_devices=True, legacy_init_op=legacy_init_op)
    else:
        builder.add_meta_graph_and_variables(
            sess, [tag_constants.SERVING],
            signature_def_map={
                # Uncomment the first two lines below and comment out the subsequent four to reset.
                # signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY:
                #    classification_signature,
                'predict_results':
                    prediction_signature,
                signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY:
                    classification_signature
            },
            clear_devices=True, legacy_init_op=legacy_init_op)
    builder.save()

这导出很好，使用 saved_model_cli 我得到以下输出：

saved_model_cli show --dir /develop/1/ --tag_set serve -- 
signature_def serving_default
The given SavedModel SignatureDef contains the following input(s):
  inputs['inputs'] tensor_info:
     dtype: DT_FLOAT
     shape: (-1, 32, 32, 3)
     name: conv2d_1_input_1:0
The given SavedModel SignatureDef contains the following output(s):
  outputs['classes'] tensor_info:
     dtype: DT_FLOAT
     shape: (-1, 10)
     name: activation_6_1/Softmax:0
Method name is: tensorflow/serving/classify

因此模型期望获得 DT_Float 的形状 (-1,32,32,3)。由于这是一个分类模型（出于某种原因，它在使用方式上与预测模型/非常/不同），我采用了@sdcbr 代码 () 并进行了一些微小的修改：

import tensorflow as tf
import numpy as np
from tensorflow_serving.apis import classification_pb2, input_pb2
from grpc.beta import implementations
from tensorflow_serving.apis import prediction_service_pb2

image = np.random.rand(32,32,3)

def _float_feature(value):
    return tf.train.Feature(float_list=tf.train.FloatList(value=value))

request = classification_pb2.ClassificationRequest()
request.model_spec.name = 'model'
request.model_spec.signature_name = signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY

image = image.flatten().tolist()
image = [float(x) for x in image]
example = tf.train.Example(features=tf.train.Features(feature={'image': _float_feature(image)}))

inp = input_pb2.Input()
inp.example_list.examples.extend([example])

request.input.CopyFrom(inp)

channel = implementations.insecure_channel('localhost', 5005)
stub = prediction_service_pb2.beta_create_PredictionService_stub(channel)
response = stub.Classify(request, 10.0)

其中 TF-Serve 运行在我机器上的本地端口上，并在启动时给出 spec_name。据我所知，这应该可以工作，但是当我运行它时，我收到以下错误（为简洁起见在此处缩短）：

grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with:
   status = StatusCode.INVALID_ARGUMENT
   details = "Expects arg[0] to be float but string is provided"
   debug_error_string = " 
      {"created":"@1533046733.211573219","description":"Error received from peer","file":"src/core/lib/surface/call.cc","file_line":1083,"grpc_message":"Expects arg[0] to be float but string is provided","grpc_status":3}"

有什么想法吗？经过数小时的搜索，这是唯一的方法，我可以将任何类型的图像数据放入分类请求中。

Answer 1

尝试使用 prediction_service_pb2_grpc.PredictionServiceStub(channel) 而不是 prediction_service_pb2.beta_create_PredictionService_stub(channel)。显然这是最近从测试版中移出的。可以参考this例子。

TF Server 提供的导出的 Keras 分类模型给出： Expects arg[0] to be float but string is provided

Exported Keras classification model served by TF Server gives: Expects arg[0] to be float but string is provided

python

keras

tensorflow-serving