Tensorflow 服务预测 REST API 'not formatted correctly for base64 data' 错误

Question

我已经保存了一个 Tensorflow 模型并使用 Tensorflow Serving（tensorflow/serving:1.12.0 和 tensorflow/serving:1.12.0-gpu）为它提供服务。

我想使用 Predict REST API，但调用失败并出现 'not formatted correctly for base64 data' 错误。

要求：

POST /v1/models/payfraud:预测

{
  "inputs": [
    {
      "payFraudInput": [[44.26, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0]]
    }
  ]
}

回复：

400

{
    "error": "JSON Value: {\n    \"payFraudInput\": [\n        [\n            44.26,\n            0,\n            0,\n            0,\n            0,\n            1,\n            0,\n            0,\n            0,\n            0,\n            0,\n            0,\n            0,\n            0\n        ]\n    ]\n} not formatted correctly for base64 data"
}

模型输入需要 DT_FLOAT，所以我认为我不需要 base64 编码。

POST /v1/models/payfraud/版本/1/元数据

{
    "model_spec": {
        "name": "payfraud",
        "signature_name": "",
        "version": "1"
    },
    "metadata": {
        "signature_def": {
            "signature_def": {
                "predict_fraud": {
                    "inputs": {
                        "payFraudInput": {
                            "dtype": "DT_FLOAT",
                            "tensor_shape": {
                                "dim": [
                                    {
                                        "size": "-1",
                                        "name": ""
                                    },
                                    {
                                        "size": "15",
                                        "name": ""
                                    }
                                ],
                                "unknown_rank": false
                            },
                            "name": "payFraudInput:0"
                        }
                    },
                    "outputs": {
                        "payFraudOutput": {
                            "dtype": "DT_FLOAT",
                            "tensor_shape": {
                                "dim": [
                                    {
                                        "size": "-1",
                                        "name": ""
                                    },
                                    {
                                        "size": "2",
                                        "name": ""
                                    }
                                ],
                                "unknown_rank": false
                            },
                            "name": "payFraudOutput:0"
                        }
                    },
                    "method_name": "tensorflow/serving/predict"
                },
                "serving_default": {
                    "inputs": {
                        "inputs": {
                            "dtype": "DT_STRING",
                            "tensor_shape": {
                                "dim": [],
                                "unknown_rank": true
                            },
                            "name": "tf_example:0"
                        }
                    },
                    "outputs": {
                        "classes": {
                            "dtype": "DT_STRING",
                            "tensor_shape": {
                                "dim": [
                                    {
                                        "size": "-1",
                                        "name": ""
                                    },
                                    {
                                        "size": "2",
                                        "name": ""
                                    }
                                ],
                                "unknown_rank": false
                            },
                            "name": "index_to_string_Lookup:0"
                        },
                        "scores": {
                            "dtype": "DT_FLOAT",
                            "tensor_shape": {
                                "dim": [
                                    {
                                        "size": "-1",
                                        "name": ""
                                    },
                                    {
                                        "size": "2",
                                        "name": ""
                                    }
                                ],
                                "unknown_rank": false
                            },
                            "name": "TopKV2:0"
                        }
                    },
                    "method_name": "tensorflow/serving/classify"
                }
            }
        }
    }
}

模型是这样保存的：

    prediction_signature = (
      tf.saved_model.signature_def_utils.build_signature_def(
          inputs={"payFraudInput": tensor_info_x},
          outputs={"payFraudOutput": tensor_info_y},
          method_name=tf.saved_model.signature_constants.PREDICT_METHOD_NAME))

    classification_signature = (
       tf.saved_model.signature_def_utils.build_signature_def(
          inputs={
              tf.saved_model.signature_constants.CLASSIFY_INPUTS:
                  classification_inputs
          },
          outputs={
              tf.saved_model.signature_constants.CLASSIFY_OUTPUT_CLASSES:
                  classification_outputs_classes,
              tf.saved_model.signature_constants.CLASSIFY_OUTPUT_SCORES:
                  classification_outputs_scores
          },
          method_name=tf.saved_model.signature_constants.CLASSIFY_METHOD_NAME))

    export_path = os.path.join(tf.compat.as_bytes(export_dir), tf.compat.as_bytes("1"))
    print('Exporting trained model to ', export_path)
    builder = tf.saved_model.builder.SavedModelBuilder(export_path)
    builder.add_meta_graph_and_variables( sess, [tf.saved_model.tag_constants.SERVING],
        signature_def_map={
            'predict_fraud':
                 prediction_signature,
             tf.saved_model.signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY:
                 classification_signature,
        },
        main_op=tf.tables_initializer(),
        strip_default_attrs=True)

    builder.save()
    print('Done exporting!')

尝试 b64 也不行：

要求

{
  "inputs": [
    {
      "payFraudInput":{"b64":"NDQuMjYsIDAsIDAsIDAsIDAsIDEsIDAsIDAsIDAsIDAsIDAsIDAsIDAsIDA="}
    }
  ]
}

回应

{
    "error": "JSON Value: {\n    \"payFraudInput\": {\n        \"b64\": \"NDQuMjYsIDAsIDAsIDAsIDAsIDEsIDAsIDAsIDAsIDAsIDAsIDAsIDAsIDA=\"\n    }\n} not formatted correctly for base64 data"
}

我做错了什么？

Answer 1

清理并简化训练脚本的模型保存部分后，我得到了预测响应。

现在的存档是这样的：

    export_path = os.path.join(tf.compat.as_bytes(export_dir), tf.compat.as_bytes("1"))
    builder = tf.saved_model.builder.SavedModelBuilder(export_path)

    predict_signature_def = (
        tf.saved_model.signature_def_utils.predict_signature_def({"x": X}, {"y": Y_hat}))
    signature_def_map = {
        tf.saved_model.signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY:
            predict_signature_def
    }
    sess.run(tf.global_variables_initializer())
    builder.add_meta_graph_and_variables(
        sess, [tf.saved_model.tag_constants.SERVING],
        signature_def_map=signature_def_map)
    builder.save()

并且我能够从 Tensorflow Serving 获得有效响应：

预测请求：

POST /v1/models/payfraud:预测

{
  "inputs": [[44.26, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0]]
}

预测响应：

{
    "outputs": [
        [
            0.5,
            0.5
        ]
    ]
}

获取/v1/models/payfraud/版本/1/元数据

{
    "model_spec": {
        "name": "payfraud",
        "signature_name": "",
        "version": "1"
    },
    "metadata": {
        "signature_def": {
            "signature_def": {
                "serving_default": {
                    "inputs": {
                        "x": {
                            "dtype": "DT_FLOAT",
                            "tensor_shape": {
                                "dim": [
                                    {
                                        "size": "-1",
                                        "name": ""
                                    },
                                    {
                                        "size": "15",
                                        "name": ""
                                    }
                                ],
                                "unknown_rank": false
                            },
                            "name": "x:0"
                        }
                    },
                    "outputs": {
                        "y": {
                            "dtype": "DT_FLOAT",
                            "tensor_shape": {
                                "dim": [
                                    {
                                        "size": "-1",
                                        "name": ""
                                    },
                                    {
                                        "size": "2",
                                        "name": ""
                                    }
                                ],
                                "unknown_rank": false
                            },
                            "name": "y:0"
                        }
                    },
                    "method_name": "tensorflow/serving/predict"
                }
            }
        }
    }
}

Tensorflow 服务预测 REST API 'not formatted correctly for base64 data' 错误

Tensorflow Serving Predict REST API 'not formatted correctly for base64 data' Error

python

rest

tensorflow

tensorflow-serving