对 TensorFlow 服务的 RaggedTensor 请求失败
RaggedTensor request to TensorFlow serving fails
我创建了一个使用 RaggedTensors 的 TensorFlow 模型。模型工作正常，在调用 model.predict 时我得到了预期的结果。
input = tf.ragged.constant([[[-0.9984272718429565, -0.9422321319580078, -0.27657580375671387, -3.185823678970337, -0.6360141634941101, -1.6579184532165527, -1.9000954627990723, -0.49169546365737915, -0.6758883595466614, -0.6677696704864502, -0.532067060470581],
[-0.9984272718429565, -0.9421600103378296, 2.2048349380493164, -1.273996114730835, -0.6360141634941101, -1.5917999744415283, 0.6147914528846741, -0.49169546365737915, -0.6673409938812256, -0.6583622694015503, -0.5273991227149963],
[-0.9984272718429565, -0.942145586013794, 2.48842453956604, -1.6836735010147095, -0.6360141634941101, -1.5785763263702393, -1.900200605392456, -0.49169546365737915, -0.6656315326690674, -0.6583622694015503, -0.5273991227149963],
]])
model.predict(input)
>> array([[0.5138151 , 0.3277698 , 0.26122513]], dtype=float32)
我已将模型部署到 TensorFlow 服务服务器并使用以下代码调用：
import json
import requests
headers = {"content-type": "application/json"}
data = json.dumps({"instances":[
[-1.3523329846758267, ... more data ],
[-1.3523329846758267, ... more data ],
[-1.3523329846758267, ... more data ],
[-1.3523329846758267, ... more data ,
[-1.3523329846758267, ... more data ],
[-1.3523329846758267, ... more data ],
[-1.3523329846758267, ... more data ],
[-1.3523329846758267, ... more data })
json_response = requests.post('http://localhost:8501/v1/models/fashion_model:predict', data=data, headers=headers)
predictions = json.loads(json_response.text)
但随后出现以下错误：
"instances is a plain list, but expecting list of objects as multiple input tensors required as per tensorinfo_map"
我的模型描述：
MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:
signature_def['__saved_model_init_op']:
The given SavedModel SignatureDef contains the following input(s):
The given SavedModel SignatureDef contains the following output(s):
outputs['__saved_model_init_op'] tensor_info:
dtype: DT_INVALID
shape: unknown_rank
name: NoOp
Method name is:
signature_def['serving_default']:
The given SavedModel SignatureDef contains the following input(s):
inputs['args_0'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 11)
name: serving_default_args_0:0
inputs['args_0_1'] tensor_info:
dtype: DT_INT64
shape: (-1)
name: serving_default_args_0_1:0
The given SavedModel SignatureDef contains the following output(s):
outputs['dense_2'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 3)
name: StatefulPartitionedCall:0
Method name is: tensorflow/serving/predict
WARNING: Logging before flag parsing goes to stderr.
W0124 09:33:16.365564 140189730998144 deprecation.py:506] From /usr/local/lib/python2.7/dist-packages/tensorflow_core/python/ops/resource_variable_ops.py:1786: calling __init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
Defined Functions:
Function Name: '__call__'
Option #1
Callable with:
Argument #1
DType: RaggedTensorSpec
Value: RaggedTensorSpec(TensorShape([None, None, 11]), tf.float32, 1, tf.int64)
Argument #2
DType: bool
Value: True
Argument #3
DType: NoneType
Value: None
Option #2
Callable with:
Argument #1
DType: RaggedTensorSpec
Value: RaggedTensorSpec(TensorShape([None, None, 11]), tf.float32, 1, tf.int64)
Argument #2
DType: bool
Value: False
Argument #3
DType: NoneType
Value: None
Function Name: '_default_save_signature'
Option #1
Callable with:
Argument #1
DType: RaggedTensorSpec
Value: RaggedTensorSpec(TensorShape([None, None, 11]), tf.float32, 1, tf.int64)
Function Name: 'call_and_return_all_conditional_losses'
Option #1
Callable with:
Argument #1
DType: RaggedTensorSpec
Value: RaggedTensorSpec(TensorShape([None, None, 11]), tf.float32, 1, tf.int64)
Argument #2
DType: bool
Value: True
Argument #3
DType: NoneType
Value: None
Option #2
Callable with:
Argument #1
DType: RaggedTensorSpec
Value: RaggedTensorSpec(TensorShape([None, None, 11]), tf.float32, 1, tf.int64)
Argument #2
DType: bool
Value: False
Argument #3
DType: NoneType
Value: None
我错过了什么？
更新：
检查 saved_model_cli 输出后，我怀疑我应该将请求作为如下对象发送，但我不确定输入...
{
"instances": [
{
"args_0": nested-list ?,
"args_0_1": ???
}
]
}
更新2
Colab 用于测试此场景，link 用于下载 Colab 中的模型。
更新 3：
根据@Niteya Shah 的建议，我将 API 称为：
data = json.dumps({
"inputs": {
"args_0": [[-0.9984272718429565, -0.9422321319580078, -0.27657580375671387, -3.185823678970337, -0.6360141634941101, -1.6579184532165527, -1.9000954627990723, -0.49169546365737915, -0.6758883595466614, -0.6677696704864502, -0.532067060470581],
[-0.9984272718429565, -0.9421600103378296, 2.2048349380493164, -1.273996114730835, -0.6360141634941101, -1.5917999744415283, 0.6147914528846741, -0.49169546365737915, -0.6673409938812256, -0.6583622694015503, -0.5273991227149963]],
"args_0_1": [1, 2] #Please Check what inputs come here ?
}
})
并得到结果（终于！）：
{'outputs': [[0.466771603, 0.455221593, 0.581544757]]}
然后使用相同的数据调用模型，如下所示：
import numpy as np
input = tf.ragged.constant([[
[-0.9984272718429565, -0.9422321319580078, -0.27657580375671387, -3.185823678970337, -0.6360141634941101, -1.6579184532165527, -1.9000954627990723, -0.49169546365737915, -0.6758883595466614, -0.6677696704864502, -0.532067060470581],
[-0.9984272718429565, -0.9421600103378296, 2.2048349380493164, -1.273996114730835, -0.6360141634941101, -1.5917999744415283, 0.6147914528846741, -0.49169546365737915, -0.6673409938812256, -0.6583622694015503, -0.5273991227149963]
]])
model.predict(input)
得到了不同的结果：
array([[0.4817084 , 0.3649785 , 0.01603118]], dtype=float32)
我想我还没到。
https://www.tensorflow.org/tfx/serving/api_rest#predict_api
我认为您需要使用 REST API 中推荐的柱状格式而不是行格式，因为您的第 0 个输入的维度不匹配。
这意味着您将不得不使用输入而不是实例。
由于您还有多个输入，因此您还必须将其作为命名输入提及。
示例数据请求可能如下所示
data = json.dumps({
"inputs": {
"args_0": [[-0.9984272718429565, -0.9422321319580078, -0.27657580375671387, -3.185823678970337, -0.6360141634941101, -1.6579184532165527, -1.9000954627990723, -0.49169546365737915, -0.6758883595466614, -0.6677696704864502, -0.532067060470581],
[-0.9984272718429565, -0.9421600103378296, 2.2048349380493164, -1.273996114730835, -0.6360141634941101, -1.5917999744415283, 0.6147914528846741, -0.49169546365737915, -0.6673409938812256, -0.6583622694015503, -0.5273991227149963]],
"args_0_1": [10, 11] #Substitute this with the correct row partition values.
}
})
编辑：
我从 here 中读到关于不规则张量的信息，似乎第二个输入可能是行分区。我在文档中找不到关于通常使用什么行分区样式的信息，所以我使用了行长度方法。
幸运的是，TensorFlow ragged 提供 methods 为我们做这件事。使用 values 和 row_splits 属性访问它们。那应该可以。
我创建了一个使用 RaggedTensors 的 TensorFlow 模型。模型工作正常，在调用 model.predict 时我得到了预期的结果。
input = tf.ragged.constant([[[-0.9984272718429565, -0.9422321319580078, -0.27657580375671387, -3.185823678970337, -0.6360141634941101, -1.6579184532165527, -1.9000954627990723, -0.49169546365737915, -0.6758883595466614, -0.6677696704864502, -0.532067060470581],
[-0.9984272718429565, -0.9421600103378296, 2.2048349380493164, -1.273996114730835, -0.6360141634941101, -1.5917999744415283, 0.6147914528846741, -0.49169546365737915, -0.6673409938812256, -0.6583622694015503, -0.5273991227149963],
[-0.9984272718429565, -0.942145586013794, 2.48842453956604, -1.6836735010147095, -0.6360141634941101, -1.5785763263702393, -1.900200605392456, -0.49169546365737915, -0.6656315326690674, -0.6583622694015503, -0.5273991227149963],
]])
model.predict(input)
>> array([[0.5138151 , 0.3277698 , 0.26122513]], dtype=float32)
我已将模型部署到 TensorFlow 服务服务器并使用以下代码调用：
import json
import requests
headers = {"content-type": "application/json"}
data = json.dumps({"instances":[
[-1.3523329846758267, ... more data ],
[-1.3523329846758267, ... more data ],
[-1.3523329846758267, ... more data ],
[-1.3523329846758267, ... more data ,
[-1.3523329846758267, ... more data ],
[-1.3523329846758267, ... more data ],
[-1.3523329846758267, ... more data ],
[-1.3523329846758267, ... more data })
json_response = requests.post('http://localhost:8501/v1/models/fashion_model:predict', data=data, headers=headers)
predictions = json.loads(json_response.text)
但随后出现以下错误：
"instances is a plain list, but expecting list of objects as multiple input tensors required as per tensorinfo_map"
我的模型描述：
MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:
signature_def['__saved_model_init_op']:
The given SavedModel SignatureDef contains the following input(s):
The given SavedModel SignatureDef contains the following output(s):
outputs['__saved_model_init_op'] tensor_info:
dtype: DT_INVALID
shape: unknown_rank
name: NoOp
Method name is:
signature_def['serving_default']:
The given SavedModel SignatureDef contains the following input(s):
inputs['args_0'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 11)
name: serving_default_args_0:0
inputs['args_0_1'] tensor_info:
dtype: DT_INT64
shape: (-1)
name: serving_default_args_0_1:0
The given SavedModel SignatureDef contains the following output(s):
outputs['dense_2'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 3)
name: StatefulPartitionedCall:0
Method name is: tensorflow/serving/predict
WARNING: Logging before flag parsing goes to stderr.
W0124 09:33:16.365564 140189730998144 deprecation.py:506] From /usr/local/lib/python2.7/dist-packages/tensorflow_core/python/ops/resource_variable_ops.py:1786: calling __init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
Defined Functions:
Function Name: '__call__'
Option #1
Callable with:
Argument #1
DType: RaggedTensorSpec
Value: RaggedTensorSpec(TensorShape([None, None, 11]), tf.float32, 1, tf.int64)
Argument #2
DType: bool
Value: True
Argument #3
DType: NoneType
Value: None
Option #2
Callable with:
Argument #1
DType: RaggedTensorSpec
Value: RaggedTensorSpec(TensorShape([None, None, 11]), tf.float32, 1, tf.int64)
Argument #2
DType: bool
Value: False
Argument #3
DType: NoneType
Value: None
Function Name: '_default_save_signature'
Option #1
Callable with:
Argument #1
DType: RaggedTensorSpec
Value: RaggedTensorSpec(TensorShape([None, None, 11]), tf.float32, 1, tf.int64)
Function Name: 'call_and_return_all_conditional_losses'
Option #1
Callable with:
Argument #1
DType: RaggedTensorSpec
Value: RaggedTensorSpec(TensorShape([None, None, 11]), tf.float32, 1, tf.int64)
Argument #2
DType: bool
Value: True
Argument #3
DType: NoneType
Value: None
Option #2
Callable with:
Argument #1
DType: RaggedTensorSpec
Value: RaggedTensorSpec(TensorShape([None, None, 11]), tf.float32, 1, tf.int64)
Argument #2
DType: bool
Value: False
Argument #3
DType: NoneType
Value: None
我错过了什么？
更新：
检查 saved_model_cli 输出后，我怀疑我应该将请求作为如下对象发送，但我不确定输入...
{
"instances": [
{
"args_0": nested-list ?,
"args_0_1": ???
}
]
}
更新2 Colab 用于测试此场景，link 用于下载 Colab 中的模型。
更新 3：
根据@Niteya Shah 的建议，我将 API 称为：
data = json.dumps({
"inputs": {
"args_0": [[-0.9984272718429565, -0.9422321319580078, -0.27657580375671387, -3.185823678970337, -0.6360141634941101, -1.6579184532165527, -1.9000954627990723, -0.49169546365737915, -0.6758883595466614, -0.6677696704864502, -0.532067060470581],
[-0.9984272718429565, -0.9421600103378296, 2.2048349380493164, -1.273996114730835, -0.6360141634941101, -1.5917999744415283, 0.6147914528846741, -0.49169546365737915, -0.6673409938812256, -0.6583622694015503, -0.5273991227149963]],
"args_0_1": [1, 2] #Please Check what inputs come here ?
}
})
并得到结果（终于！）：
{'outputs': [[0.466771603, 0.455221593, 0.581544757]]}
然后使用相同的数据调用模型，如下所示：
import numpy as np
input = tf.ragged.constant([[
[-0.9984272718429565, -0.9422321319580078, -0.27657580375671387, -3.185823678970337, -0.6360141634941101, -1.6579184532165527, -1.9000954627990723, -0.49169546365737915, -0.6758883595466614, -0.6677696704864502, -0.532067060470581],
[-0.9984272718429565, -0.9421600103378296, 2.2048349380493164, -1.273996114730835, -0.6360141634941101, -1.5917999744415283, 0.6147914528846741, -0.49169546365737915, -0.6673409938812256, -0.6583622694015503, -0.5273991227149963]
]])
model.predict(input)
得到了不同的结果：
array([[0.4817084 , 0.3649785 , 0.01603118]], dtype=float32)
我想我还没到。
https://www.tensorflow.org/tfx/serving/api_rest#predict_api
我认为您需要使用 REST API 中推荐的柱状格式而不是行格式，因为您的第 0 个输入的维度不匹配。 这意味着您将不得不使用输入而不是实例。 由于您还有多个输入，因此您还必须将其作为命名输入提及。
示例数据请求可能如下所示
data = json.dumps({
"inputs": {
"args_0": [[-0.9984272718429565, -0.9422321319580078, -0.27657580375671387, -3.185823678970337, -0.6360141634941101, -1.6579184532165527, -1.9000954627990723, -0.49169546365737915, -0.6758883595466614, -0.6677696704864502, -0.532067060470581],
[-0.9984272718429565, -0.9421600103378296, 2.2048349380493164, -1.273996114730835, -0.6360141634941101, -1.5917999744415283, 0.6147914528846741, -0.49169546365737915, -0.6673409938812256, -0.6583622694015503, -0.5273991227149963]],
"args_0_1": [10, 11] #Substitute this with the correct row partition values.
}
})
编辑：
我从 here 中读到关于不规则张量的信息，似乎第二个输入可能是行分区。我在文档中找不到关于通常使用什么行分区样式的信息，所以我使用了行长度方法。
幸运的是，TensorFlow ragged 提供 methods 为我们做这件事。使用 values 和 row_splits 属性访问它们。那应该可以。