如何为张量流服务准备预热请求文件?
How to prepare warmup request file for tensorflow serving?
当前版本的 tensorflow-serving 尝试从 assets.extra/tf_serving_warmup_requests 文件加载预热请求。
2018-08-16 16:05:28.513085: I tensorflow_serving/servables/tensorflow/saved_model_warmup.cc:83] No warmup data file found at /tmp/faster_rcnn_inception_v2_coco_2018_01_28_string_input_version-export/1/assets.extra/tf_serving_warmup_requests
不知道tensorflow有没有提供通用的api导出请求到位置?或者我们应该手动向该位置写入请求?
此时没有通用的 API 可以将预热数据导出到 assets.extra。写个脚本比较简单(类似下面):
import tensorflow as tf
from tensorflow_serving.apis import model_pb2
from tensorflow_serving.apis import predict_pb2
from tensorflow_serving.apis import prediction_log_pb2
def main():
with tf.python_io.TFRecordWriter("tf_serving_warmup_requests") as writer:
request = predict_pb2.PredictRequest(
model_spec=model_pb2.ModelSpec(name="<add here>"),
inputs={"examples": tf.make_tensor_proto([<add here>])}
)
log = prediction_log_pb2.PredictionLog(
predict_log=prediction_log_pb2.PredictLog(request=request))
writer.write(log.SerializeToString())
if __name__ == "__main__":
main()
我们参考了official doc
特别是,我们使用分类而不是预测,因此我们将该代码更改为
log = prediction_log_pb2.PredictionLog(
classify_log=prediction_log_pb2.ClassifyLog(request=<request>))
这是使用 ResNet model 的对象检测系统的完整示例。预测由图像组成。
import tensorflow as tf
import requests
import base64
from tensorflow.python.framework import tensor_util
from tensorflow_serving.apis import predict_pb2
from tensorflow_serving.apis import prediction_log_pb2
IMAGE_URL = 'https://tensorflow.org/images/blogs/serving/cat.jpg'
NUM_RECORDS = 100
def get_image_bytes():
image_content = requests.get(IMAGE_URL, stream=True)
image_content.raise_for_status()
return image_content.content
def main():
"""Generate TFRecords for warming up."""
with tf.io.TFRecordWriter("tf_serving_warmup_requests") as writer:
image_bytes = get_image_bytes()
predict_request = predict_pb2.PredictRequest()
predict_request.model_spec.name = 'resnet'
predict_request.model_spec.signature_name = 'serving_default'
predict_request.inputs['image_bytes'].CopyFrom(
tensor_util.make_tensor_proto([image_bytes], tf.string))
log = prediction_log_pb2.PredictionLog(
predict_log=prediction_log_pb2.PredictLog(request=predict_request))
for r in range(NUM_RECORDS):
writer.write(log.SerializeToString())
if __name__ == "__main__":
main()
此脚本将创建一个名为“tf_serving_warmup_requests”的文件
我将此文件移动到 /your_model_location/resnet/1538687457/assets.extra/
,然后重新启动我的 docker 图像以获取新的更改。
当前版本的 tensorflow-serving 尝试从 assets.extra/tf_serving_warmup_requests 文件加载预热请求。
2018-08-16 16:05:28.513085: I tensorflow_serving/servables/tensorflow/saved_model_warmup.cc:83] No warmup data file found at /tmp/faster_rcnn_inception_v2_coco_2018_01_28_string_input_version-export/1/assets.extra/tf_serving_warmup_requests
不知道tensorflow有没有提供通用的api导出请求到位置?或者我们应该手动向该位置写入请求?
此时没有通用的 API 可以将预热数据导出到 assets.extra。写个脚本比较简单(类似下面):
import tensorflow as tf
from tensorflow_serving.apis import model_pb2
from tensorflow_serving.apis import predict_pb2
from tensorflow_serving.apis import prediction_log_pb2
def main():
with tf.python_io.TFRecordWriter("tf_serving_warmup_requests") as writer:
request = predict_pb2.PredictRequest(
model_spec=model_pb2.ModelSpec(name="<add here>"),
inputs={"examples": tf.make_tensor_proto([<add here>])}
)
log = prediction_log_pb2.PredictionLog(
predict_log=prediction_log_pb2.PredictLog(request=request))
writer.write(log.SerializeToString())
if __name__ == "__main__":
main()
我们参考了official doc
特别是,我们使用分类而不是预测,因此我们将该代码更改为
log = prediction_log_pb2.PredictionLog(
classify_log=prediction_log_pb2.ClassifyLog(request=<request>))
这是使用 ResNet model 的对象检测系统的完整示例。预测由图像组成。
import tensorflow as tf
import requests
import base64
from tensorflow.python.framework import tensor_util
from tensorflow_serving.apis import predict_pb2
from tensorflow_serving.apis import prediction_log_pb2
IMAGE_URL = 'https://tensorflow.org/images/blogs/serving/cat.jpg'
NUM_RECORDS = 100
def get_image_bytes():
image_content = requests.get(IMAGE_URL, stream=True)
image_content.raise_for_status()
return image_content.content
def main():
"""Generate TFRecords for warming up."""
with tf.io.TFRecordWriter("tf_serving_warmup_requests") as writer:
image_bytes = get_image_bytes()
predict_request = predict_pb2.PredictRequest()
predict_request.model_spec.name = 'resnet'
predict_request.model_spec.signature_name = 'serving_default'
predict_request.inputs['image_bytes'].CopyFrom(
tensor_util.make_tensor_proto([image_bytes], tf.string))
log = prediction_log_pb2.PredictionLog(
predict_log=prediction_log_pb2.PredictLog(request=predict_request))
for r in range(NUM_RECORDS):
writer.write(log.SerializeToString())
if __name__ == "__main__":
main()
此脚本将创建一个名为“tf_serving_warmup_requests”的文件
我将此文件移动到 /your_model_location/resnet/1538687457/assets.extra/
,然后重新启动我的 docker 图像以获取新的更改。