如何在 tensorflow-2.0.0beta0 中提供我的特定模型
How do I serve my specific model in tensorflow-2.0.0beta0
我想基于 3 列将文档分类为 4 个类别（位置），burks 是一个 4 位代码，lifnr 是一个字典值，waers 也是一个带有 LinarClassifier 的字典值。
然后保存模型，为其提供服务，并向其抛出 burks、lifnr 和 waers 值以获得预测。
我的训练数据是这样的：
bukrs;lifnr;waers;location
5280;1004008999;EUR;0
5280;1004009000;EUR;2
5280;1004003061;EUR;1
...
而且我可以成功训练模型并保存它，结果是
saved_model.pb 和变量文件夹。
到目前为止一切顺利。
我已经检查过模型本身是否像这样工作：
saved_model_cli show --dir 1561324458 --all
这给了我：
MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:
signature_def['classification']:
The given SavedModel SignatureDef contains the following input(s):
inputs['inputs'] tensor_info:
dtype: DT_STRING
shape: (-1)
name: input_example_tensor:0
The given SavedModel SignatureDef contains the following output(s):
outputs['classes'] tensor_info:
dtype: DT_STRING
shape: (-1, 4)
name: head/Tile:0
outputs['scores'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 4)
name: head/predictions/probabilities:0
Method name is: tensorflow/serving/classify
signature_def['predict']:
The given SavedModel SignatureDef contains the following input(s):
inputs['examples'] tensor_info:
dtype: DT_STRING
shape: (-1)
name: input_example_tensor:0
The given SavedModel SignatureDef contains the following output(s):
outputs['all_class_ids'] tensor_info:
dtype: DT_INT32
shape: (-1, 4)
name: head/predictions/Tile:0
outputs['all_classes'] tensor_info:
dtype: DT_STRING
shape: (-1, 4)
name: head/predictions/Tile_1:0
outputs['class_ids'] tensor_info:
dtype: DT_INT64
shape: (-1, 1)
name: head/predictions/ExpandDims:0
outputs['classes'] tensor_info:
dtype: DT_STRING
shape: (-1, 1)
name: head/predictions/str_classes:0
outputs['logits'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 4)
name: linear/linear_model/linear/linear_model/linear/linear_model/weighted_sum:0
outputs['probabilities'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 4)
name: head/predictions/probabilities:0
Method name is: tensorflow/serving/predict
signature_def['serving_default']:
The given SavedModel SignatureDef contains the following input(s):
inputs['inputs'] tensor_info:
dtype: DT_STRING
shape: (-1)
name: input_example_tensor:0
The given SavedModel SignatureDef contains the following output(s):
outputs['classes'] tensor_info:
dtype: DT_STRING
shape: (-1, 4)
name: head/Tile:0
outputs['scores'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 4)
name: head/predictions/probabilities:0
Method name is: tensorflow/serving/classify
这对我来说很不错。
这是我的整个训练python脚本：
from __future__ import absolute_import, division, print_function, unicode_literals
import tensorflow as tf
import matplotlib.pyplot as plt
import numpy as np
import os
import pandas as pd
import re
import seaborn as sns
from tensorflow import feature_column
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report
import itertools
from itertools import islice
#read data
dataframe = pd.read_csv('invoices_classed2.csv', sep=';',header=0)
dataframe.head()
#cut in sets
train, test = train_test_split(dataframe, test_size=0.3)
train, val = train_test_split(train, test_size=0.3)
#print metrics
print(len(train), 'train examples')
print(len(val), 'validation examples')
print(len(test), 'test examples')
# A utility method to create a tf.data dataset from a Pandas Dataframe
labels = pd.Series();
def df_to_dataset(dataframe, shuffle=False, batch_size=32):
dataframe = dataframe.copy()
labels = dataframe.pop('location')
ds = tf.data.Dataset.from_tensor_slices((dict(dataframe), labels))
if shuffle:
ds = ds.shuffle(buffer_size=len(dataframe))
ds = ds.batch(batch_size)
return ds
# A utility method to create a tf.data dataset from a Pandas Dataframe and use it as functional variable
def make_input_fn(dataframe=None, n_epochs=None, shuffle=False, batch_size=32):
def input_fn():
internal_dataframe = dataframe.copy()
labels = internal_dataframe.pop('location')
ds = tf.data.Dataset.from_tensor_slices((dict(internal_dataframe), labels))
if shuffle:
ds = ds.shuffle(buffer_size=len(internal_dataframe))
ds = ds.repeat(n_epochs)
ds = ds.batch(batch_size)
return ds
return input_fn
#building feature columns
bukrs = feature_column.numeric_column("bukrs")
lifnr = feature_column.categorical_column_with_vocabulary_list(
'lifnr',['1004000409','1004003061','1004008999','1004009001','1004009000','1004003768','1004009002'])
lifnr_one_hot = feature_column.indicator_column(lifnr)
waers = feature_column.categorical_column_with_vocabulary_list(
'waers', ['EUR', 'GBP', 'USD','JPY','CZK','HUF'])
waers_one_hot = feature_column.indicator_column(waers)
actual_feature_columns = []
actual_feature_columns.append(bukrs)
actual_feature_columns.append(lifnr_one_hot)
actual_feature_columns.append(waers_one_hot)
#making datasets
train_ds = make_input_fn(train)
val_ds = make_input_fn(val)
test_ds = make_input_fn(test)
print ('####################creating model####################')
linear_est = tf.estimator.LinearClassifier(feature_columns=actual_feature_columns,n_classes=4,model_dir="C:\Users\70D4867\Desktop\invoicemodel")
print ('####################Train model####################')
#Train model.
linear_est.train(train_ds,max_steps=10000)
print ('####################Evaluation####################')
# Evaluation.
result = linear_est.evaluate(val_ds, steps=1000)
print ('####################printing result####################')
print(result)
print ('####################Done evaluating####################')
for key in sorted(result):
print (key, result[key])
print ('####################predictions####################')
y_generator = linear_est.predict(test_ds)
print ('####################slice predictions####################')
predictions = list(itertools.islice(y_generator,len(test)))
print ('####################predictions output####################')
final_preds = []
template = ('\nPrediction is "{}" ({:.1f}%)')
i = 0;
for pred in (predictions):
final_preds.append(pred['class_ids'][0])
class_id = pred['class_ids'][0]
probability = pred['probabilities'][class_id]
i = i +1
expected = []
for index, row in test.iterrows():
expected.append(row['location'])
print ('####################Test Results####################')
print(classification_report(expected,final_preds))
print ('####################Saving Model####################')
feature_spec = tf.feature_column.make_parse_example_spec(actual_feature_columns)
print(feature_spec)
my_serving_input_receiver_fn = tf.estimator.export.build_parsing_serving_input_receiver_fn(feature_spec)
linear_est.export_saved_model(export_dir_base="invoicemodel\1",serving_input_receiver_fn=my_serving_input_receiver_fn)
但是当我想从这样的模型中得到预测时：
saved_model_cli run --dir invoicemodel61324458 --tag_set serve --signature_def predict --input_examples 'examples=[{"bukrs": 5280, "lifnr": "1004003930", "waers": "EUR"}]'
我希望是这样的：
[0]
我收到错误：
NameError: name 'bukrs' is not defined
我也试着给它提供一个 .npy 文件。
我用一些训练数据创建了这个文件：
bukrs;lifnr;waers
5280;1004008999;EUR
5280;1004009000;EUR
5280;1004003061;EUR
...
像这样：
csv_fn = "invoices_classed_npy.csv"
file = pd.read_csv(csv_fn)
np.save('invoices_classed_npy.npy', file, allow_pickle = True);
但是当我尝试时：
saved_model_cli run --dir .\invoicemodel61324458 --tag_set serve --signature_def classification --inputs 'inputs="invoices_classed_npy.npy"'
我预计
[1],[2],[0]
我得到了：
ValueError: Cannot feed value of shape (55276, 1) for Tensor
'input_example_tensor:0', which has shape '(None,)'
好的，所以我也尝试在容器中提供它：
docker run -t --rm -p 8501:8501 \
> -v "/data/container/tensorflow/model:/models/saved_model" \
> -e MODEL_NAME=saved_model \
> tensorflow/serving
我得到了：
/usr/bin/tf_serving_entrypoint.sh: line 3: 6 Illegal instruction (core dumped) tensorflow_model_server --port=8500 --rest_api_port=8501 --model_name=${MODEL_NAME} --model_base_path=${MODEL_BASE_PATH}/${MODEL_NAME} "$@"
我做错了什么？
我如何正确地从我的模型中得到预测？
我知道我的问题是什么了。
您必须在训练模型之前指定 serving_input_receiver_fn，并且以某种方式将其从特征列中拉出是行不通的。
这是我的脚本末尾不起作用的部分：
feature_spec = tf.feature_column.make_parse_example_spec(actual_feature_columns)
my_serving_input_receiver_fn = tf.estimator.export.build_parsing_serving_input_receiver_fn(feature_spec)
linear_est.export_saved_model(export_dir_base="invoicemodel\1",serving_input_receiver_fn=my_serving_input_receiver_fn)
这导致 signature_def 的预测看起来像：
signature_def['predict']:
The given SavedModel SignatureDef contains the following input(s):
inputs['examples'] tensor_info:
dtype: DT_STRING
shape: (-1)
name: input_example_tensor:0
以及错误：
NameError: name 'bukrs' is not defined
我把 serving_input_receiver_fn 的定义改成这样
feature_placeholder = {'bukrs': tf.placeholder(tf.int32, [1],
name='bukrs_placeholder'),'lifnr': tf.placeholder('string', [1],
name='lifnr_placeholder'),'waers': tf.placeholder('string', [1],
name='waers_placeholder')}
my_serving_input_receiver_fn =
tf.estimator.export.build_raw_serving_input_receiver_fn(feature_placeholder)
####now train the model####
linear_est.train(train_ds,max_steps=10000)
#evaluation ...
#test ....
#### then save at the end ###
linear_est.export_saved_model(export_dir_base="invoicemodel\1",serving_input_receiver_fn=my_serving_input_receiver_fn)
导致模型的 signature_def 正确：
signature_def['predict']:
The given SavedModel SignatureDef contains the following input(s):
inputs['bukrs'] tensor_info:
dtype: DT_INT32
shape: (-1)
name: bukrs_placeholder:0
inputs['lifnr'] tensor_info:
dtype: DT_STRING
shape: (-1)
name: lifnr_placeholder:0
inputs['waers'] tensor_info:
dtype: DT_STRING
shape: (-1)
name: waers_placeholder:0
现在我可以从模型中得到预测：
saved_model_cli run --dir 61727347 --tag_set serve --signature_def predict --input_exprs="bukrs=[5280];lifnr=['1004002578'];waers=['EUR']"
这导致预测：
Result for output key all_class_ids:
[[0 1 2 3]]
Result for output key all_classes:
[[b'0' b'1' b'2' b'3']]
Result for output key class_ids:
[[0]]
Result for output key classes:
[[b'0']]
Result for output key logits:
[[493.99664 475.6605 482.86667 483.922 ]]
Result for output key probabilities:
[[9.99943256e-01 1.08814335e-08 1.46652310e-05 4.21320110e-05]]
我希望这对某人有所帮助。
我想基于 3 列将文档分类为 4 个类别（位置），burks 是一个 4 位代码，lifnr 是一个字典值，waers 也是一个带有 LinarClassifier 的字典值。 然后保存模型，为其提供服务，并向其抛出 burks、lifnr 和 waers 值以获得预测。
我的训练数据是这样的：
bukrs;lifnr;waers;location
5280;1004008999;EUR;0
5280;1004009000;EUR;2
5280;1004003061;EUR;1
...
而且我可以成功训练模型并保存它，结果是 saved_model.pb 和变量文件夹。
到目前为止一切顺利。
我已经检查过模型本身是否像这样工作：
saved_model_cli show --dir 1561324458 --all
这给了我：
MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:
signature_def['classification']:
The given SavedModel SignatureDef contains the following input(s):
inputs['inputs'] tensor_info:
dtype: DT_STRING
shape: (-1)
name: input_example_tensor:0
The given SavedModel SignatureDef contains the following output(s):
outputs['classes'] tensor_info:
dtype: DT_STRING
shape: (-1, 4)
name: head/Tile:0
outputs['scores'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 4)
name: head/predictions/probabilities:0
Method name is: tensorflow/serving/classify
signature_def['predict']:
The given SavedModel SignatureDef contains the following input(s):
inputs['examples'] tensor_info:
dtype: DT_STRING
shape: (-1)
name: input_example_tensor:0
The given SavedModel SignatureDef contains the following output(s):
outputs['all_class_ids'] tensor_info:
dtype: DT_INT32
shape: (-1, 4)
name: head/predictions/Tile:0
outputs['all_classes'] tensor_info:
dtype: DT_STRING
shape: (-1, 4)
name: head/predictions/Tile_1:0
outputs['class_ids'] tensor_info:
dtype: DT_INT64
shape: (-1, 1)
name: head/predictions/ExpandDims:0
outputs['classes'] tensor_info:
dtype: DT_STRING
shape: (-1, 1)
name: head/predictions/str_classes:0
outputs['logits'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 4)
name: linear/linear_model/linear/linear_model/linear/linear_model/weighted_sum:0
outputs['probabilities'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 4)
name: head/predictions/probabilities:0
Method name is: tensorflow/serving/predict
signature_def['serving_default']:
The given SavedModel SignatureDef contains the following input(s):
inputs['inputs'] tensor_info:
dtype: DT_STRING
shape: (-1)
name: input_example_tensor:0
The given SavedModel SignatureDef contains the following output(s):
outputs['classes'] tensor_info:
dtype: DT_STRING
shape: (-1, 4)
name: head/Tile:0
outputs['scores'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 4)
name: head/predictions/probabilities:0
Method name is: tensorflow/serving/classify
这对我来说很不错。
这是我的整个训练python脚本：
from __future__ import absolute_import, division, print_function, unicode_literals
import tensorflow as tf
import matplotlib.pyplot as plt
import numpy as np
import os
import pandas as pd
import re
import seaborn as sns
from tensorflow import feature_column
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report
import itertools
from itertools import islice
#read data
dataframe = pd.read_csv('invoices_classed2.csv', sep=';',header=0)
dataframe.head()
#cut in sets
train, test = train_test_split(dataframe, test_size=0.3)
train, val = train_test_split(train, test_size=0.3)
#print metrics
print(len(train), 'train examples')
print(len(val), 'validation examples')
print(len(test), 'test examples')
# A utility method to create a tf.data dataset from a Pandas Dataframe
labels = pd.Series();
def df_to_dataset(dataframe, shuffle=False, batch_size=32):
dataframe = dataframe.copy()
labels = dataframe.pop('location')
ds = tf.data.Dataset.from_tensor_slices((dict(dataframe), labels))
if shuffle:
ds = ds.shuffle(buffer_size=len(dataframe))
ds = ds.batch(batch_size)
return ds
# A utility method to create a tf.data dataset from a Pandas Dataframe and use it as functional variable
def make_input_fn(dataframe=None, n_epochs=None, shuffle=False, batch_size=32):
def input_fn():
internal_dataframe = dataframe.copy()
labels = internal_dataframe.pop('location')
ds = tf.data.Dataset.from_tensor_slices((dict(internal_dataframe), labels))
if shuffle:
ds = ds.shuffle(buffer_size=len(internal_dataframe))
ds = ds.repeat(n_epochs)
ds = ds.batch(batch_size)
return ds
return input_fn
#building feature columns
bukrs = feature_column.numeric_column("bukrs")
lifnr = feature_column.categorical_column_with_vocabulary_list(
'lifnr',['1004000409','1004003061','1004008999','1004009001','1004009000','1004003768','1004009002'])
lifnr_one_hot = feature_column.indicator_column(lifnr)
waers = feature_column.categorical_column_with_vocabulary_list(
'waers', ['EUR', 'GBP', 'USD','JPY','CZK','HUF'])
waers_one_hot = feature_column.indicator_column(waers)
actual_feature_columns = []
actual_feature_columns.append(bukrs)
actual_feature_columns.append(lifnr_one_hot)
actual_feature_columns.append(waers_one_hot)
#making datasets
train_ds = make_input_fn(train)
val_ds = make_input_fn(val)
test_ds = make_input_fn(test)
print ('####################creating model####################')
linear_est = tf.estimator.LinearClassifier(feature_columns=actual_feature_columns,n_classes=4,model_dir="C:\Users\70D4867\Desktop\invoicemodel")
print ('####################Train model####################')
#Train model.
linear_est.train(train_ds,max_steps=10000)
print ('####################Evaluation####################')
# Evaluation.
result = linear_est.evaluate(val_ds, steps=1000)
print ('####################printing result####################')
print(result)
print ('####################Done evaluating####################')
for key in sorted(result):
print (key, result[key])
print ('####################predictions####################')
y_generator = linear_est.predict(test_ds)
print ('####################slice predictions####################')
predictions = list(itertools.islice(y_generator,len(test)))
print ('####################predictions output####################')
final_preds = []
template = ('\nPrediction is "{}" ({:.1f}%)')
i = 0;
for pred in (predictions):
final_preds.append(pred['class_ids'][0])
class_id = pred['class_ids'][0]
probability = pred['probabilities'][class_id]
i = i +1
expected = []
for index, row in test.iterrows():
expected.append(row['location'])
print ('####################Test Results####################')
print(classification_report(expected,final_preds))
print ('####################Saving Model####################')
feature_spec = tf.feature_column.make_parse_example_spec(actual_feature_columns)
print(feature_spec)
my_serving_input_receiver_fn = tf.estimator.export.build_parsing_serving_input_receiver_fn(feature_spec)
linear_est.export_saved_model(export_dir_base="invoicemodel\1",serving_input_receiver_fn=my_serving_input_receiver_fn)
但是当我想从这样的模型中得到预测时：
saved_model_cli run --dir invoicemodel61324458 --tag_set serve --signature_def predict --input_examples 'examples=[{"bukrs": 5280, "lifnr": "1004003930", "waers": "EUR"}]'
我希望是这样的：
[0]
我收到错误：
NameError: name 'bukrs' is not defined
我也试着给它提供一个 .npy 文件。 我用一些训练数据创建了这个文件：
bukrs;lifnr;waers
5280;1004008999;EUR
5280;1004009000;EUR
5280;1004003061;EUR
...
像这样：
csv_fn = "invoices_classed_npy.csv"
file = pd.read_csv(csv_fn)
np.save('invoices_classed_npy.npy', file, allow_pickle = True);
但是当我尝试时：
saved_model_cli run --dir .\invoicemodel61324458 --tag_set serve --signature_def classification --inputs 'inputs="invoices_classed_npy.npy"'
我预计
[1],[2],[0]
我得到了：
ValueError: Cannot feed value of shape (55276, 1) for Tensor
'input_example_tensor:0', which has shape '(None,)'
好的，所以我也尝试在容器中提供它：
docker run -t --rm -p 8501:8501 \
> -v "/data/container/tensorflow/model:/models/saved_model" \
> -e MODEL_NAME=saved_model \
> tensorflow/serving
我得到了：
/usr/bin/tf_serving_entrypoint.sh: line 3: 6 Illegal instruction (core dumped) tensorflow_model_server --port=8500 --rest_api_port=8501 --model_name=${MODEL_NAME} --model_base_path=${MODEL_BASE_PATH}/${MODEL_NAME} "$@"
我做错了什么？ 我如何正确地从我的模型中得到预测？
我知道我的问题是什么了。
您必须在训练模型之前指定 serving_input_receiver_fn，并且以某种方式将其从特征列中拉出是行不通的。
这是我的脚本末尾不起作用的部分：
feature_spec = tf.feature_column.make_parse_example_spec(actual_feature_columns)
my_serving_input_receiver_fn = tf.estimator.export.build_parsing_serving_input_receiver_fn(feature_spec)
linear_est.export_saved_model(export_dir_base="invoicemodel\1",serving_input_receiver_fn=my_serving_input_receiver_fn)
这导致 signature_def 的预测看起来像：
signature_def['predict']:
The given SavedModel SignatureDef contains the following input(s):
inputs['examples'] tensor_info:
dtype: DT_STRING
shape: (-1)
name: input_example_tensor:0
以及错误：
NameError: name 'bukrs' is not defined
我把 serving_input_receiver_fn 的定义改成这样
feature_placeholder = {'bukrs': tf.placeholder(tf.int32, [1],
name='bukrs_placeholder'),'lifnr': tf.placeholder('string', [1],
name='lifnr_placeholder'),'waers': tf.placeholder('string', [1],
name='waers_placeholder')}
my_serving_input_receiver_fn =
tf.estimator.export.build_raw_serving_input_receiver_fn(feature_placeholder)
####now train the model####
linear_est.train(train_ds,max_steps=10000)
#evaluation ...
#test ....
#### then save at the end ###
linear_est.export_saved_model(export_dir_base="invoicemodel\1",serving_input_receiver_fn=my_serving_input_receiver_fn)
导致模型的 signature_def 正确：
signature_def['predict']:
The given SavedModel SignatureDef contains the following input(s):
inputs['bukrs'] tensor_info:
dtype: DT_INT32
shape: (-1)
name: bukrs_placeholder:0
inputs['lifnr'] tensor_info:
dtype: DT_STRING
shape: (-1)
name: lifnr_placeholder:0
inputs['waers'] tensor_info:
dtype: DT_STRING
shape: (-1)
name: waers_placeholder:0
现在我可以从模型中得到预测：
saved_model_cli run --dir 61727347 --tag_set serve --signature_def predict --input_exprs="bukrs=[5280];lifnr=['1004002578'];waers=['EUR']"
这导致预测：
Result for output key all_class_ids:
[[0 1 2 3]]
Result for output key all_classes:
[[b'0' b'1' b'2' b'3']]
Result for output key class_ids:
[[0]]
Result for output key classes:
[[b'0']]
Result for output key logits:
[[493.99664 475.6605 482.86667 483.922 ]]
Result for output key probabilities:
[[9.99943256e-01 1.08814335e-08 1.46652310e-05 4.21320110e-05]]
我希望这对某人有所帮助。