如何使用 tf.estimator 导入已保存的 Tensorflow 模型训练并预测输入数据
How to import an saved Tensorflow model train using tf.estimator and predict on input data
我使用 tf.estimator.method export_savedmodel 保存模型如下：
export_dir="exportModel/"
feature_spec = tf.feature_column.make_parse_example_spec(feature_columns)
input_receiver_fn = tf.estimator.export.build_parsing_serving_input_receiver_fn(feature_spec)
classifier.export_savedmodel(export_dir, input_receiver_fn, as_text=False, checkpoint_path="Model/model.ckpt-400")
如何导入这个保存的模型并用于预测？
我试图搜索一个好的基础示例，但似乎该主题的文档和示例有点分散。那么让我们从一个基本示例开始：tf.estimator quickstart.
该特定示例实际上并未导出模型，所以让我们这样做（用例 1 不需要）：
def serving_input_receiver_fn():
"""Build the serving inputs."""
# The outer dimension (None) allows us to batch up inputs for
# efficiency. However, it also means that if we want a prediction
# for a single instance, we'll need to wrap it in an outer list.
inputs = {"x": tf.placeholder(shape=[None, 4], dtype=tf.float32)}
return tf.estimator.export.ServingInputReceiver(inputs, inputs)
export_dir = classifier.export_savedmodel(
export_dir_base="/path/to/model",
serving_input_receiver_fn=serving_input_receiver_fn)
此代码上的巨大星号：TensorFlow 1.3 中似乎存在一个错误，不允许您在 "canned" 估算器上执行上述导出操作（例如 DNNClassifier）。有关解决方法，请参阅 "Appendix: Workaround" 部分。
下面的代码引用了 export_dir（导出步骤中的 return 值）以强调它是 而不是 “/path/to/model”，而是该目录的一个子目录，其名称是一个时间戳。
用例 1：在与训练相同的过程中执行预测
这是一种 sci-kit 学习类型的体验，并且已经通过示例进行了举例说明。为了完整起见，您只需在训练模型上调用 predict：
classifier.train(input_fn=train_input_fn, steps=2000)
# [...snip...]
predictions = list(classifier.predict(input_fn=predict_input_fn))
predicted_classes = [p["classes"] for p in predictions]
用例 2：将 SavedModel 加载到 Python/Java/C++ 并执行预测
Python 客户端
如果您想在 Python 中进行预测，也许最简单的方法是 SavedModelPredictor。在将使用 SavedModel 的 Python 程序中，我们需要这样的代码：
from tensorflow.contrib import predictor
predict_fn = predictor.from_saved_model(export_dir)
predictions = predict_fn(
{"x": [[6.4, 3.2, 4.5, 1.5],
[5.8, 3.1, 5.0, 1.7]]})
print(predictions['scores'])
Java 客户端
package dummy;
import java.nio.FloatBuffer;
import java.util.Arrays;
import java.util.List;
import org.tensorflow.SavedModelBundle;
import org.tensorflow.Session;
import org.tensorflow.Tensor;
public class Client {
public static void main(String[] args) {
Session session = SavedModelBundle.load(args[0], "serve").session();
Tensor x =
Tensor.create(
new long[] {2, 4},
FloatBuffer.wrap(
new float[] {
6.4f, 3.2f, 4.5f, 1.5f,
5.8f, 3.1f, 5.0f, 1.7f
}));
// Doesn't look like Java has a good way to convert the
// input/output name ("x", "scores") to their underlying tensor,
// so we hard code them ("Placeholder:0", ...).
// You can inspect them on the command-line with saved_model_cli:
//
// $ saved_model_cli show --dir $EXPORT_DIR --tag_set serve --signature_def serving_default
final String xName = "Placeholder:0";
final String scoresName = "dnn/head/predictions/probabilities:0";
List<Tensor> outputs = session.runner()
.feed(xName, x)
.fetch(scoresName)
.run();
// Outer dimension is batch size; inner dimension is number of classes
float[][] scores = new float[2][3];
outputs.get(0).copyTo(scores);
System.out.println(Arrays.deepToString(scores));
}
}
C++ 客户端
您可能想要使用 tensorflow::LoadSavedModel with Session。
#include <unordered_set>
#include <utility>
#include <vector>
#include "tensorflow/cc/saved_model/loader.h"
#include "tensorflow/core/framework/tensor.h"
#include "tensorflow/core/public/session.h"
namespace tf = tensorflow;
int main(int argc, char** argv) {
const string export_dir = argv[1];
tf::SavedModelBundle bundle;
tf::Status load_status = tf::LoadSavedModel(
tf::SessionOptions(), tf::RunOptions(), export_dir, {"serve"}, &bundle);
if (!load_status.ok()) {
std::cout << "Error loading model: " << load_status << std::endl;
return -1;
}
// We should get the signature out of MetaGraphDef, but that's a bit
// involved. We'll take a shortcut like we did in the Java example.
const string x_name = "Placeholder:0";
const string scores_name = "dnn/head/predictions/probabilities:0";
auto x = tf::Tensor(tf::DT_FLOAT, tf::TensorShape({2, 4}));
auto matrix = x.matrix<float>();
matrix(0, 0) = 6.4;
matrix(0, 1) = 3.2;
matrix(0, 2) = 4.5;
matrix(0, 3) = 1.5;
matrix(0, 1) = 5.8;
matrix(0, 2) = 3.1;
matrix(0, 3) = 5.0;
matrix(0, 4) = 1.7;
std::vector<std::pair<string, tf::Tensor>> inputs = {{x_name, x}};
std::vector<tf::Tensor> outputs;
tf::Status run_status =
bundle.session->Run(inputs, {scores_name}, {}, &outputs);
if (!run_status.ok()) {
cout << "Error running session: " << run_status << std::endl;
return -1;
}
for (const auto& tensor : outputs) {
std::cout << tensor.matrix<float>() << std::endl;
}
}
用例 3：使用 TensorFlow Serving 为模型提供服务
以服务于 Classification model 的方式导出模型要求输入是 tf.Example 对象。以下是我们如何导出 TensorFlow 服务模型：
def serving_input_receiver_fn():
"""Build the serving inputs."""
# The outer dimension (None) allows us to batch up inputs for
# efficiency. However, it also means that if we want a prediction
# for a single instance, we'll need to wrap it in an outer list.
example_bytestring = tf.placeholder(
shape=[None],
dtype=tf.string,
)
features = tf.parse_example(
example_bytestring,
tf.feature_column.make_parse_example_spec(feature_columns)
)
return tf.estimator.export.ServingInputReceiver(
features, {'examples': example_bytestring})
export_dir = classifier.export_savedmodel(
export_dir_base="/path/to/model",
serving_input_receiver_fn=serving_input_receiver_fn)
reader参考了TensorFlow Serving的文档，了解更多关于如何设置TensorFlow Serving的说明，所以我这里只提供客户端代码：
# Omitting a bunch of connection/initialization code...
# But at some point we end up with a stub whose lifecycle
# is generally longer than that of a single request.
stub = create_stub(...)
# The actual values for prediction. We have two examples in this
# case, each consisting of a single, multi-dimensional feature `x`.
# This data here is the equivalent of the map passed to the
# `predict_fn` in use case #2.
examples = [
tf.train.Example(
features=tf.train.Features(
feature={"x": tf.train.Feature(
float_list=tf.train.FloatList(value=[6.4, 3.2, 4.5, 1.5]))})),
tf.train.Example(
features=tf.train.Features(
feature={"x": tf.train.Feature(
float_list=tf.train.FloatList(value=[5.8, 3.1, 5.0, 1.7]))})),
]
# Build the RPC request.
predict_request = predict_pb2.PredictRequest()
predict_request.model_spec.name = "default"
predict_request.inputs["examples"].CopyFrom(
tensor_util.make_tensor_proto(examples, tf.float32))
# Perform the actual prediction.
stub.Predict(request, PREDICT_DEADLINE_SECS)
请注意，predict_request.inputs 中引用的密钥 examples 需要在导出时与 serving_input_receiver_fn 中使用的密钥相匹配（参见构造函数 ServingInputReceiver 在该代码中）。
附录：解决 TF 1.3 中固定模型的导出问题
TensorFlow 1.3 中似乎存在一个错误，其中固定模型无法为用例 2 正确导出（"custom" 估计器不存在该问题）。这是包装 DNNClassifier 以使其正常工作的解决方法，特别是对于 Iris 示例：
# Build 3 layer DNN with 10, 20, 10 units respectively.
class Wrapper(tf.estimator.Estimator):
def __init__(self, **kwargs):
dnn = tf.estimator.DNNClassifier(**kwargs)
def model_fn(mode, features, labels):
spec = dnn._call_model_fn(features, labels, mode)
export_outputs = None
if spec.export_outputs:
export_outputs = {
"serving_default": tf.estimator.export.PredictOutput(
{"scores": spec.export_outputs["serving_default"].scores,
"classes": spec.export_outputs["serving_default"].classes})}
# Replace the 3rd argument (export_outputs)
copy = list(spec)
copy[4] = export_outputs
return tf.estimator.EstimatorSpec(mode, *copy)
super(Wrapper, self).__init__(model_fn, kwargs["model_dir"], dnn.config)
classifier = Wrapper(feature_columns=feature_columns,
hidden_units=[10, 20, 10],
n_classes=3,
model_dir="/tmp/iris_model")
TensorFlow 团队似乎不同意在 1.3 版中使用固定估计器在用例 #2 下导出模型的错误。我在这里提交了错误报告：
https://github.com/tensorflow/tensorflow/issues/13477
我从 TensorFlow 收到的回复是输入只能是一个字符串张量。似乎有一种方法可以使用序列化 TF.examples 将多个特征合并到一个字符串张量中，但我还没有找到一个明确的方法来做到这一点。如果有人有显示如何执行此操作的代码，我将不胜感激。
我认为预装 Estimator 没有错误（或者更确切地说，如果有的话，它已经被修复了）。我能够使用 Python 成功导出固定估算器模型并将其导入 Java。
这是我导出模型的代码：
a = tf.feature_column.numeric_column("a");
b = tf.feature_column.numeric_column("b");
feature_columns = [a, b];
model = tf.estimator.DNNClassifier(feature_columns=feature_columns ...);
# To export
feature_spec = tf.feature_column.make_parse_example_spec(feature_columns);
export_input_fn = tf.estimator.export.build_parsing_serving_input_receiver_fn(feature_spec);
servable_model_path = model.export_savedmodel(servable_model_dir, export_input_fn, as_text=True);
为了在Java中导入模型，我使用了上面rhaertel80提供的Java客户端代码并且它有效。希望这也能回答 Ben Fowler 上面的问题。
您需要使用 tf.contrib.export_savedmodel 导出保存的模型，并且您需要定义输入接收器函数以将输入传递给。
稍后您可以从磁盘加载保存的模型（通常saved.model.pb）并提供它。
我使用 tf.estimator.method export_savedmodel 保存模型如下：
export_dir="exportModel/"
feature_spec = tf.feature_column.make_parse_example_spec(feature_columns)
input_receiver_fn = tf.estimator.export.build_parsing_serving_input_receiver_fn(feature_spec)
classifier.export_savedmodel(export_dir, input_receiver_fn, as_text=False, checkpoint_path="Model/model.ckpt-400")
如何导入这个保存的模型并用于预测？
我试图搜索一个好的基础示例，但似乎该主题的文档和示例有点分散。那么让我们从一个基本示例开始：tf.estimator quickstart.
该特定示例实际上并未导出模型，所以让我们这样做（用例 1 不需要）：
def serving_input_receiver_fn():
"""Build the serving inputs."""
# The outer dimension (None) allows us to batch up inputs for
# efficiency. However, it also means that if we want a prediction
# for a single instance, we'll need to wrap it in an outer list.
inputs = {"x": tf.placeholder(shape=[None, 4], dtype=tf.float32)}
return tf.estimator.export.ServingInputReceiver(inputs, inputs)
export_dir = classifier.export_savedmodel(
export_dir_base="/path/to/model",
serving_input_receiver_fn=serving_input_receiver_fn)
此代码上的巨大星号：TensorFlow 1.3 中似乎存在一个错误，不允许您在 "canned" 估算器上执行上述导出操作（例如 DNNClassifier）。有关解决方法，请参阅 "Appendix: Workaround" 部分。
下面的代码引用了 export_dir（导出步骤中的 return 值）以强调它是 而不是 “/path/to/model”，而是该目录的一个子目录，其名称是一个时间戳。
用例 1：在与训练相同的过程中执行预测
这是一种 sci-kit 学习类型的体验，并且已经通过示例进行了举例说明。为了完整起见，您只需在训练模型上调用 predict：
classifier.train(input_fn=train_input_fn, steps=2000)
# [...snip...]
predictions = list(classifier.predict(input_fn=predict_input_fn))
predicted_classes = [p["classes"] for p in predictions]
用例 2：将 SavedModel 加载到 Python/Java/C++ 并执行预测
Python 客户端
如果您想在 Python 中进行预测，也许最简单的方法是 SavedModelPredictor。在将使用 SavedModel 的 Python 程序中，我们需要这样的代码：
from tensorflow.contrib import predictor
predict_fn = predictor.from_saved_model(export_dir)
predictions = predict_fn(
{"x": [[6.4, 3.2, 4.5, 1.5],
[5.8, 3.1, 5.0, 1.7]]})
print(predictions['scores'])
Java 客户端
package dummy;
import java.nio.FloatBuffer;
import java.util.Arrays;
import java.util.List;
import org.tensorflow.SavedModelBundle;
import org.tensorflow.Session;
import org.tensorflow.Tensor;
public class Client {
public static void main(String[] args) {
Session session = SavedModelBundle.load(args[0], "serve").session();
Tensor x =
Tensor.create(
new long[] {2, 4},
FloatBuffer.wrap(
new float[] {
6.4f, 3.2f, 4.5f, 1.5f,
5.8f, 3.1f, 5.0f, 1.7f
}));
// Doesn't look like Java has a good way to convert the
// input/output name ("x", "scores") to their underlying tensor,
// so we hard code them ("Placeholder:0", ...).
// You can inspect them on the command-line with saved_model_cli:
//
// $ saved_model_cli show --dir $EXPORT_DIR --tag_set serve --signature_def serving_default
final String xName = "Placeholder:0";
final String scoresName = "dnn/head/predictions/probabilities:0";
List<Tensor> outputs = session.runner()
.feed(xName, x)
.fetch(scoresName)
.run();
// Outer dimension is batch size; inner dimension is number of classes
float[][] scores = new float[2][3];
outputs.get(0).copyTo(scores);
System.out.println(Arrays.deepToString(scores));
}
}
C++ 客户端
您可能想要使用 tensorflow::LoadSavedModel with Session。
#include <unordered_set>
#include <utility>
#include <vector>
#include "tensorflow/cc/saved_model/loader.h"
#include "tensorflow/core/framework/tensor.h"
#include "tensorflow/core/public/session.h"
namespace tf = tensorflow;
int main(int argc, char** argv) {
const string export_dir = argv[1];
tf::SavedModelBundle bundle;
tf::Status load_status = tf::LoadSavedModel(
tf::SessionOptions(), tf::RunOptions(), export_dir, {"serve"}, &bundle);
if (!load_status.ok()) {
std::cout << "Error loading model: " << load_status << std::endl;
return -1;
}
// We should get the signature out of MetaGraphDef, but that's a bit
// involved. We'll take a shortcut like we did in the Java example.
const string x_name = "Placeholder:0";
const string scores_name = "dnn/head/predictions/probabilities:0";
auto x = tf::Tensor(tf::DT_FLOAT, tf::TensorShape({2, 4}));
auto matrix = x.matrix<float>();
matrix(0, 0) = 6.4;
matrix(0, 1) = 3.2;
matrix(0, 2) = 4.5;
matrix(0, 3) = 1.5;
matrix(0, 1) = 5.8;
matrix(0, 2) = 3.1;
matrix(0, 3) = 5.0;
matrix(0, 4) = 1.7;
std::vector<std::pair<string, tf::Tensor>> inputs = {{x_name, x}};
std::vector<tf::Tensor> outputs;
tf::Status run_status =
bundle.session->Run(inputs, {scores_name}, {}, &outputs);
if (!run_status.ok()) {
cout << "Error running session: " << run_status << std::endl;
return -1;
}
for (const auto& tensor : outputs) {
std::cout << tensor.matrix<float>() << std::endl;
}
}
用例 3：使用 TensorFlow Serving 为模型提供服务
以服务于 Classification model 的方式导出模型要求输入是 tf.Example 对象。以下是我们如何导出 TensorFlow 服务模型：
def serving_input_receiver_fn():
"""Build the serving inputs."""
# The outer dimension (None) allows us to batch up inputs for
# efficiency. However, it also means that if we want a prediction
# for a single instance, we'll need to wrap it in an outer list.
example_bytestring = tf.placeholder(
shape=[None],
dtype=tf.string,
)
features = tf.parse_example(
example_bytestring,
tf.feature_column.make_parse_example_spec(feature_columns)
)
return tf.estimator.export.ServingInputReceiver(
features, {'examples': example_bytestring})
export_dir = classifier.export_savedmodel(
export_dir_base="/path/to/model",
serving_input_receiver_fn=serving_input_receiver_fn)
reader参考了TensorFlow Serving的文档，了解更多关于如何设置TensorFlow Serving的说明，所以我这里只提供客户端代码：
# Omitting a bunch of connection/initialization code...
# But at some point we end up with a stub whose lifecycle
# is generally longer than that of a single request.
stub = create_stub(...)
# The actual values for prediction. We have two examples in this
# case, each consisting of a single, multi-dimensional feature `x`.
# This data here is the equivalent of the map passed to the
# `predict_fn` in use case #2.
examples = [
tf.train.Example(
features=tf.train.Features(
feature={"x": tf.train.Feature(
float_list=tf.train.FloatList(value=[6.4, 3.2, 4.5, 1.5]))})),
tf.train.Example(
features=tf.train.Features(
feature={"x": tf.train.Feature(
float_list=tf.train.FloatList(value=[5.8, 3.1, 5.0, 1.7]))})),
]
# Build the RPC request.
predict_request = predict_pb2.PredictRequest()
predict_request.model_spec.name = "default"
predict_request.inputs["examples"].CopyFrom(
tensor_util.make_tensor_proto(examples, tf.float32))
# Perform the actual prediction.
stub.Predict(request, PREDICT_DEADLINE_SECS)
请注意，predict_request.inputs 中引用的密钥 examples 需要在导出时与 serving_input_receiver_fn 中使用的密钥相匹配（参见构造函数 ServingInputReceiver 在该代码中）。
附录：解决 TF 1.3 中固定模型的导出问题
TensorFlow 1.3 中似乎存在一个错误，其中固定模型无法为用例 2 正确导出（"custom" 估计器不存在该问题）。这是包装 DNNClassifier 以使其正常工作的解决方法，特别是对于 Iris 示例：
# Build 3 layer DNN with 10, 20, 10 units respectively.
class Wrapper(tf.estimator.Estimator):
def __init__(self, **kwargs):
dnn = tf.estimator.DNNClassifier(**kwargs)
def model_fn(mode, features, labels):
spec = dnn._call_model_fn(features, labels, mode)
export_outputs = None
if spec.export_outputs:
export_outputs = {
"serving_default": tf.estimator.export.PredictOutput(
{"scores": spec.export_outputs["serving_default"].scores,
"classes": spec.export_outputs["serving_default"].classes})}
# Replace the 3rd argument (export_outputs)
copy = list(spec)
copy[4] = export_outputs
return tf.estimator.EstimatorSpec(mode, *copy)
super(Wrapper, self).__init__(model_fn, kwargs["model_dir"], dnn.config)
classifier = Wrapper(feature_columns=feature_columns,
hidden_units=[10, 20, 10],
n_classes=3,
model_dir="/tmp/iris_model")
TensorFlow 团队似乎不同意在 1.3 版中使用固定估计器在用例 #2 下导出模型的错误。我在这里提交了错误报告： https://github.com/tensorflow/tensorflow/issues/13477
我从 TensorFlow 收到的回复是输入只能是一个字符串张量。似乎有一种方法可以使用序列化 TF.examples 将多个特征合并到一个字符串张量中，但我还没有找到一个明确的方法来做到这一点。如果有人有显示如何执行此操作的代码，我将不胜感激。
我认为预装 Estimator 没有错误（或者更确切地说，如果有的话，它已经被修复了）。我能够使用 Python 成功导出固定估算器模型并将其导入 Java。
这是我导出模型的代码：
a = tf.feature_column.numeric_column("a");
b = tf.feature_column.numeric_column("b");
feature_columns = [a, b];
model = tf.estimator.DNNClassifier(feature_columns=feature_columns ...);
# To export
feature_spec = tf.feature_column.make_parse_example_spec(feature_columns);
export_input_fn = tf.estimator.export.build_parsing_serving_input_receiver_fn(feature_spec);
servable_model_path = model.export_savedmodel(servable_model_dir, export_input_fn, as_text=True);
为了在Java中导入模型，我使用了上面rhaertel80提供的Java客户端代码并且它有效。希望这也能回答 Ben Fowler 上面的问题。
您需要使用 tf.contrib.export_savedmodel 导出保存的模型，并且您需要定义输入接收器函数以将输入传递给。 稍后您可以从磁盘加载保存的模型（通常saved.model.pb）并提供它。