Tensorflow：如何在优化结束前保存模型？

Question

我正在使用 ScipyOptimizerInterface 训练 tensorflow 模型。 (tensorflow 1.13.1)

在训练过程中，如果 loss 值低于阈值，我希望训练过程在超过阈值之前停止并保存模型。

下面是我试过的脚本。这个想法是引发异常以退出 optimizer.minimize，然后使用 tf.train.Saver 保存模型。

但是，这不起作用。通过比较 初始 loss 值 和 保存模型 计算的 loss 值，您可以看到。这两个值相同，表示保存的是初始随机模型，而不是所需模型。

根据@Patol75 的回答，我了解到最佳模型未保存，因为更新的 tf.Variables 在解释训练课程时死亡。

如何保存想要的模型？

import numpy as np

import tensorflow as tf
from tensorflow.contrib.opt import ScipyOptimizerInterface

class test(Exception):
    pass

def construct_graph():
    graph = tf.Graph()
    with graph.as_default():
        x = tf.placeholder('float', shape = [None, 1])
        w = tf.get_variable('w_0', shape = [1, 1], initializer = tf.contrib.layers.xavier_initializer())
        b = tf.get_variable('b_0', shape = [1], initializer = tf.contrib.layers.xavier_initializer())
        y_out = tf.matmul(x, w) + b
        y = tf.placeholder('float', shape = [None, 1])
        loss = tf.reduce_mean(tf.square(y - y_out))
    return graph, x, y, loss

# create example datasets
x_train = np.linspace(1, 6, 100) + 0.1 * np.random.random(100)
x_train = x_train.reshape(100, 1)
y_train = np.sin(x_train)
x_val = np.linspace(6, 11, 100)
x_val = x_val.reshape(100, 1)
y_val = np.sin(x_val)

tf.reset_default_graph()
graph, x, y, loss = construct_graph()
feeddict_train = {x: x_train, y: y_train}
feeddict_val = {x: x_val, y: y_val}

with graph.as_default():

    def step_callbackfun(x):
        global iteration
        train_part, val_part = valfunc_train(x), valfunc_val(x) 
        print('%10.5f %10.5f' % (*train_part, *val_part))
        iteration += 1
        if iteration == 5:
            raise test()

    sess = tf.Session()
    saver = tf.train.Saver()
    sess.run(tf.global_variables_initializer())
    optimizer = ScipyOptimizerInterface(loss, method='l-BFGS-b')

    iteration = 0
    valfunc_train = optimizer._make_eval_func(tensors=loss, session=sess, feed_dict=feeddict_train, fetches=[])
    valfunc_val = optimizer._make_eval_func(tensors=loss, session=sess, feed_dict=feeddict_val, fetches=[])

    print('The initial loss is %f' % sess.run(loss, feeddict_train))    
    try:
        optimizer.minimize(sess, feeddict_train, step_callback=step_callbackfun)
    except test:
        saver.save(sess, 'model/model.ckpt')
        graph2, x2, y2, loss2 = construct_graph()
        with tf.Session(graph=graph2) as sess2:
            feeddict_two = {x2: x_train, y2: y_train}
            sess2.run(tf.global_variables_initializer())
            saver2 = tf.train.Saver()
            saver2.restore(sess2, 'model/model.ckpt')
            loss_val2 = sess2.run(loss2, feeddict_two)
            print('Outside', loss_val2)

Answer 1

看看doc

Variables subject to optimization are updated in-place at the end of optimization.

这就是为什么以您的方式中断流程确实会给您带来原始价值的原因。我从未使用过 TensorFlow，但关键字 loss_callback 听起来很有希望。

A function to be called every time the loss and gradients are computed, with evaluated fetches supplied as positional arguments.

看看.

Tensorflow：如何在优化结束前保存模型？

Tensorflow: how to save a model before the end of optimization?

python

scipy

tensorflow

scipy-optimize