如何防止 TensorFlow eval 梯度

Question

假设我有像这样的 MNIST 数据的简单 TensorFlow 模型

import tensorflow as tf
from tensorflow.examples.tutorials.mnist import input_data
mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)

x = tf.placeholder(tf.float32, [None, 784]) 
y_ = tf.placeholder(tf.float32, [None, 10])
W = tf.Variable(tf.zeros([784, 10]))
b = tf.Variable(tf.zeros([10]))

y = tf.nn.softmax(tf.matmul(x, W) + b)

cross_entropy = tf.reduce_mean(-tf.reduce_sum(y_ * tf.log(y), reduction_indices=[1]))
train_step = tf.train.GradientDescentOptimizer(0.5).minimize(cross_entropy)
init = tf.initialize_all_variables()
sess = tf.Session()
sess.run(init)

for i in range(1000):
    batch_xs, batch_ys = mnist.train.next_batch(100)
    sess.run(train_step, feed_dict={x: batch_xs, y_: batch_ys})

训练完成后，我想将测试数据转化为输出

y_transformed = sess.run(y, feed_dict={x: mnist.test.images})

但是，据我了解，这样做会导致 TensorFlow 计算 W 和 b 的梯度。在更复杂的情况下，这是很多开销。那么，如何避免这种梯度计算呢？

Answer 1

Tensor.eval 只是一个 shorthand for Session.run for single tensor，这不会提高这里的性能。

在Session.run的document中说：

This method runs one "step" of TensorFlow computation, by running the necessary graph fragment to execute every Operation and evaluate every Tensor in fetches, substituting the values in feed_dict for the corresponding input values.

而变量y的梯度显然不是必要的片段。因此我不认为它会是运行除非你做一些像用优化器训练它的事情。

如有错误请指正

如何防止 TensorFlow eval 梯度

How to prevent TensorFlow eval gradients

python

machine-learning

tensorflow