TensorFlow：如何检查急切执行中的梯度和权重？

Question

我在 Eager Execution 中使用 TensorFlow 1.12，我想在训练期间检查我的梯度值和我在不同点的权重以进行调试。 uses TensorBoard to get nice graphs of weight and gradient distribution over epochs, which is what I would like. However, when I use Keras' TensorBoard callback，我明白了：

WARNING:tensorflow:Weight and gradient histograms not supported for eagerexecution, setting `histogram_freq` to `0`.

换句话说，这与急切执行不兼容。还有其他方法可以打印渐变 and/or 权重吗？大多数非 TensorBoard 答案似乎都依赖于基于图形的执行。

Answer 1

在eager execution中，可以直接打印权重。至于梯度，您可以使用 tf.GradientTape 来获得损失函数相对于某些权重的梯度。这是一个显示如何打印渐变和权重的示例：

import tensorflow as tf

tf.enable_eager_execution()

x = tf.ones(shape=(4, 3))
y = tf.ones(shape=(4, 1))
dense = tf.layers.Dense(1)

# Print gradients
with tf.GradientTape() as t:
    h = dense(x)
    loss = tf.losses.mean_squared_error(y, h)
gradients = t.gradient(loss, dense.kernel)
print('Gradients: ', gradients)

# Print weights
weights = dense.get_weights()
print('Weights: ', weights)

TensorFlow：如何检查急切执行中的梯度和权重？

TensorFlow: How can I inspect gradients and weights in eager execution?

tensorflow

gradient-descent

eager-execution