Official ZeroOut gradient example error: AttributeError: 'list' object has no attribute 'eval'

Question

按照tensorflow官网的官方教程：https://www.tensorflow.org/extend/adding_an_op 还描述了如何调用教程中示例 ZeroOut 的渐变，我想在下面的这个简短代码片段中尝试。

我在这里找到了代码：https://github.com/MatteoRagni/tf.ZeroOut.gpu

import numpy as np
import tensorflow as tf
from tensorflow.python.framework import ops
from tensorflow.python.ops import array_ops
from tensorflow.python.ops import sparse_ops
zero_out_module = tf.load_op_library('./libzeroout.so')
@ops.RegisterGradient("ZeroOut")
def _zero_out_grad(op, grad):
     to_zero = op.inputs[0]
     shape = array_ops.shape(to_zero)
     index = array_ops.zeros_like(shape)
     first_grad = array_ops.reshape(grad, [-1])[0]
     to_zero_grad = sparse_ops.sparse_to_dense([index], shape, first_grad, 0)
     return [to_zero_grad]  # List of one Tensor, since we have one input

t_in = tf.placeholder(tf.int32, [None,None])
ret = zero_out_module.zero_out(t_in)
grad = tf.gradients(ys=tf.reduce_sum(ret), xs=t_in)
with tf.Session(''):
    feed_dict = {t_in: [[1, 2], [3, 4]]}
    print "ret val: ", ret.eval(feed_dict=feed_dict)
    print "grad: ", grad
    print "grad: ", grad.eval(feed_dict=feed_dict)

我收到这个错误...

AttributeError: 'list' object has no attribute 'eval'

...但我可以做到 ret.eval()。

为什么我不能调用 grad.eval()？我想在渐变张量中看到这些值。如何调试梯度？

Answer 1

老问题的答案

实施

def _zero_out_grad(op, *grads):
    topdiff = grads[0]
    bottom = op.inputs[0]

    shape = array_ops.shape(bottom)
    index = array_ops.zeros_like(shape)
    first_grad = array_ops.reshape(topdiff, [-1])[0]
    to_zero_grad = sparse_ops.sparse_to_dense([index], shape, first_grad, 0)

    return to_zero_grad

在这里工作得很好。您确定在 tf.Session() 之前执行“@ops.RegisterGradient("ZeroOut")”吗？

通常

zero_out_module = tf.load_op_library('./libzeroout.so')
@ops.RegisterGradient("ZeroOut")
def _zero_out_grad(op, grad):
    # ...

放在不同的文件中，刚刚导入。即使使用最新的 TensorFlow 版本，一个完整的工作示例是 here.

完全改变问题的答案

你的渐变函数returns一个列表和一个Python列表没有'eval()'。试试：

grad = tf.gradients(ys=tf.reduce_sum(ret), xs=t_in)[0]

或遵循最佳实践并使用

grad = tf.gradients(ys=tf.reduce_sum(ret), xs=t_in)
with tf.Session() as sess:
    sess.run(grad, feed_dict=feed_dict)

请不要更改您的整个问题

Official ZeroOut gradient example error: AttributeError: 'list' object has no attribute 'eval'

Official ZeroOut gradient example error: AttributeError: 'list' object has no attribute 'eval'

gradient

python-2.7

tensorflow