tf.hessians() - ValueError: None values not supported

Question

这是回溯：

Traceback (most recent call last):
  File "test.py", line 39, in <module>
    hess = tf.hessians(loss, wrt_variables)
  File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/ops/gradients_impl.py", line 970, in hessians
    _gradients = array_ops.unstack(_gradients)
  File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/ops/array_ops.py", line 952, in unstack
    value = ops.convert_to_tensor(value)
  File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/framework/ops.py", line 639, in convert_to_tensor
    as_ref=False)
  File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/framework/ops.py", line 704, in internal_convert_to_tensor
    ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
  File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/framework/constant_op.py", line 113, in _constant_tensor_conversion_function
    return constant(v, dtype=dtype, name=name)
  File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/framework/constant_op.py", line 102, in constant
    tensor_util.make_tensor_proto(value, dtype=dtype, shape=shape, verify_shape=verify_shape))
  File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/framework/tensor_util.py", line 360, in make_tensor_proto
    raise ValueError("None values not supported.")
ValueError: None values not supported.

变量：

import tensorflow as tf

data_x = [0., 1., 2.]
data_y = [-1., 1., 3.]
batch_size = len(data_x)

x = tf.placeholder(shape=[batch_size], dtype=tf.float32, name="x")
y = tf.placeholder(shape=[batch_size], dtype=tf.float32, name="y")

W = tf.Variable(tf.ones(shape=[1]), dtype=tf.float32, name="W")
b = tf.Variable(tf.zeros(shape=[1]), dtype=tf.float32, name="b")

pred = x * W + b
loss = tf.reduce_mean(0.5 * (y - pred)**2)

然后，跟进这段代码就可以了：

wrt_variables = [W, b]
hess = tf.hessians(loss, wrt_variables)

但这失败了：

wrt_variables = tf.concat([W, b], axis=0)
hess = tf.hessians(loss, wrt_variables)

这也会失败：

wrt_variables = [tf.concat([W, b], axis=0)]
hess = tf.hessians(loss, wrt_variables)

重塑操作也失败。

可以在此处查看此代码的完整版本和注释： https://gist.github.com/guillaume-chevalier/6b01c4e43a123abf8db69fa97532993f

谢谢！

Answer 1

这是因为在您的图表中，节点 loss 不依赖于节点 tf.concat([W,b], axis=0)。没有一个到另一个的反向传播，因此没有导数。

Tensorflow 不是正式的微积分引擎，它只能估计一个节点对另一个节点的导数，前提是前者在后者的下游。所以例如偶

tf.hessian(loss, 2*W)

会因为同样的原因失败（2*W 是一个新节点，loss 不依赖于它）即使与 tf.hessian(loss, W) 的关系是直接的。

请注意，情况与 tf.gradients 相同，尽管它失败的方式不同：它 returns None 而不是抛出异常。

tf.hessians() - ValueError: None values not supported

tf.hessians() - ValueError: None values not supported

tensorflow

mathematical-optimization

derivative