如何用tensorflow计算AUC?
How to calculate AUC with tensorflow?
我已经使用 Tensorflow 构建了一个二元分类器,现在我想使用 AUC 和准确性评估分类器。
就准确性而言,我可以很容易地这样做:
X = tf.placeholder('float', [None, n_input])
y = tf.placeholder('float', [None, n_classes])
pred = mlp(X, weights, biases, dropout_keep_prob)
correct_prediction = tf.equal(tf.argmax(pred, 1), tf.argmax(y, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))
计算 AUC 时我使用以下方法:
print(tf.argmax(pred, 1).dtype.name)
print(tf.argmax(pred, 1).dtype.name)
a = tf.cast(tf.argmax(pred, 1),tf.float32)
b = tf.cast(tf.argmax(y,1),tf.float32)
auc = tf.contrib.metrics.streaming_auc(a, b)
并且在训练循环中:
train_acc = sess.run(accuracy, feed_dict={X: batch_xs, y: batch_ys, dropout_keep_prob:1.})
train_auc = sess.run(auc, feed_dict={X: batch_xs, y: batch_ys, dropout_keep_prob:1.})
这给了我以下输出(和错误)错误:
int64
int64
/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/array_ops.py:1197: VisibleDeprecationWarning: converting an array with ndim > 0 to an index will result in an error in the future
result_shape.insert(dim, 1)
Net built successfully...
Starting training...
Epoch: 000/300 cost: 0.618990561
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 715, in _do_call
return fn(*args)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 697, in _run_fn
status, run_metadata)
File "/usr/lib/python3.5/contextlib.py", line 66, in __exit__
next(self.gen)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/errors.py", line 450, in raise_exception_on_not_ok_status
pywrap_tensorflow.TF_GetCode(status))
tensorflow.python.framework.errors.FailedPreconditionError: Attempting to use uninitialized value auc/false_positives
[[Node: auc/false_positives/read = Identity[T=DT_FLOAT, _class=["loc:@auc/false_positives"], _device="/job:localhost/replica:0/task:0/cpu:0"](auc/false_positives)]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "./mlp_.py", line 152, in <module>
train_auc = sess.run(auc, feed_dict={X: batch_xs, y: batch_ys, dropout_keep_prob:1.})
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 372, in run
run_metadata_ptr)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 636, in _run
feed_dict_string, options, run_metadata)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 708, in _do_run
target_list, options, run_metadata)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 728, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors.FailedPreconditionError: Attempting to use uninitialized value auc/false_positives
[[Node: auc/false_positives/read = Identity[T=DT_FLOAT, _class=["loc:@auc/false_positives"], _device="/job:localhost/replica:0/task:0/cpu:0"](auc/false_positives)]]
Caused by op 'auc/false_positives/read', defined at:
File "./mlp_.py", line 121, in <module>
auc = tf.contrib.metrics.streaming_auc(a, b)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/contrib/metrics/python/ops/metric_ops.py", line 718, in streaming_auc
predictions, labels, thresholds, ignore_mask)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/contrib/metrics/python/ops/metric_ops.py", line 603, in _tp_fn_tn_fp
false_positives = _create_local('false_positives', shape=[num_thresholds])
File "/usr/local/lib/python3.5/dist-packages/tensorflow/contrib/metrics/python/ops/metric_ops.py", line 75, in _create_local
collections=collections)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/variables.py", line 211, in __init__
dtype=dtype)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/variables.py", line 319, in _init_from_args
self._snapshot = array_ops.identity(self._variable, name="read")
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/gen_array_ops.py", line 831, in identity
result = _op_def_lib.apply_op("Identity", input=input, name=name)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/op_def_library.py", line 704, in apply_op
op_def=op_def)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 2260, in create_op
original_op=self._default_original_op, op_def=op_def)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 1230, in __init__
self._traceback = _extract_stack()
我不明白我做错了什么以及为什么在仅使用准确性时代码运行良好但在使用 AUC 时会抛出此错误。
您能否向我提示正确的方向以了解如何解决此问题?
我的objective是计算AUC和ROC,以便更好地评估二元分类器的性能。
我在 github 上发现了同样的问题。目前,您似乎还需要 运行 sess.run(tf.initialize_local_variables())
才能使 tf.contrib.metrics.streaming_auc()
工作。他们正在努力。
这里有一个演示如何解决此问题的示例:
import tensorflow as tf
a = tf.Variable([0.1, 0.5])
b = tf.Variable([0.2, 0.6])
auc = tf.contrib.metrics.streaming_auc(a, b)
sess = tf.Session()
sess.run(tf.initialize_all_variables())
sess.run(tf.initialize_local_variables()) # try commenting this line and you'll get the error
train_auc = sess.run(auc)
print(train_auc)
我已经使用 Tensorflow 构建了一个二元分类器,现在我想使用 AUC 和准确性评估分类器。
就准确性而言,我可以很容易地这样做:
X = tf.placeholder('float', [None, n_input])
y = tf.placeholder('float', [None, n_classes])
pred = mlp(X, weights, biases, dropout_keep_prob)
correct_prediction = tf.equal(tf.argmax(pred, 1), tf.argmax(y, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))
计算 AUC 时我使用以下方法:
print(tf.argmax(pred, 1).dtype.name)
print(tf.argmax(pred, 1).dtype.name)
a = tf.cast(tf.argmax(pred, 1),tf.float32)
b = tf.cast(tf.argmax(y,1),tf.float32)
auc = tf.contrib.metrics.streaming_auc(a, b)
并且在训练循环中:
train_acc = sess.run(accuracy, feed_dict={X: batch_xs, y: batch_ys, dropout_keep_prob:1.})
train_auc = sess.run(auc, feed_dict={X: batch_xs, y: batch_ys, dropout_keep_prob:1.})
这给了我以下输出(和错误)错误:
int64
int64
/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/array_ops.py:1197: VisibleDeprecationWarning: converting an array with ndim > 0 to an index will result in an error in the future
result_shape.insert(dim, 1)
Net built successfully...
Starting training...
Epoch: 000/300 cost: 0.618990561
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 715, in _do_call
return fn(*args)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 697, in _run_fn
status, run_metadata)
File "/usr/lib/python3.5/contextlib.py", line 66, in __exit__
next(self.gen)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/errors.py", line 450, in raise_exception_on_not_ok_status
pywrap_tensorflow.TF_GetCode(status))
tensorflow.python.framework.errors.FailedPreconditionError: Attempting to use uninitialized value auc/false_positives
[[Node: auc/false_positives/read = Identity[T=DT_FLOAT, _class=["loc:@auc/false_positives"], _device="/job:localhost/replica:0/task:0/cpu:0"](auc/false_positives)]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "./mlp_.py", line 152, in <module>
train_auc = sess.run(auc, feed_dict={X: batch_xs, y: batch_ys, dropout_keep_prob:1.})
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 372, in run
run_metadata_ptr)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 636, in _run
feed_dict_string, options, run_metadata)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 708, in _do_run
target_list, options, run_metadata)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 728, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors.FailedPreconditionError: Attempting to use uninitialized value auc/false_positives
[[Node: auc/false_positives/read = Identity[T=DT_FLOAT, _class=["loc:@auc/false_positives"], _device="/job:localhost/replica:0/task:0/cpu:0"](auc/false_positives)]]
Caused by op 'auc/false_positives/read', defined at:
File "./mlp_.py", line 121, in <module>
auc = tf.contrib.metrics.streaming_auc(a, b)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/contrib/metrics/python/ops/metric_ops.py", line 718, in streaming_auc
predictions, labels, thresholds, ignore_mask)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/contrib/metrics/python/ops/metric_ops.py", line 603, in _tp_fn_tn_fp
false_positives = _create_local('false_positives', shape=[num_thresholds])
File "/usr/local/lib/python3.5/dist-packages/tensorflow/contrib/metrics/python/ops/metric_ops.py", line 75, in _create_local
collections=collections)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/variables.py", line 211, in __init__
dtype=dtype)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/variables.py", line 319, in _init_from_args
self._snapshot = array_ops.identity(self._variable, name="read")
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/gen_array_ops.py", line 831, in identity
result = _op_def_lib.apply_op("Identity", input=input, name=name)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/op_def_library.py", line 704, in apply_op
op_def=op_def)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 2260, in create_op
original_op=self._default_original_op, op_def=op_def)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 1230, in __init__
self._traceback = _extract_stack()
我不明白我做错了什么以及为什么在仅使用准确性时代码运行良好但在使用 AUC 时会抛出此错误。 您能否向我提示正确的方向以了解如何解决此问题?
我的objective是计算AUC和ROC,以便更好地评估二元分类器的性能。
我在 github 上发现了同样的问题。目前,您似乎还需要 运行 sess.run(tf.initialize_local_variables())
才能使 tf.contrib.metrics.streaming_auc()
工作。他们正在努力。
这里有一个演示如何解决此问题的示例:
import tensorflow as tf
a = tf.Variable([0.1, 0.5])
b = tf.Variable([0.2, 0.6])
auc = tf.contrib.metrics.streaming_auc(a, b)
sess = tf.Session()
sess.run(tf.initialize_all_variables())
sess.run(tf.initialize_local_variables()) # try commenting this line and you'll get the error
train_auc = sess.run(auc)
print(train_auc)