在 Keras 中使用 tf.metrics?
Use tf.metrics in Keras?
我对specificity_at_sensitivity
. Looking through the Keras docs特别感兴趣:
from keras import metrics
model.compile(loss='mean_squared_error',
optimizer='sgd',
metrics=[metrics.mae, metrics.categorical_accuracy])
但看起来 metrics
列表必须具有 arity 2 的函数,接受 (y_true, y_pred)
并返回单个张量值。
编辑:目前我是这样做的:
from sklearn.metrics import confusion_matrix
predictions = model.predict(x_test)
y_test = np.argmax(y_test, axis=-1)
predictions = np.argmax(predictions, axis=-1)
c = confusion_matrix(y_test, predictions)
print('Confusion matrix:\n', c)
print('sensitivity', c[0, 0] / (c[0, 1] + c[0, 0]))
print('specificity', c[1, 1] / (c[1, 1] + c[1, 0]))
这种方法的缺点是我只在训练完成后得到我关心的输出。更愿意每 10 个时期左右获取一次指标。
我认为对于只有两个传入参数没有严格的限制,在metrics.py函数中只有三个传入参数,但是k选择了默认值5。
def sparse_top_k_categorical_accuracy(y_true, y_pred, k=5):
return K.mean(K.in_top_k(y_pred, K.cast(K.max(y_true, axis=-1), 'int32'), k), axis=-1)
我在 github, and it seems that tf.metrics
are still not supported by Keras models. However, in case you are very interested in using tf.metrics.specificity_at_sensitivity, I would suggest the following workaround (inspired by BogdanRuzh's 解决方案中发现了相关问题):
def specificity_at_sensitivity(sensitivity, **kwargs):
def metric(labels, predictions):
# any tensorflow metric
value, update_op = tf.metrics.specificity_at_sensitivity(labels, predictions, sensitivity, **kwargs)
# find all variables created for this metric
metric_vars = [i for i in tf.local_variables() if 'specificity_at_sensitivity' in i.name.split('/')[2]]
# Add metric variables to GLOBAL_VARIABLES collection.
# They will be initialized for new session.
for v in metric_vars:
tf.add_to_collection(tf.GraphKeys.GLOBAL_VARIABLES, v)
# force to update metric values
with tf.control_dependencies([update_op]):
value = tf.identity(value)
return value
return metric
model.compile(loss='mean_squared_error',
optimizer='sgd',
metrics=[metrics.mae,
metrics.categorical_accuracy,
specificity_at_sensitivity(0.5)])
更新:
您可以使用 model.evaluate 在训练后检索指标。
我对specificity_at_sensitivity
. Looking through the Keras docs特别感兴趣:
from keras import metrics
model.compile(loss='mean_squared_error',
optimizer='sgd',
metrics=[metrics.mae, metrics.categorical_accuracy])
但看起来 metrics
列表必须具有 arity 2 的函数,接受 (y_true, y_pred)
并返回单个张量值。
编辑:目前我是这样做的:
from sklearn.metrics import confusion_matrix
predictions = model.predict(x_test)
y_test = np.argmax(y_test, axis=-1)
predictions = np.argmax(predictions, axis=-1)
c = confusion_matrix(y_test, predictions)
print('Confusion matrix:\n', c)
print('sensitivity', c[0, 0] / (c[0, 1] + c[0, 0]))
print('specificity', c[1, 1] / (c[1, 1] + c[1, 0]))
这种方法的缺点是我只在训练完成后得到我关心的输出。更愿意每 10 个时期左右获取一次指标。
我认为对于只有两个传入参数没有严格的限制,在metrics.py函数中只有三个传入参数,但是k选择了默认值5。
def sparse_top_k_categorical_accuracy(y_true, y_pred, k=5):
return K.mean(K.in_top_k(y_pred, K.cast(K.max(y_true, axis=-1), 'int32'), k), axis=-1)
我在 github, and it seems that tf.metrics
are still not supported by Keras models. However, in case you are very interested in using tf.metrics.specificity_at_sensitivity, I would suggest the following workaround (inspired by BogdanRuzh's 解决方案中发现了相关问题):
def specificity_at_sensitivity(sensitivity, **kwargs):
def metric(labels, predictions):
# any tensorflow metric
value, update_op = tf.metrics.specificity_at_sensitivity(labels, predictions, sensitivity, **kwargs)
# find all variables created for this metric
metric_vars = [i for i in tf.local_variables() if 'specificity_at_sensitivity' in i.name.split('/')[2]]
# Add metric variables to GLOBAL_VARIABLES collection.
# They will be initialized for new session.
for v in metric_vars:
tf.add_to_collection(tf.GraphKeys.GLOBAL_VARIABLES, v)
# force to update metric values
with tf.control_dependencies([update_op]):
value = tf.identity(value)
return value
return metric
model.compile(loss='mean_squared_error',
optimizer='sgd',
metrics=[metrics.mae,
metrics.categorical_accuracy,
specificity_at_sensitivity(0.5)])
更新:
您可以使用 model.evaluate 在训练后检索指标。