如何使用 tf.MonitoredTrainingSession 在训练和验证数据集之间切换?
How to switch between training and validation dataset with tf.MonitoredTrainingSession?
我想在 tensorflow 数据集 API 中使用 feedable
迭代器设计,因此我可以在一些训练步骤后切换到验证数据。但是如果我切换到验证数据,它将结束整个会话。
下面的代码演示了我想要做什么:
import tensorflow as tf
graph = tf.Graph()
with graph.as_default():
training_ds = tf.data.Dataset.range(32).batch(4)
validation_ds = tf.data.Dataset.range(8).batch(4)
handle = tf.placeholder(tf.string, shape=[])
iterator = tf.data.Iterator.from_string_handle(
handle, training_ds.output_types, training_ds.output_shapes)
next_element = iterator.get_next()
training_iterator = training_ds.make_initializable_iterator()
validation_iterator = validation_ds.make_initializable_iterator()
with graph.as_default():
with tf.train.MonitoredTrainingSession() as sess:
training_handle = sess.run(training_iterator.string_handle())
validation_handle = sess.run(validation_iterator.string_handle())
sess.run(training_iterator.initializer)
count_training = 0
while not sess.should_stop():
x = sess.run(next_element, feed_dict={handle: training_handle})
count_training += 1
print('{} [training] {}'.format(count_training, x.shape))
# print(x)
# we do periodic validation
if count_training % 4 == 0:
sess.run(validation_iterator.initializer)
count_validation = 0
while not sess.should_stop():
y = sess.run(next_element, feed_dict={handle: validation_handle})
count_validation += 1
print(' {} [validation] {}'.format(count_validation, y.shape))
# print(y)
训练数据有32个元素,4个batch,所以得到8个batch
我们每 4 个步骤进行一次验证,所以我预计:
# 1 [training]
# 2 [training]
# 3 [training]
# 4 [training]
# 1 [validation]
# 2 [validation]
# 5 [training]
# 6 [training]
# 7 [training]
# 8 [training]
# 1 [validation]
# 2 [validation]
但在第一次验证完成后停止:
# 1 [training]
# 2 [training]
# 3 [training]
# 4 [training]
# 1 [validation]
# 2 [validation]
那么,如何在tf.MonitoredTrainingSession
中使用这个feedable
迭代器呢?
我建议在验证数据集末尾捕获 tf.errors.OutOfRangeError
(您也可以在官方 API 中检查 the processing multiple epochs section 以使用 repeat
数据集):
while not sess.should_stop():
x = sess.run(next_element, feed_dict={handle: training_handle})
count_training += 1
print('{} [training] {}'.format(count_training, x.shape))
# we do periodic validation
if count_training % 4 == 0:
sess.run(validation_iterator.initializer)
count_validation = 0
while True:
try:
y = sess.run(next_element, feed_dict={handle: validation_handle})
count_validation += 1
print(' {} [validation] {}'.format(count_validation, y.shape))
except tf.errors.OutOfRangeError:
break
这段代码打印:
1 [training] (4,)
2 [training] (4,)
3 [training] (4,)
4 [training] (4,)
1 [validation] (4,)
2 [validation] (4,)
5 [training] (4,)
6 [training] (4,)
7 [training] (4,)
8 [training] (4,)
1 [validation] (4,)
2 [validation] (4,)
我想在 tensorflow 数据集 API 中使用 feedable
迭代器设计,因此我可以在一些训练步骤后切换到验证数据。但是如果我切换到验证数据,它将结束整个会话。
下面的代码演示了我想要做什么:
import tensorflow as tf
graph = tf.Graph()
with graph.as_default():
training_ds = tf.data.Dataset.range(32).batch(4)
validation_ds = tf.data.Dataset.range(8).batch(4)
handle = tf.placeholder(tf.string, shape=[])
iterator = tf.data.Iterator.from_string_handle(
handle, training_ds.output_types, training_ds.output_shapes)
next_element = iterator.get_next()
training_iterator = training_ds.make_initializable_iterator()
validation_iterator = validation_ds.make_initializable_iterator()
with graph.as_default():
with tf.train.MonitoredTrainingSession() as sess:
training_handle = sess.run(training_iterator.string_handle())
validation_handle = sess.run(validation_iterator.string_handle())
sess.run(training_iterator.initializer)
count_training = 0
while not sess.should_stop():
x = sess.run(next_element, feed_dict={handle: training_handle})
count_training += 1
print('{} [training] {}'.format(count_training, x.shape))
# print(x)
# we do periodic validation
if count_training % 4 == 0:
sess.run(validation_iterator.initializer)
count_validation = 0
while not sess.should_stop():
y = sess.run(next_element, feed_dict={handle: validation_handle})
count_validation += 1
print(' {} [validation] {}'.format(count_validation, y.shape))
# print(y)
训练数据有32个元素,4个batch,所以得到8个batch 我们每 4 个步骤进行一次验证,所以我预计:
# 1 [training]
# 2 [training]
# 3 [training]
# 4 [training]
# 1 [validation]
# 2 [validation]
# 5 [training]
# 6 [training]
# 7 [training]
# 8 [training]
# 1 [validation]
# 2 [validation]
但在第一次验证完成后停止:
# 1 [training]
# 2 [training]
# 3 [training]
# 4 [training]
# 1 [validation]
# 2 [validation]
那么,如何在tf.MonitoredTrainingSession
中使用这个feedable
迭代器呢?
我建议在验证数据集末尾捕获 tf.errors.OutOfRangeError
(您也可以在官方 API 中检查 the processing multiple epochs section 以使用 repeat
数据集):
while not sess.should_stop():
x = sess.run(next_element, feed_dict={handle: training_handle})
count_training += 1
print('{} [training] {}'.format(count_training, x.shape))
# we do periodic validation
if count_training % 4 == 0:
sess.run(validation_iterator.initializer)
count_validation = 0
while True:
try:
y = sess.run(next_element, feed_dict={handle: validation_handle})
count_validation += 1
print(' {} [validation] {}'.format(count_validation, y.shape))
except tf.errors.OutOfRangeError:
break
这段代码打印:
1 [training] (4,)
2 [training] (4,)
3 [training] (4,)
4 [training] (4,)
1 [validation] (4,)
2 [validation] (4,)
5 [training] (4,)
6 [training] (4,)
7 [training] (4,)
8 [training] (4,)
1 [validation] (4,)
2 [validation] (4,)