TensorBoard 显示了很多 'nodes' 来自以前的模型
TensorBoard showing lots of 'nodes' from previous models
我正在使用 MNIST 数据训练模型，我正在使用 tensorboard 可视化训练和验证损失。
这是我正在尝试的当前模型的代码：
model=tf.keras.models.Sequential()
#callback=tf.keras.callbacks.EarlyStopping(monitor='accuracy', min_delta=0, patience=0, verbose=0, mode='auto',restore_best_weights=False)
#model.add(tf.keras.layers.InputLayer(input_shape=[28,28]))
log_dir = "logs/fit/" + datetime.datetime.now().strftime("%Y%m%d-%H%M%S")
tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=log_dir, histogram_freq=1)
reduce_lr=tf.keras.callbacks.ReduceLROnPlateau(
monitor='val_loss', factor=0.1, patience=5, verbose=0, mode='auto',
min_delta=0.0001, cooldown=0, min_lr=0)
optimizer=tf.keras.optimizers.Adam(learning_rate=0.001, beta_1=0.9, beta_2=0.999, epsilon=1e-07, amsgrad=True,
name='Adam', clipnorm=5)
# if hparams[HP_OPTIMIZER]=='adam':
# optimizer=tf.keras.optimizers.Adam(learning_rate=0.001, beta_1=0.9, beta_2=0.999, epsilon=1e-07, amsgrad=True,
# name='Adam', clipnorm=5)
# elif hparams[HP_OPTIMIZER]=='sgd':
# tf.keras.optimizers.SGD(learning_rate=0.01, momentum=0.9, nesterov=False, name='SGD', **kwargs)
model.add(tf.keras.layers.Flatten(input_shape=[28,28]))
l2_new=tf.keras.regularizers.L2(
l2=0.05)
model.add(tf.keras.layers.BatchNormalization(
axis=-1, momentum=0.99, epsilon=0.001, center=True, scale=True,
beta_initializer='zeros', gamma_initializer='ones',
moving_mean_initializer='zeros', moving_variance_initializer='ones',
beta_regularizer=None, gamma_regularizer=None, beta_constraint=None,
gamma_constraint=None, renorm=True, renorm_clipping=None, renorm_momentum=0.99))
model.add(tf.keras.layers.Dense(300,activation='relu',kernel_initializer="he_normal",
kernel_regularizer=l2_new, bias_regularizer=l2_new))
model.add(tf.keras.layers.BatchNormalization(
axis=-1, momentum=0.99, epsilon=0.001, center=True, scale=True,
beta_initializer='zeros', gamma_initializer='ones',
moving_mean_initializer='zeros', moving_variance_initializer='ones',
beta_regularizer=None, gamma_regularizer=None, beta_constraint=None,
gamma_constraint=None, renorm=True, renorm_clipping=None, renorm_momentum=0.99))
model.add(tf.keras.layers.Dense(300,activation='relu',kernel_initializer="he_normal",
kernel_regularizer=l2_new, bias_regularizer=l2_new))
model.add(tf.keras.layers.BatchNormalization(
axis=-1, momentum=0.99, epsilon=0.001, center=True, scale=True,
beta_initializer='zeros', gamma_initializer='ones',
moving_mean_initializer='zeros', moving_variance_initializer='ones',
beta_regularizer=None, gamma_regularizer=None, beta_constraint=None,
gamma_constraint=None, renorm=True, renorm_clipping=None, renorm_momentum=0.99))
model.add(tf.keras.layers.Dense(10,activation='softmax'))
抱歉，如果有点乱。我实际上是在用
创建一个顺序模型
- Flatten 输入层
- 一个Batch Norm层
3.A 300 个神经元密集层
- Batch Norm 层
- 300 个神经元的密集层
- Batch Norm 层
- 具有 10 个神经元的 Softmax 输出层。
我的模型还使用了 'Adam' 优化器和学习率衰减。
当我在张量板的图表子标题下查看我的模型时，我得到以下图片：
如您所见，有很多 'nodes'，我猜这是因为我训练了多个模型。我如何摆脱所有以前的尝试。
我试过使用 del model 和 tf.keras.backend.clear_session() 但它们没有用。
编辑: 我听从了'Aniket Bote'的建议，删除了日志。这是新的输出：
我仍然不确定它是否正确。从我的代码来看，我不认为我的图应该有 2 个分支，而且我仍然在右边得到大量的批量标准化 'nodes'。
第二个分支本身不是一个图，而是一个子图。
Tensorflow 构建它执行的操作的图表，以加速代码的执行。如果你点击那些你可以看到它们是批归一化层使用的函数，而不是层本身。您可以在主图上看到所有图层信息。
如果您不想要这些节点，您可以通过将 BatchNormalization 的可训练属性设置为 False 来摆脱它们。
在这种情况下，层的权重不会改变，TensorFlow 将不再需要为该层计算任何东西，即。不会生成函数节点。
代码：
import tensorflow as tf
import numpy as np
np.random.seed(100)
x = tf.constant(np.random.randint(50, size =(1000,28,28)), dtype = tf.float32)
y = tf.constant(np.random.randint(10, size =(1000,)), dtype = tf.int32)
model=tf.keras.models.Sequential()
log_dir = "logs"
tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=log_dir, histogram_freq=1, profile_batch = 0)
optimizer=tf.keras.optimizers.Adam(learning_rate=0.001, beta_1=0.9, beta_2=0.999, epsilon=1e-07, amsgrad=True,
name='Adam', clipnorm=5)
model.add(tf.keras.layers.Flatten(input_shape=[28,28]))
l2_new=tf.keras.regularizers.L2(
l2=0.05)
model.add(tf.keras.layers.BatchNormalization(
axis=-1, momentum=0.99, epsilon=0.001, center=True, scale=True,
beta_initializer='zeros', gamma_initializer='ones',
moving_mean_initializer='zeros', moving_variance_initializer='ones',
beta_regularizer=None, gamma_regularizer=None, beta_constraint=None,
gamma_constraint=None, renorm=True, renorm_clipping=None, renorm_momentum=0.99,trainable = False))
model.add(tf.keras.layers.Dense(300,activation='relu',kernel_initializer="he_normal",
kernel_regularizer=l2_new, bias_regularizer=l2_new))
model.add(tf.keras.layers.BatchNormalization(
axis=-1, momentum=0.99, epsilon=0.001, center=True, scale=True,
beta_initializer='zeros', gamma_initializer='ones',
moving_mean_initializer='zeros', moving_variance_initializer='ones',
beta_regularizer=None, gamma_regularizer=None, beta_constraint=None,
gamma_constraint=None, renorm=True, renorm_clipping=None, renorm_momentum=0.99,trainable = False))
model.add(tf.keras.layers.Dense(300,activation='relu',kernel_initializer="he_normal",
kernel_regularizer=l2_new, bias_regularizer=l2_new))
model.add(tf.keras.layers.BatchNormalization(
axis=-1, momentum=0.99, epsilon=0.001, center=True, scale=True,
beta_initializer='zeros', gamma_initializer='ones',
moving_mean_initializer='zeros', moving_variance_initializer='ones',
beta_regularizer=None, gamma_regularizer=None, beta_constraint=None,
gamma_constraint=None, renorm=True, renorm_clipping=None, renorm_momentum=0.99,trainable = False))
model.add(tf.keras.layers.Dense(10,activation='softmax'))
model.compile(optimizer='adam',
loss='sparse_categorical_crossentropy',
metrics=['accuracy'])
tf.keras.utils.plot_model(model, 'my_first_model.png', show_shapes=True)
model.fit(x,y, epochs = 10, callbacks = tensorboard_callback)
输出：
Epoch 1/10
32/32 [==============================] - 0s 10ms/step - loss: 89.0275 - accuracy: 0.1100
Epoch 2/10
32/32 [==============================] - 0s 9ms/step - loss: 56.7906 - accuracy: 0.1310
Epoch 3/10
32/32 [==============================] - 0s 9ms/step - loss: 48.5681 - accuracy: 0.1490
Epoch 4/10
32/32 [==============================] - 0s 9ms/step - loss: 42.8176 - accuracy: 0.1850
Epoch 5/10
32/32 [==============================] - 0s 9ms/step - loss: 38.5857 - accuracy: 0.2110
Epoch 6/10
32/32 [==============================] - 0s 9ms/step - loss: 35.1675 - accuracy: 0.2540
Epoch 7/10
32/32 [==============================] - 0s 9ms/step - loss: 32.3327 - accuracy: 0.2750
Epoch 8/10
32/32 [==============================] - 0s 9ms/step - loss: 29.8839 - accuracy: 0.3420
Epoch 9/10
32/32 [==============================] - 0s 9ms/step - loss: 27.7426 - accuracy: 0.3940
Epoch 10/10
32/32 [==============================] - 0s 10ms/step - loss: 25.6565 - accuracy: 0.4930
Tensorboard 图图像：
我正在使用 MNIST 数据训练模型，我正在使用 tensorboard 可视化训练和验证损失。
这是我正在尝试的当前模型的代码：
model=tf.keras.models.Sequential()
#callback=tf.keras.callbacks.EarlyStopping(monitor='accuracy', min_delta=0, patience=0, verbose=0, mode='auto',restore_best_weights=False)
#model.add(tf.keras.layers.InputLayer(input_shape=[28,28]))
log_dir = "logs/fit/" + datetime.datetime.now().strftime("%Y%m%d-%H%M%S")
tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=log_dir, histogram_freq=1)
reduce_lr=tf.keras.callbacks.ReduceLROnPlateau(
monitor='val_loss', factor=0.1, patience=5, verbose=0, mode='auto',
min_delta=0.0001, cooldown=0, min_lr=0)
optimizer=tf.keras.optimizers.Adam(learning_rate=0.001, beta_1=0.9, beta_2=0.999, epsilon=1e-07, amsgrad=True,
name='Adam', clipnorm=5)
# if hparams[HP_OPTIMIZER]=='adam':
# optimizer=tf.keras.optimizers.Adam(learning_rate=0.001, beta_1=0.9, beta_2=0.999, epsilon=1e-07, amsgrad=True,
# name='Adam', clipnorm=5)
# elif hparams[HP_OPTIMIZER]=='sgd':
# tf.keras.optimizers.SGD(learning_rate=0.01, momentum=0.9, nesterov=False, name='SGD', **kwargs)
model.add(tf.keras.layers.Flatten(input_shape=[28,28]))
l2_new=tf.keras.regularizers.L2(
l2=0.05)
model.add(tf.keras.layers.BatchNormalization(
axis=-1, momentum=0.99, epsilon=0.001, center=True, scale=True,
beta_initializer='zeros', gamma_initializer='ones',
moving_mean_initializer='zeros', moving_variance_initializer='ones',
beta_regularizer=None, gamma_regularizer=None, beta_constraint=None,
gamma_constraint=None, renorm=True, renorm_clipping=None, renorm_momentum=0.99))
model.add(tf.keras.layers.Dense(300,activation='relu',kernel_initializer="he_normal",
kernel_regularizer=l2_new, bias_regularizer=l2_new))
model.add(tf.keras.layers.BatchNormalization(
axis=-1, momentum=0.99, epsilon=0.001, center=True, scale=True,
beta_initializer='zeros', gamma_initializer='ones',
moving_mean_initializer='zeros', moving_variance_initializer='ones',
beta_regularizer=None, gamma_regularizer=None, beta_constraint=None,
gamma_constraint=None, renorm=True, renorm_clipping=None, renorm_momentum=0.99))
model.add(tf.keras.layers.Dense(300,activation='relu',kernel_initializer="he_normal",
kernel_regularizer=l2_new, bias_regularizer=l2_new))
model.add(tf.keras.layers.BatchNormalization(
axis=-1, momentum=0.99, epsilon=0.001, center=True, scale=True,
beta_initializer='zeros', gamma_initializer='ones',
moving_mean_initializer='zeros', moving_variance_initializer='ones',
beta_regularizer=None, gamma_regularizer=None, beta_constraint=None,
gamma_constraint=None, renorm=True, renorm_clipping=None, renorm_momentum=0.99))
model.add(tf.keras.layers.Dense(10,activation='softmax'))
抱歉，如果有点乱。我实际上是在用
创建一个顺序模型- Flatten 输入层
- 一个Batch Norm层 3.A 300 个神经元密集层
- Batch Norm 层
- 300 个神经元的密集层
- Batch Norm 层
- 具有 10 个神经元的 Softmax 输出层。
我的模型还使用了 'Adam' 优化器和学习率衰减。
当我在张量板的图表子标题下查看我的模型时，我得到以下图片：
如您所见，有很多 'nodes'，我猜这是因为我训练了多个模型。我如何摆脱所有以前的尝试。
我试过使用 del model 和 tf.keras.backend.clear_session() 但它们没有用。
编辑: 我听从了'Aniket Bote'的建议，删除了日志。这是新的输出：
我仍然不确定它是否正确。从我的代码来看，我不认为我的图应该有 2 个分支，而且我仍然在右边得到大量的批量标准化 'nodes'。
第二个分支本身不是一个图，而是一个子图。
Tensorflow 构建它执行的操作的图表，以加速代码的执行。如果你点击那些你可以看到它们是批归一化层使用的函数，而不是层本身。您可以在主图上看到所有图层信息。
如果您不想要这些节点，您可以通过将 BatchNormalization 的可训练属性设置为 False 来摆脱它们。
在这种情况下，层的权重不会改变，TensorFlow 将不再需要为该层计算任何东西，即。不会生成函数节点。
代码：
import tensorflow as tf
import numpy as np
np.random.seed(100)
x = tf.constant(np.random.randint(50, size =(1000,28,28)), dtype = tf.float32)
y = tf.constant(np.random.randint(10, size =(1000,)), dtype = tf.int32)
model=tf.keras.models.Sequential()
log_dir = "logs"
tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=log_dir, histogram_freq=1, profile_batch = 0)
optimizer=tf.keras.optimizers.Adam(learning_rate=0.001, beta_1=0.9, beta_2=0.999, epsilon=1e-07, amsgrad=True,
name='Adam', clipnorm=5)
model.add(tf.keras.layers.Flatten(input_shape=[28,28]))
l2_new=tf.keras.regularizers.L2(
l2=0.05)
model.add(tf.keras.layers.BatchNormalization(
axis=-1, momentum=0.99, epsilon=0.001, center=True, scale=True,
beta_initializer='zeros', gamma_initializer='ones',
moving_mean_initializer='zeros', moving_variance_initializer='ones',
beta_regularizer=None, gamma_regularizer=None, beta_constraint=None,
gamma_constraint=None, renorm=True, renorm_clipping=None, renorm_momentum=0.99,trainable = False))
model.add(tf.keras.layers.Dense(300,activation='relu',kernel_initializer="he_normal",
kernel_regularizer=l2_new, bias_regularizer=l2_new))
model.add(tf.keras.layers.BatchNormalization(
axis=-1, momentum=0.99, epsilon=0.001, center=True, scale=True,
beta_initializer='zeros', gamma_initializer='ones',
moving_mean_initializer='zeros', moving_variance_initializer='ones',
beta_regularizer=None, gamma_regularizer=None, beta_constraint=None,
gamma_constraint=None, renorm=True, renorm_clipping=None, renorm_momentum=0.99,trainable = False))
model.add(tf.keras.layers.Dense(300,activation='relu',kernel_initializer="he_normal",
kernel_regularizer=l2_new, bias_regularizer=l2_new))
model.add(tf.keras.layers.BatchNormalization(
axis=-1, momentum=0.99, epsilon=0.001, center=True, scale=True,
beta_initializer='zeros', gamma_initializer='ones',
moving_mean_initializer='zeros', moving_variance_initializer='ones',
beta_regularizer=None, gamma_regularizer=None, beta_constraint=None,
gamma_constraint=None, renorm=True, renorm_clipping=None, renorm_momentum=0.99,trainable = False))
model.add(tf.keras.layers.Dense(10,activation='softmax'))
model.compile(optimizer='adam',
loss='sparse_categorical_crossentropy',
metrics=['accuracy'])
tf.keras.utils.plot_model(model, 'my_first_model.png', show_shapes=True)
model.fit(x,y, epochs = 10, callbacks = tensorboard_callback)
输出：
Epoch 1/10
32/32 [==============================] - 0s 10ms/step - loss: 89.0275 - accuracy: 0.1100
Epoch 2/10
32/32 [==============================] - 0s 9ms/step - loss: 56.7906 - accuracy: 0.1310
Epoch 3/10
32/32 [==============================] - 0s 9ms/step - loss: 48.5681 - accuracy: 0.1490
Epoch 4/10
32/32 [==============================] - 0s 9ms/step - loss: 42.8176 - accuracy: 0.1850
Epoch 5/10
32/32 [==============================] - 0s 9ms/step - loss: 38.5857 - accuracy: 0.2110
Epoch 6/10
32/32 [==============================] - 0s 9ms/step - loss: 35.1675 - accuracy: 0.2540
Epoch 7/10
32/32 [==============================] - 0s 9ms/step - loss: 32.3327 - accuracy: 0.2750
Epoch 8/10
32/32 [==============================] - 0s 9ms/step - loss: 29.8839 - accuracy: 0.3420
Epoch 9/10
32/32 [==============================] - 0s 9ms/step - loss: 27.7426 - accuracy: 0.3940
Epoch 10/10
32/32 [==============================] - 0s 10ms/step - loss: 25.6565 - accuracy: 0.4930
Tensorboard 图图像：