Tensorflow CNN - 密集层作为 Softmax 层输入

Question

我正在尝试复制中描述的 CNN https://pdfs.semanticscholar.org/3b57/85ca3c29c963ae396c2f94ba1a805c787cc8.pdf

我卡在了最后一层。我已经像这样模拟了 cnn

# Model function for CNN
def cnn_model_fn(features, labels, mode):

  # Input Layer
  # Reshape X to 4-D tensor: [batch_size, width, height, channels]
  # Taxes images are 150x150 pixels, and have one color channel
  input_layer = tf.reshape(features, [-1, 150, 150, 1])

  # Convolutional Layer #1
  # Input Tensor Shape: [batch_size, 150, 150, 1]
  # Output Tensor Shape: [batch_size, 144, 144, 20]
  conv1 = tf.layers.conv2d(
      inputs=input_layer,
      filters=20,
      kernel_size=[7, 7],
      padding="valid",
      activation=tf.nn.relu)

  # Pooling Layer #1
  # Input Tensor Shape: [batch_size, 144, 144, 20]
  # Output Tensor Shape: [batch_size, 36, 36, 20]
  pool1 = tf.layers.max_pooling2d(inputs=conv1, pool_size=[4, 4], strides=4)

  # Convolutional Layer #2
  # Input Tensor Shape: [batch_size, 36, 36, 20]
  # Output Tensor Shape: [batch_size, 32, 32, 50]
  conv2 = tf.layers.conv2d(
      inputs=pool1,
      filters=50,
      kernel_size=[5, 5],
      padding="valid",
      activation=tf.nn.relu)

  # Pooling Layer #2
  # Input Tensor Shape: [batch_size, 32, 32, 50]
  # Output Tensor Shape: [batch_size, 8, 8, 50]
  pool2 = tf.layers.max_pooling2d(inputs=conv2, pool_size=[4, 4], strides=4)

  # Flatten tensor into a batch of vectors
  # Input Tensor Shape: [batch_size, 8, 8, 50]
  # Output Tensor Shape: [batch_size, 8 * 8 * 50]
  pool2_flat = tf.reshape(pool2, [-1, 8 * 8 * 50])

  # Dense Layer #1
  # Densely connected layer with 1000 neurons
  # Input Tensor Shape: [batch_size, 8 * 8 * 50]
  # Output Tensor Shape: [batch_size, 1000]
  dense1 = tf.layers.dense(inputs=pool2_flat, units=1000, activation=tf.nn.relu)

  # Dense Layer #2
  # Densely connected layer with 1000 neurons
  # Input Tensor Shape: [batch_size, 1000]
  # Output Tensor Shape: [batch_size, 1000]
  dense2 = tf.layers.dense(inputs=dense1, units=1000, activation=tf.nn.relu)

  # Add dropout operation; 0.5 probability that element will be kept
  dropout = tf.layers.dropout(
      inputs=dense2, rate=0.5, training=mode == learn.ModeKeys.TRAIN)

  # Logits layer
  # Input Tensor Shape: [batch_size, 1000]
  # Output Tensor Shape: [batch_size, 4]
  logits = tf.layers.dense(inputs=dropout, units=nClass)

  loss = None
  train_op = None

  # Calculate Loss (for both TRAIN and EVAL modes)
  if mode != learn.ModeKeys.INFER:
    onehot_labels = tf.one_hot(indices=tf.cast(labels, tf.int32), depth=nClass)
    loss = tf.losses.softmax_cross_entropy(
        onehot_labels=onehot_labels, logits=logits)

  # Configure the Training Op (for TRAIN mode)
  if mode == learn.ModeKeys.TRAIN:
    train_op = tf.contrib.layers.optimize_loss(
        loss=loss,
        global_step=tf.contrib.framework.get_global_step(),
        learning_rate=0.001,
        optimizer="SGD")

  # Generate Predictions
  predictions = {
      "classes": tf.argmax(
          input=logits, axis=1)
  }

  # Return a ModelFnOps object
  return model_fn_lib.ModelFnOps(
      mode=mode, predictions=predictions, loss=loss, train_op=train_op)

但最终准确率真的很差（0.25）。所以我意识到实际上这篇论文指出最后一层是 softmax 层。所以我尝试将我的 logits 层更改为

logits = tf.layers.softmax(dropout)

但是当我运行它时，它说

ValueError: Shapes (?, 1000) and (?, 4) are incompatible

那么，我在这里缺少什么？

Answer 1

原文正确。在使用 tf.losses.softmax_cross_entropy 计算损失时应用 softmax 激活。如果你想单独计算它，你应该在 logits 计算之后添加它，但不要像你那样替换它。

logits = tf.layers.dense(inputs=dropout, units=nClass)
softmax = tf.layers.softmax(logits)

或者您可以将两者合二为一，但我不推荐这样做。最好用loss计算softmax

logits = tf.layers.dense(inputs=dropout, units=nClass, activation=tf.layers.softmax)

你的分类器并不比随机分类器做得好，所以我想说问题出在其他地方，可能在数据加载和预处理中。

Tensorflow CNN - 密集层作为 Softmax 层输入

Tensorflow CNN - Dense layer as Softmax layer input

python

convolution

tensorflow

softmax