Tensorflow LSTM 逐像素分类

Question

如何对 LSTM 网络进行逐像素分类？具体来说，在 Tensorflow 中。

我的直觉告诉我输出张量（来自代码的pred &y）应该是一个与输入图像具有相同分辨率的二维张量。换句话说，输入图像为 200x200，输出分类为 200x200。

Udacity 课程包括一个示例 LSTM 网络，其中输入图像为 28x28。然而它是一个图像（作为一个整体——手写MNIST数据集）分类网络。

我的想法是，我可以用 [n_input][n_steps] 替换所有尺寸为 [n_classes] 的张量（代码如下）。但是它在矩阵乘法时抛出错误。

Udacity 示例代码部分如下所示：

n_input = 28 # MNIST data input (img shape: 28*28)
n_steps = 28 # timesteps
n_hidden = 128 # hidden layer num of features
n_classes = 10 # MNIST total classes (0-9 digits)

# tf Graph input
x = tf.placeholder("float", [None, n_steps, n_input])
y = tf.placeholder("float", [None, n_classes])

# Define weights
weights = {
    'hidden': tf.Variable(tf.random_normal([n_input, n_hidden])),
    'out': tf.Variable(tf.random_normal([n_hidden, n_classes]))
}
biases = {
    'hidden': tf.Variable(tf.random_normal([n_hidden])),
    'out': tf.Variable(tf.random_normal([n_classes]))
}


def RNN(x, weights, biases):

    # Prepare data shape to match `rnn` function requirements
    # Current data input shape: (batch_size, n_steps, n_input)
    # Permuting batch_size and n_steps
    x = tf.transpose(x, [1, 0, 2])
    # Reshaping to (n_steps*batch_size, n_input)
    x = tf.reshape(x, [-1, n_input])
    # Split to get a list of 'n_steps' tensors of shape (batch_size, n_hidden)
    # This input shape is required by `rnn` function
    x = tf.split(0, n_steps, x)

    # Define a lstm cell with tensorflow
    lstm_cell = rnn_cell.BasicLSTMCell(n_hidden, forget_bias=1.0)
    pdb.set_trace()
    # Get lstm cell output
    outputs, states = rnn.rnn(lstm_cell, x, dtype=tf.float32)

    # Linear activation, using rnn inner loop last output
    return tf.matmul(outputs[-1], weights['out']) + biases['out']

pred = RNN(x, weights, biases)

-------------------------------------------- ------------------------------

然后我的代码如下所示：

n_input = 200 # data data input (img shape: 28*28)
n_steps = 200 # timesteps
n_hidden = 128 # hidden layer num of features
n_classes = 2 # data total classes (0-9 digits)

# tf Graph input
x = tf.placeholder("float", [None, n_input, n_steps])
y = tf.placeholder("float", [None, n_input, n_steps])


# Define weights
weights = {
    'hidden': tf.Variable(tf.random_normal([n_input, n_hidden]), dtype="float32"),
    'out': tf.Variable(tf.random_normal([n_hidden, n_input, n_steps]), dtype="float32")
}
biases = {
    'hidden': tf.Variable(tf.random_normal([n_hidden]), dtype="float32"),
    'out': tf.Variable(tf.random_normal([n_input, n_steps]), dtype="float32")
}


def RNN(x, weights, biases):

    # Prepare data shape to match `rnn` function requirements
    # Current data input shape: (batch_size, n_steps, n_input)
    # Permuting batch_size and n_steps
    pdb.set_trace()
    x = tf.transpose(x, [1, 0, 2])
    # Reshaping to (n_steps*batch_size, n_input)
    x = tf.reshape(x, [-1, n_input])
    # Split to get a list of 'n_steps' tensors of shape (batch_size, n_hidden)
    # This input shape is required by `rnn` function
    x = tf.split(0, n_steps, x)

    # Define a lstm cell with tensorflow
    lstm_cell = rnn_cell.BasicLSTMCell(n_hidden, forget_bias=1.0)
    pdb.set_trace()

    # Get lstm cell output
    outputs, states = rnn.rnn(lstm_cell, x, dtype=tf.float32)

    # Linear activation, using rnn inner loop last output
    # return tf.matmul(outputs[-1], weights['out']) + biases['out']
    return tf.batch_matmul(outputs[-1], weights['out']) + biases['out']

pred = RNN(x, weights, biases)

第return tf.batch_matmul(outputs[-1], weights['out']) + biases['out']行就是问题所在。因为 outputs 是 2D 张量的向量而 weights['out'] 是 3D 张量的向量。

我想也许我可以更改 outputs 的维度，但这需要深入研究 RNN 对象（在 API 中）。

我在这里有哪些选择？我可以做一些重塑吗？如果是这样，我应该重塑什么，以什么方式重塑？

Answer 1

您不能对 3 维形状 [n_hidden, n_input, n_step] 的矩阵进行矩阵乘法。
您可以做的是输出一个维度为 [batch_size, n_input * n_step] 的向量，然后将其重新整形为 [batch_size, n_input, n_step].

weights = {
    'hidden': ... ,
    'out': tf.Variable(tf.random_normal([n_hidden, n_input * n_steps]), dtype="float32")
}
biases = {
    'hidden': ... ,
    'out': tf.Variable(tf.random_normal([n_input * n_steps]), dtype="float32")
}
# ...

pred = RNN(x, weights, biases)
pred = tf.reshape(pred, [-1, n_input, n_steps])

在您的模型上

但是，您在这里所做的是对图像的每一列进行 RNN。您正在尝试获取图像的每个切片（总共 200 个）并对其进行迭代，这根本不会产生好的结果。

如果你想在图像上工作，我建议你看看this tutorial来自TensorFlow，在那里你可以学习使用卷积，比RNN更有效在图片上。

Tensorflow LSTM 逐像素分类

Tensorflow LSTM pixel-wise classification

python

machine-learning

neural-network

lstm

tensorflow

-------------------------------------------- ------------------------------

在您的模型上