如何在 Tensorflow 中使用 sampled_softmax_loss

Question

我是tensorflow的初学者。我已经建立了简单的模型，但还没有尝试过像多层 LSTM 这样的东西，所以非常感谢任何类型的反馈:)

我目前正在尝试从头开始重新编码由 sherjilozair 构建的字符级模型，仅仅是因为我想知道如何使用 tensorflow（我之前构建了自己的非常小的 DL-由 cs231n 分配的库）。现在我正在努力构建一个简单的 2 层 LSTM 模型，我不确定哪里出了问题。这是我到目前为止编写的代码：

class Model():
    def __init__(self, batch_size, seq_length, lstm_size, num_layers, grad_clip, vocab_size):
        self.lr = tf.Variable(0.0, trainable=False)        

        #Define input and output
        self.input_data = tf.placeholder(tf.float32, [batch_size, seq_length])
        self.output_data = tf.placeholder(tf.float32, [batch_size, seq_length]) #although int would be better for character level..

        #Define the model
        cell = tf.nn.rnn_cell.BasicLSTMCell(num_units=lstm_size) #can choose if basic or otherwise later on...
        self.cell = cell = rnn_cell.MultiRNNCell([cell] * num_layers)
        self.initial_state = cell.zero_state(batch_size, tf.float32)


        with tf.variable_scope("lstm"):
            softmax_w = tf.get_variable("softmax_w", [lstm_size, vocab_size])
            softmax_b = tf.get_variable("softmax_b", [vocab_size])

        #_, enc_state = rnn.rnn(cell, encoder_inputs, dtype=dtype)
        #outputs, states = rnn_decoder(decoder_inputs, enc_state, cell)


        outputs, states = seq2seq.basic_rnn_seq2seq(
                            [self.input_data],
                            [self.output_data], 
                            cell,
                            scope='lstm'
                        )


        #see how attention helps improving this model state...

        #was told that we should actually use samples softmax loss
        self.loss = tf.nn.sampled_softmax_loss(
                                    softmax_w, 
                                    softmax_b,
                                    outputs, 
                                    self.output_data,
                                    batch_size,
                                    vocab_size
                )

我目前遇到 tf.nn.sampled_softmax_loss 的问题。我在调试方面走了很长一段路，但不了解 Tensorflow 的输入约定。我每次都必须输入张量列表吗？

我收到以下错误：

Traceback (most recent call last):
  File "Model.py", line 76, in <module>
vocab_size=82
  File "Model.py", line 52, in __init__
vocab_size
  File "/usr/local/lib/python2.7/site-packages/tensorflow/python/ops/nn.py", line 1104, in sampled_softmax_loss
name=name)
  File "/usr/local/lib/python2.7/site-packages/tensorflow/python/ops/nn.py", line 913, in _compute_sampled_logits
array_ops.expand_dims(inputs, 1),
  File "/usr/local/lib/python2.7/site-packages/tensorflow/python/ops/gen_array_ops.py", line 506, in expand_dims
return _op_def_lib.apply_op("ExpandDims", input=input, dim=dim, name=name)
  File "/usr/local/lib/python2.7/site-packages/tensorflow/python/ops/op_def_library.py", line 411, in apply_op
as_ref=input_arg.is_ref)
  File "/usr/local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 566, in convert_to_tensor
ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
  File "/usr/local/lib/python2.7/site-packages/tensorflow/python/ops/constant_op.py", line 179, in _constant_tensor_conversion_function
return constant(v, dtype=dtype, name=name)
  File "/usr/local/lib/python2.7/site-packages/tensorflow/python/ops/constant_op.py", line 162, in constant
tensor_util.make_tensor_proto(value, dtype=dtype, shape=shape))
  File "/usr/local/lib/python2.7/site-packages/tensorflow/python/framework/tensor_util.py", line 332, in make_tensor_proto
_AssertCompatible(values, dtype)
  File "/usr/local/lib/python2.7/site-packages/tensorflow/python/framework/tensor_util.py", line 269, in _AssertCompatible
    raise TypeError("List of Tensors when single Tensor expected")
TypeError: List of Tensors when single Tensor expected

我不确定我在输入或变量生成等方面做错了什么。问题 - 如前所述 - 似乎在 sampled_softmax_loss 函数中，但我真的不确定.. 我正在使用以下参数调用 class（就像占位符一样，只是为了测试模型是否为 'runnable'）：

Model = Model(batch_size=32, 
              seq_length=128, 
              lstm_size=512, 
              num_layers=2, 
              grad_clip=5,
              vocab_size=82
             )

另外，如果我还有其他错误等等，请在评论中告诉我！这是我在 tensorflow 中使用 seq2seq 模型的第一个模型，非常感谢任何建议！

Answer 1

这个特殊错误是关于传递 outputs 这是一个列表，而 tf.nn.sampled_softmax_loss 需要一个张量。

seq2seq.basic_rnn_seq2seq function returns a list of tensors of size [batch_size x output_size] as the first output. Assuming each of your outputs is one-dimensional, you want to concatenate the output list using tf.concat (creating a tensor of size [seq_len x batch_size x 1]), tf.squeeze the last dimension (resulting [seq_len x batch_size]) and tf.transpose 使 output 的大小为 [batch_size x seq_len]，与 self.output_data 相同。

要调试问题，请使用 print(output.get_shape()) 打印张量大小。

如何在 Tensorflow 中使用 sampled_softmax_loss

How to use sampled_softmax_loss in Tensorflow

python

numpy

deep-learning

lstm

tensorflow