当 layer_norm（在 LayerNormBasicLSTMCell 中）是张量时，dynamic_rnn 给出“`tf.Tensor` 作为 Python `bool`”错误

Question

我想在以下代码中使用在 tensorflow here 中实现层归一化的 LSTM 单元：

import tensorflow as tf 
import numpy as np 

n = 10
batch_size_series = 16

x = tf.placeholder(shape=[None, None, 1], dtype=tf.float32, name="x")  
layer_norm = tf.placeholder(tf.bool, name="normalize_layer")

cell = tf.contrib.rnn.LayerNormBasicLSTMCell(n, layer_norm=layer_norm)

initial_state = cell.zero_state(batch_size_series, dtype=tf.float32)

output_state, current_state = tf.nn.dynamic_rnn(cell, inputs=x, initial_state=initial_state, dtype=tf.float32)

当 layer_norm 是 Tensor 时出现以下错误：

TypeError: Using a `tf.Tensor` as a Python `bool` is not allowed. Use `if t is not None:` instead of `if t:` to test if a tensor is defined, and use TensorFlow ops such as tf.cond to execute subgraphs conditioned on the value of a tensor.

跟踪可以追溯到 dynamic_rnn:

Traceback (most recent call last):
  File "test.py", line 19, in <module>
    output_state, current_state = tf.nn.dynamic_rnn(multi_rnn_cell, inputs=x, initial_state=initial_state, dtype=tf.float32)
  File "/Users/fghavamian/ve_tf2/lib/python3.6/site-packages/tensorflow/python/ops/rnn.py", line 614, in dynamic_rnn
    dtype=dtype)
  File "/Users/fghavamian/ve_tf2/lib/python3.6/site-packages/tensorflow/python/ops/rnn.py", line 777, in _dynamic_rnn_loop
    swap_memory=swap_memory)
  File "/Users/fghavamian/ve_tf2/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2816, in while_loop
    result = loop_context.BuildLoop(cond, body, loop_vars, shape_invariants)
  File "/Users/fghavamian/ve_tf2/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2640, in BuildLoop
    pred, body, original_loop_vars, loop_vars, shape_invariants)
  File "/Users/fghavamian/ve_tf2/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2590, in _BuildLoop
    body_result = body(*packed_vars_for_body)
  File "/Users/fghavamian/ve_tf2/lib/python3.6/site-packages/tensorflow/python/ops/rnn.py", line 762, in _time_step
    (output, new_state) = call_cell()
  File "/Users/fghavamian/ve_tf2/lib/python3.6/site-packages/tensorflow/python/ops/rnn.py", line 748, in <lambda>
    call_cell = lambda: cell(input_t, state)
  File "/Users/fghavamian/ve_tf2/lib/python3.6/site-packages/tensorflow/python/ops/rnn_cell_impl.py", line 183, in __call__
    return super(RNNCell, self).__call__(inputs, state)
  File "/Users/fghavamian/ve_tf2/lib/python3.6/site-packages/tensorflow/python/layers/base.py", line 575, in __call__
    outputs = self.call(inputs, *args, **kwargs)
  File "/Users/fghavamian/ve_tf2/lib/python3.6/site-packages/tensorflow/python/ops/rnn_cell_impl.py", line 1066, in call
    cur_inp, new_state = cell(cur_inp, cur_state)
  File "/Users/fghavamian/ve_tf2/lib/python3.6/site-packages/tensorflow/python/ops/rnn_cell_impl.py", line 183, in __call__
    return super(RNNCell, self).__call__(inputs, state)
  File "/Users/fghavamian/ve_tf2/lib/python3.6/site-packages/tensorflow/python/layers/base.py", line 575, in __call__
    outputs = self.call(inputs, *args, **kwargs)
  File "/Users/fghavamian/ve_tf2/lib/python3.6/site-packages/tensorflow/contrib/rnn/python/ops/rnn_cell.py", line 1340, in call
    concat = self._linear(args)
  File "/Users/fghavamian/ve_tf2/lib/python3.6/site-packages/tensorflow/contrib/rnn/python/ops/rnn_cell.py", line 1331, in _linear
    if not self._layer_norm:
  File "/Users/fghavamian/ve_tf2/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 528, in __bool__

我是不是遗漏了什么？

Answer 1

tf.contrib.rnn.LayerNormBasicLSTMCell initializer expects a Python boolean and not a tf.Tensor as the layer_norm argument. The reason for this is that the value of this argument needs to be known at graph construction time, in order to create the appropriate variables for layer normalization (e.g. the "gamma" and "beta" variables created here).

当 layer_norm（在 LayerNormBasicLSTMCell 中）是张量时，dynamic_rnn 给出“`tf.Tensor` 作为 Python `bool`”错误

dynamic_rnn gives "`tf.Tensor` as a Python `bool`" error when layer_norm (in LayerNormBasicLSTMCell) is a Tensor

python

tensorflow

rnn