Tensorflow 2 急切执行在自定义层内被禁用

Question

我在 ubuntu 18.04 box

中使用通过 pip 安装的 TF2

$ pip freeze | grep "tensorflow"
tensorflow==2.0.0
tensorflow-estimator==2.0.1

我正在玩自定义图层。

import tensorflow as tf
from tensorflow.keras.preprocessing import sequence
from tensorflow.keras.layers import Input, Concatenate, Dense, Bidirectional, LSTM, Embedding
from tensorflow.keras.models import Model
from tensorflow.keras.datasets import imdb

class Attention(tf.keras.layers.Layer):

    def __init__(self, units):
        super(Attention, self).__init__()
        self.W1 = Dense(units)
        self.W2 = Dense(units)
        self.V = Dense(1)

    def call(self, features, hidden):
        hidden_with_time_axis = tf.expand_dims(hidden, 1)
        score = tf.nn.tanh(self.W1(features) + self.W2(hidden_with_time_axis))
        attention_weights = tf.nn.softmax(self.V(score), axis=1)
        context_vector = attention_weights * features
        context_vector = tf.reduce_sum(context_vector, axis=1)

        return context_vector, attention_weights

vocab_size = 10000

(x_train, y_train), (x_test, y_test) = imdb.load_data(num_words=vocab_size)

max_len = 200
rnn_cell_size = 128

x_train = sequence.pad_sequences(x_train, maxlen=max_len, padding='post')
x_test = sequence.pad_sequences(x_test, maxlen=max_len, truncating='post', padding='post')

# Network

sequence_input = Input(shape=(max_len,), dtype='int32')

embedded_sequences = Embedding(vocab_size, 128, input_length=max_len)(sequence_input)

# lstm = Bidirectional(LSTM(rnn_cell_size, dropout=0.3, return_sequences=True, return_state=True), name="bi_lstm_0")(embedded_sequences)

lstm, forward_h, forward_c, backward_h, backward_c = Bidirectional(LSTM(rnn_cell_size, dropout=0.2, return_sequences=True, return_state=True))(embedded_sequences)

state_h = Concatenate()([forward_h, backward_h])
state_c = Concatenate()([forward_c, backward_c])

attention = Attention(8)

context_vector, attention_weights = attention(lstm, state_h)

output = Dense(1, activation='sigmoid')(context_vector)

model = Model(inputs=sequence_input, outputs=output)

# summarize layers
print(model.summary())

model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

history = model.fit(x_train, y_train, epochs=10, batch_size=200, validation_split=.3, verbose=1)

result = model.evaluate(x_test, y_test)
print(result)

我想 debug/inspect Attention.call() 函数，但是我无法在设置断点时获取张量值在函数内部。

在启动 .fit() 之前，我可以验证是否已启用即时执行

print(tf.executing_eagerly())
True

但是在 Attention.call() 函数内部禁用了急切执行

print(tf.executing_eagerly())
False

在 call() 执行期间 eager execution 是否为 false 的任何原因？如何启用它？

Answer 1

默认情况下，tf.keras 模型被编译为静态图以提供最佳执行性能。想想 @tf.function 默认情况下为 tf.keras 模型注释。

https://www.tensorflow.org/api_docs/python/tf/keras/Model#run_eagerly

要为 tf.keras 模型显式启用 eager 模式，请在您的代码中使用 run_eagerly=True 编译模型。

model.compile(optimizer='adam', run_eagerly = True, loss='binary_crossentropy', metrics=['accuracy'])

Tensorflow 2 急切执行在自定义层内被禁用

Tensorflow 2 eager execution disabled inside a custom layer

python

tensorflow

eager-execution

tensorflow2.0