我正在尝试拟合 CuDNNLSTM 模型，但出现错误

Question

我正在学习递归神经网络，我发现了 CuDNNLSTM 层，它比通常的 LSTM 快得多。因此，我尝试拟合 CuDNNLSTM 模型，但唯一的问题是，程序显示的是“Epoch 1”，然后什么也没有发生，我的内核快死了（我在 jupyter-notebook 中工作）。在 jupyer 终端我发现了这个：

2022-05-25 22:22:59.693801: I tensorflow/stream_executor/cuda/cuda_dnn.cc:384] Loaded cuDNN version 8100

2022-05-25 22:23:00.149065: E tensorflow/stream_executor/cuda/cuda_driver.cc:1018] failed to synchronize the stop event: CUDA_ERROR_LAUNCH_FAILED: unspecified launch failure

2022-05-25 22:23:00.149218: E tensorflow/stream_executor/gpu/gpu_timer.cc:55] INTERNAL: Error destroying CUDA event: CUDA_ERROR_LAUNCH_FAILED: unspecified launch failure

2022-05-25 22:23:00.150008: E tensorflow/stream_executor/gpu/gpu_timer.cc:60] INTERNAL: Error destroying CUDA event: CUDA_ERROR_LAUNCH_FAILED: unspecified launch failure

2022-05-25 22:23:00.150355: F tensorflow/stream_executor/cuda/cuda_dnn.cc:217] Check failed: status== CUDNN_STATUS_SUCCESS (7 vs. 0)Failed to set cuDNN stream.

我已经为我的 tensorflow 版本安装了 tensorflow-gpu 和兼容的 CuDNN 和 CUDA

张量流版本：2.9.0

CUDA 版本：11.2

CuDNN 版本：8.1

我也尝试过相同的模型，但是使用 LSTM 层并且已经奏效，但仍然很慢，所以我想弄清楚如何使用 CuDNNLSTM 模型。

我的代码：

import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Dropout, LSTM
from tensorflow.compat.v1.keras.layers import CuDNNLSTM

mnist = tf.keras.datasets.mnist

(X_train, y_train), (X_test, y_test) = mnist.load_data()

X_train = X_train/255.0
X_test = X_test/255.0

model = Sequential()

model.add(CuDNNLSTM(128, input_shape=(X_train.shape[1:]), return_sequences=True)) 
model.add(Dropout(0.2))
      
model.add(CuDNNLSTM(128))
model.add(Dropout(0.2))
      
model.add(Dense(32, activation="relu"))
model.add(Dropout(0.2))
      
model.add(Dense(10, activation="softmax"))
      
opt = tf.keras.optimizers.Adam(learning_rate=1e-3, decay=1e-5)
      
model.compile(loss="sparse_categorical_crossentropy",
              optimizer=opt,
              metrics=["accuracy"])
      
model.fit(X_train, y_train, epochs=3, validation_data=(X_test, y_test))

如果有人遇到同样的问题或知道如何解决，我将不胜感激。提前致谢。

Answer 1

你用 tanh 激活函数试过吗？据我了解，它必须使用它。详情如下：

Long Short-Term Memory layer - Hochreiter 1997.
See the Keras RNN API guide
for details about the usage of RNN API.
Based on available runtime hardware and constraints, this layer
will choose different implementations (cuDNN-based or pure-TensorFlow)
to maximize the performance. If a GPU is available and all
the arguments to the layer meet the requirement of the cuDNN kernel
(see below for details), the layer will use a fast cuDNN implementation.
The requirements to use the cuDNN implementation are:

activation == tanh

recurrent_activation == sigmoid

recurrent_dropout == 0

unroll is False

use_bias is True

Inputs, if use masking, are strictly right-padded.

Eager execution is enabled in the outermost context.

我正在尝试拟合 CuDNNLSTM 模型，但出现错误

I am trying to fit an CuDNNLSTM model and I am getting an error

python

tensorflow

keras

jupyter-notebook

cudnn