Tensorflow 字符级 CNN - 输入形状
Tensorflow character-level CNN - input shape
我正在尝试将 2 层堆叠字符级 CNN 添加到更大的神经网络系统中,但输入维度出现 ValueError。
我想要实现的是通过替换字符(根据大小写,或者是数字或字母)并将它们输入 CNN 来获得输入单词的正字法表示。我知道这可以通过 LSTM/RNN 实现,但要求表明使用 CNN,因此使用另一个 NN 不是可选的。
大多数例子自然地使用图像数据集(MNIST 等)而不是文本数据集。所以我很困惑,不确定如何 "reshape" 字符嵌入,以便它们可以成为 CNN 的有效输入。
所以这是我要尝试的代码部分 运行:
# ...
# shape = (batch size, max length of sentence, max length of word)
self.char_ids = tf.placeholder(tf.int32, shape=[None, None, None],
name="char_ids")
# ...
# Char embedding lookup
_char_embeddings = tf.get_variable(
name="_char_embeddings",
dtype=tf.float32,
shape=[self.config.nchars, self.config.dim_char])
char_embeddings = tf.nn.embedding_lookup(_char_embeddings,
self.char_ids, name="char_embeddings")
# Reshape for CNN?
s = tf.shape(char_embeddings)
char_embeddings = tf.reshape(char_embeddings, shape=[s[0]*s[1], self.config.dim_char, s[2]])
# Conv #1
conv1 = tf.layers.conv1d(
inputs=char_embeddings,
filters=64,
kernel_size=3,
padding="valid",
activation=tf.nn.relu)
# Conv #2
conv2 = tf.layers.conv1d(
inputs=conv1,
filters=64,
kernel_size=3,
padding="valid",
activation=tf.nn.relu)
pool2 = tf.layers.max_pooling1d(inputs=conv2, pool_size=2, strides=2)
# Dense Layer
output = tf.layers.dense(inputs=pool2, units=32, activation=tf.nn.relu)
# ...
这是我遇到的错误:
File "/home/emre/blstm-crf-ner/model/ner_model.py", line 159, in add_word_embeddings_op activation=tf.nn.relu)
File "/home/emre/blstm-crf-ner/virtner/lib/python3.4/site-packages/tensorflow/python/layers/convolutional.py", line 411, in conv1d return layer.apply(inputs)
File "/home/emre/blstm-crf-ner/virtner/lib/python3.4/site-packages/tensorflow/python/layers/base.py", line 809, in apply return self.__call__(inputs, *args, **kwargs)
File "/home/emre/blstm-crf-ner/virtner/lib/python3.4/site-packages/tensorflow/python/layers/base.py", line 680, in __call__ self.build(input_shapes)
File "/home/emre/blstm-crf-ner/virtner/lib/python3.4/site-packages/tensorflow/python/layers/convolutional.py", line 132, in build raise ValueError('The channel dimension of the inputs '
ValueError: The channel dimension of the inputs should be defined. Found `None`.
如有任何帮助,我们将不胜感激。
谢谢。
更新
所以在阅读了一些博文 1, 2 并感谢 vijay m 之后,我了解到我们必须事先提供输入维度(与提供 sequence_length
s 和 RNN/LSTM 不同)。所以这是最终的代码片段:
# Char embedding lookup
_char_embeddings = tf.get_variable(
name="_char_embeddings",
dtype=tf.float32,
shape=[self.config.nchars, self.config.dim_char])
char_embeddings = tf.nn.embedding_lookup(_char_embeddings,
self.char_ids, name="char_embeddings")
# max_len_of_word: 20
# Just pad shorter words and truncate the longer ones.
s = tf.shape(char_embeddings)
char_embeddings = tf.reshape(char_embeddings, shape=[-1, self.config.dim_char, self.config.max_len_of_word])
# Conv #1
conv1 = tf.layers.conv1d(
inputs=char_embeddings,
filters=64,
kernel_size=3,
padding="valid",
activation=tf.nn.relu)
# Conv #2
conv2 = tf.layers.conv1d(
inputs=conv1,
filters=64,
kernel_size=3,
padding="valid",
activation=tf.nn.relu)
pool2 = tf.layers.max_pooling1d(inputs=conv2, pool_size=2, strides=2)
# Dense Layer
output = tf.layers.dense(inputs=pool2, units=32, activation=tf.nn.relu)
conv1d
期望在创建图形期间定义通道维度。所以你不能将尺寸传递为 None
.
您需要进行以下更改:
char_ids = tf.placeholder(tf.int32, shape=[None, max_len_sen, max_len_word],
name="char_ids")
#max_len_sen and max_len_word has to be set.
#Reshapping for CNN, should be
s = char_embeddings.get_shape()
char_embeddings = tf.reshape(char_embeddings, shape=[-1, dim_char, s[2]])
Conv1d 中输入的默认格式具有形状(批次、长度、通道),也许 char_embeddings 应该是这样的:
s = char_embeddings.get_shape()
char_embeddings = tf.reshape(char_embeddings, shape=[-1, s[2], dim_char])
谢谢!
我正在尝试将 2 层堆叠字符级 CNN 添加到更大的神经网络系统中,但输入维度出现 ValueError。
我想要实现的是通过替换字符(根据大小写,或者是数字或字母)并将它们输入 CNN 来获得输入单词的正字法表示。我知道这可以通过 LSTM/RNN 实现,但要求表明使用 CNN,因此使用另一个 NN 不是可选的。
大多数例子自然地使用图像数据集(MNIST 等)而不是文本数据集。所以我很困惑,不确定如何 "reshape" 字符嵌入,以便它们可以成为 CNN 的有效输入。
所以这是我要尝试的代码部分 运行:
# ...
# shape = (batch size, max length of sentence, max length of word)
self.char_ids = tf.placeholder(tf.int32, shape=[None, None, None],
name="char_ids")
# ...
# Char embedding lookup
_char_embeddings = tf.get_variable(
name="_char_embeddings",
dtype=tf.float32,
shape=[self.config.nchars, self.config.dim_char])
char_embeddings = tf.nn.embedding_lookup(_char_embeddings,
self.char_ids, name="char_embeddings")
# Reshape for CNN?
s = tf.shape(char_embeddings)
char_embeddings = tf.reshape(char_embeddings, shape=[s[0]*s[1], self.config.dim_char, s[2]])
# Conv #1
conv1 = tf.layers.conv1d(
inputs=char_embeddings,
filters=64,
kernel_size=3,
padding="valid",
activation=tf.nn.relu)
# Conv #2
conv2 = tf.layers.conv1d(
inputs=conv1,
filters=64,
kernel_size=3,
padding="valid",
activation=tf.nn.relu)
pool2 = tf.layers.max_pooling1d(inputs=conv2, pool_size=2, strides=2)
# Dense Layer
output = tf.layers.dense(inputs=pool2, units=32, activation=tf.nn.relu)
# ...
这是我遇到的错误:
File "/home/emre/blstm-crf-ner/model/ner_model.py", line 159, in add_word_embeddings_op activation=tf.nn.relu)
File "/home/emre/blstm-crf-ner/virtner/lib/python3.4/site-packages/tensorflow/python/layers/convolutional.py", line 411, in conv1d return layer.apply(inputs)
File "/home/emre/blstm-crf-ner/virtner/lib/python3.4/site-packages/tensorflow/python/layers/base.py", line 809, in apply return self.__call__(inputs, *args, **kwargs)
File "/home/emre/blstm-crf-ner/virtner/lib/python3.4/site-packages/tensorflow/python/layers/base.py", line 680, in __call__ self.build(input_shapes)
File "/home/emre/blstm-crf-ner/virtner/lib/python3.4/site-packages/tensorflow/python/layers/convolutional.py", line 132, in build raise ValueError('The channel dimension of the inputs '
ValueError: The channel dimension of the inputs should be defined. Found `None`.
如有任何帮助,我们将不胜感激。
谢谢。
更新
所以在阅读了一些博文 1, 2 并感谢 vijay m 之后,我了解到我们必须事先提供输入维度(与提供 sequence_length
s 和 RNN/LSTM 不同)。所以这是最终的代码片段:
# Char embedding lookup
_char_embeddings = tf.get_variable(
name="_char_embeddings",
dtype=tf.float32,
shape=[self.config.nchars, self.config.dim_char])
char_embeddings = tf.nn.embedding_lookup(_char_embeddings,
self.char_ids, name="char_embeddings")
# max_len_of_word: 20
# Just pad shorter words and truncate the longer ones.
s = tf.shape(char_embeddings)
char_embeddings = tf.reshape(char_embeddings, shape=[-1, self.config.dim_char, self.config.max_len_of_word])
# Conv #1
conv1 = tf.layers.conv1d(
inputs=char_embeddings,
filters=64,
kernel_size=3,
padding="valid",
activation=tf.nn.relu)
# Conv #2
conv2 = tf.layers.conv1d(
inputs=conv1,
filters=64,
kernel_size=3,
padding="valid",
activation=tf.nn.relu)
pool2 = tf.layers.max_pooling1d(inputs=conv2, pool_size=2, strides=2)
# Dense Layer
output = tf.layers.dense(inputs=pool2, units=32, activation=tf.nn.relu)
conv1d
期望在创建图形期间定义通道维度。所以你不能将尺寸传递为 None
.
您需要进行以下更改:
char_ids = tf.placeholder(tf.int32, shape=[None, max_len_sen, max_len_word],
name="char_ids")
#max_len_sen and max_len_word has to be set.
#Reshapping for CNN, should be
s = char_embeddings.get_shape()
char_embeddings = tf.reshape(char_embeddings, shape=[-1, dim_char, s[2]])
Conv1d 中输入的默认格式具有形状(批次、长度、通道),也许 char_embeddings 应该是这样的:
s = char_embeddings.get_shape()
char_embeddings = tf.reshape(char_embeddings, shape=[-1, s[2], dim_char])
谢谢!