预期 input_1 有 3 个维度，但得到形状为 (3, 4) 的数组
expected input_1 to have 3 dimensions, but got array with shape (3, 4)
这是我的代码的简化版本，它引发了标题中提到的错误：
import tensorflow as tf
BATCH_SIZE = 3
SEQ_LENGTH = 4
NUM_CLASSES = 2
LSTM_UNITS = 64
NUM_SHARDS = 4
NUM_CHANNELS = 2
tf.enable_eager_execution()
def keras_model():
inputs = tf.keras.layers.Input(shape=(SEQ_LENGTH, NUM_CHANNELS))
x = tf.keras.layers.Bidirectional(
tf.keras.layers.LSTM(LSTM_UNITS, return_sequences=True))(inputs)
outputs = tf.keras.layers.TimeDistributed(tf.keras.layers.Dense(NUM_CLASSES, activation='relu'))(x)
return tf.keras.Model(inputs, outputs)
dataset = tf.data.experimental.CsvDataset(filenames='../../input/aFile.csv', header=True,record_defaults=[tf.int64] * 3, select_cols=[0,1,2])
dataset= dataset.window(size=SEQ_LENGTH, shift=1, drop_remainder=True).flat_map(lambda f1,f2, label:
tf.data.Dataset.zip((tf.data.Dataset.zip((f1.batch(SEQ_LENGTH),f2.batch(SEQ_LENGTH))), label.batch(SEQ_LENGTH))))
dataset = dataset.batch(BATCH_SIZE, drop_remainder=True)
train_iterator = dataset.make_one_shot_iterator()
train_features, train_labels = train_iterator.get_next()
print(train_features)
print(train_labels)
model = keras_model()
model.summary()
model.compile(optimizer='adam',loss='sparse_categorical_crossentropy',metrics=['accuracy'])
model.fit(x=train_features,y=train_labels, batch_size=BATCH_SIZE,epochs=1, steps_per_epoch=10)
这是代码的输出：
...
(<tf.Tensor: id=44, shape=(3, 4), dtype=int64, numpy=
array([[0, 1, 2, 3],
[1, 2, 3, 4],
[2, 3, 4, 5]], dtype=int64)>, <tf.Tensor: id=45, shape=(3, 4), dtype=int64, numpy=
array([[100, 101, 102, 103],
[101, 102, 103, 104],
[102, 103, 104, 105]], dtype=int64)>)
tf.Tensor(
[[0 0 0 0]
[0 0 0 1]
[0 0 1 0]], shape=(3, 4), dtype=int64)
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_1 (InputLayer) (None, 4, 2) 0
_________________________________________________________________
bidirectional (Bidirectional (None, 4, 128) 34304
_________________________________________________________________
time_distributed (TimeDistri (None, 4, 2) 258
=================================================================
Total params: 34,562
Trainable params: 34,562
Non-trainable params: 0
_________________________________________________________________
...
ValueError: Error when checking input: expected input_1 to have 3 dimensions, but got array with shape (3, 4)
Process finished with exit code 1
我将此 csv 文件用于演示
f1,f2,label
0,100,0
1,101,0
2,102,0
3,103,0
4,104,1
5,105,0
6,106,0
7,107,0
8,108,1
9,109,0
10,110,0
前两列是来自两个不同渠道的特征列，最后一列包含标签。我需要使用例如四行数据的序列作为输入模型的时间，而批量大小例如为三，因此输入形状就像三批四行，其中每行包含两个值。
我想我需要使用某种重塑功能，但不知道如何使用。
有人可以告诉我如何解决这个问题吗？
在第一次尝试解决问题时，我更改了代码以首先组合通道来构建特征列，然后制作特征列的序列。这将输入的形状从 [batch_size、channel_num、sequence_length] 更改为 [batch_size、sequence_length、channel_num] 并添加了维度到模型所期望的标签。
这是新代码：
import tensorflow as tf
import numpy as np
BATCH_SIZE = 3
SEQ_LENGTH = 4
NUM_CLASSES = 2
LSTM_UNITS = 64
NUM_SHARDS = 4
NUM_CHANNELS = 2
tf.enable_eager_execution()
def parse_values(f1, f2, label):
features = [f1,f2]
return features, label
def keras_model():
inputs = tf.keras.layers.Input(shape=(SEQ_LENGTH,NUM_CHANNELS))
x = tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(LSTM_UNITS, return_sequences=True))(inputs)
outputs = tf.keras.layers.TimeDistributed(tf.keras.layers.Dense(NUM_CLASSES, activation='relu'))(x)
return tf.keras.Model(inputs, outputs)
dataset = tf.data.experimental.CsvDataset(filenames='../../input/aFile.csv', header=True,record_defaults=[tf.int64] * 3, select_cols=[0,1,2])
dataset= dataset.map(parse_values).window(size=SEQ_LENGTH, shift=1, drop_remainder=True).flat_map(lambda features, label:
tf.data.Dataset.zip((features.batch(SEQ_LENGTH), label.batch(SEQ_LENGTH))))
dataset = dataset.batch(BATCH_SIZE, drop_remainder=True)
train_iterator = dataset.make_one_shot_iterator()
train_features, train_labels = train_iterator.get_next()
print(train_features)
#train_labels = train_labels[:,SEQ_LENGTH-1] # output => [0 1 0]
#print(train_labels)
train_labels = np.expand_dims(train_labels, axis=2)
print(train_labels)
model = keras_model()
model.summary()
model.compile(optimizer='adam',loss='sparse_categorical_crossentropy',metrics=['accuracy'])
model.fit(x=train_features,y=train_labels, batch_size=BATCH_SIZE,epochs=1, steps_per_epoch=10)
下面是输出：
...tf.Tensor(
[[[ 0 100]
[ 1 101]
[ 2 102]
[ 3 103]]
[[ 1 101]
[ 2 102]
[ 3 103]
[ 4 104]]
[[ 2 102]
[ 3 103]
[ 4 104]
[ 5 105]]], shape=(3, 4, 2), dtype=int64)
[[[0]
[0]
[0]
[0]]
[[0]
[0]
[0]
[1]]
[[0]
[0]
[1]
[0]]]
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_1 (InputLayer) (None, 4, 2) 0
_________________________________________________________________
bidirectional (Bidirectional (None, 4, 128) 34304
_________________________________________________________________
time_distributed (TimeDistri (None, 4, 2) 258
=================================================================
Total params: 34,562
Trainable params: 34,562
Non-trainable params: 0
_________________________________________________________________
1/10 [==>...........................] - ETA: 8s - loss: 13.3860 - acc: 0.1667
10/10 [==============================] - 1s 101ms/step - loss: 12.9909 - acc: 0.1667
Process finished with exit code 0
每个序列只有一个标签来确定序列是属于类别 0 还是类别 1 对我来说更有意义（在我的例子中，每个批次有 3 个值，因为批次大小为 3）。我试图通过添加一行代码（如下所示）来做到这一点，因为它导致异常 'Incompatible shapes: [3] vs. [3,4]'
我后来不得不注释掉它
train_labels = train_labels[:,SEQ_LENGTH-1] # output => [0 1 0]
我无法弄清楚如何修复该错误，因此正如您在输出中看到的那样，我将序列中包含的所有行的标签提供给了模型。
后来我想出了一个技巧，可以为序列中的所有项目设置相同的标签。我决定将序列中的所有标签设置为序列的最后一个标签。例如 [0 0 0 1] 会变成 [1 1 1 1] 而 [0 0 1 0] 会变成 [0 0 0 0]。我还将损失函数更改为 'binary_crossentropy'，因为这里的问题是二元分类。下面是代码：
import tensorflow as tf
import numpy as np
BATCH_SIZE = 3
SEQ_LENGTH = 4
NUM_CLASSES = 1
LSTM_UNITS = 64
NUM_SHARDS = 4
NUM_CHANNELS = 2
tf.enable_eager_execution()
def parse_values(f1, f2, label):
features = [f1,f2]
return features, label
def map_label(features, label):
sequence_label1 = tf.fill([SEQ_LENGTH],label[SEQ_LENGTH-1])
return features, sequence_label1
def keras_model():
inputs = tf.keras.layers.Input(shape=(SEQ_LENGTH,NUM_CHANNELS),batch_size=BATCH_SIZE)
x = tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(LSTM_UNITS, return_sequences=True))(inputs)
outputs = tf.keras.layers.TimeDistributed(tf.keras.layers.Dense(NUM_CLASSES, activation='sigmoid'))(x)
return tf.keras.Model(inputs, outputs)
dataset = tf.data.experimental.CsvDataset(filenames='../../input/aFile.csv', header=True,record_defaults=[tf.int64] * 3, select_cols=[0,1,2])
dataset= dataset.map(parse_values).window(size=SEQ_LENGTH, shift=1, drop_remainder=True).flat_map(lambda features, label:
tf.data.Dataset.zip((features.batch(SEQ_LENGTH), label.batch(SEQ_LENGTH)))).map(map_label)
dataset = dataset.batch(BATCH_SIZE, drop_remainder=True)
train_iterator = dataset.make_one_shot_iterator()
train_features, train_labels = train_iterator.get_next()
print(train_features)
train_labels = np.expand_dims(train_labels, axis=2)
print(train_labels)
model = keras_model()
model.summary()
model.compile(optimizer='adam',loss='binary_crossentropy',metrics=['accuracy'])
model.fit(x=train_features,y=train_labels, batch_size=BATCH_SIZE,epochs=1, steps_per_epoch=10)
下面是输出：
...tf.Tensor(
[[[ 0 100]
[ 1 101]
[ 2 102]
[ 3 103]]
[[ 1 101]
[ 2 102]
[ 3 103]
[ 4 104]]
[[ 2 102]
[ 3 103]
[ 4 104]
[ 5 105]]], shape=(3, 4, 2), dtype=int64)
[[[0]
[0]
[0]
[0]]
[[1]
[1]
[1]
[1]]
[[0]
[0]
[0]
[0]]]
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_1 (InputLayer) (3, 4, 2) 0
_________________________________________________________________
bidirectional (Bidirectional (3, 4, 128) 34304
_________________________________________________________________
time_distributed (TimeDistri (3, 4, 1) 129
=================================================================
Total params: 34,433
Trainable params: 34,433
Non-trainable params: 0
_________________________________________________________________
...
1/10 [==>...........................] - ETA: 10s - loss: 0.6866 - acc: 0.5833
10/10 [==============================] - 1s 124ms/step - loss: 0.6571 - acc: 0.6500
Process finished with exit code 0
我希望这对面临类似问题的任何人有所帮助。
我今天再次调查了这个问题，我认为你可以通过像这样修改解析函数来解决这个问题：
def parse_values(f1, f2, label):
features = tf.stack([f1, f2], 0)
return features, label
这是我的代码的简化版本，它引发了标题中提到的错误：
import tensorflow as tf
BATCH_SIZE = 3
SEQ_LENGTH = 4
NUM_CLASSES = 2
LSTM_UNITS = 64
NUM_SHARDS = 4
NUM_CHANNELS = 2
tf.enable_eager_execution()
def keras_model():
inputs = tf.keras.layers.Input(shape=(SEQ_LENGTH, NUM_CHANNELS))
x = tf.keras.layers.Bidirectional(
tf.keras.layers.LSTM(LSTM_UNITS, return_sequences=True))(inputs)
outputs = tf.keras.layers.TimeDistributed(tf.keras.layers.Dense(NUM_CLASSES, activation='relu'))(x)
return tf.keras.Model(inputs, outputs)
dataset = tf.data.experimental.CsvDataset(filenames='../../input/aFile.csv', header=True,record_defaults=[tf.int64] * 3, select_cols=[0,1,2])
dataset= dataset.window(size=SEQ_LENGTH, shift=1, drop_remainder=True).flat_map(lambda f1,f2, label:
tf.data.Dataset.zip((tf.data.Dataset.zip((f1.batch(SEQ_LENGTH),f2.batch(SEQ_LENGTH))), label.batch(SEQ_LENGTH))))
dataset = dataset.batch(BATCH_SIZE, drop_remainder=True)
train_iterator = dataset.make_one_shot_iterator()
train_features, train_labels = train_iterator.get_next()
print(train_features)
print(train_labels)
model = keras_model()
model.summary()
model.compile(optimizer='adam',loss='sparse_categorical_crossentropy',metrics=['accuracy'])
model.fit(x=train_features,y=train_labels, batch_size=BATCH_SIZE,epochs=1, steps_per_epoch=10)
这是代码的输出：
...
(<tf.Tensor: id=44, shape=(3, 4), dtype=int64, numpy=
array([[0, 1, 2, 3],
[1, 2, 3, 4],
[2, 3, 4, 5]], dtype=int64)>, <tf.Tensor: id=45, shape=(3, 4), dtype=int64, numpy=
array([[100, 101, 102, 103],
[101, 102, 103, 104],
[102, 103, 104, 105]], dtype=int64)>)
tf.Tensor(
[[0 0 0 0]
[0 0 0 1]
[0 0 1 0]], shape=(3, 4), dtype=int64)
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_1 (InputLayer) (None, 4, 2) 0
_________________________________________________________________
bidirectional (Bidirectional (None, 4, 128) 34304
_________________________________________________________________
time_distributed (TimeDistri (None, 4, 2) 258
=================================================================
Total params: 34,562
Trainable params: 34,562
Non-trainable params: 0
_________________________________________________________________
...
ValueError: Error when checking input: expected input_1 to have 3 dimensions, but got array with shape (3, 4)
Process finished with exit code 1
我将此 csv 文件用于演示
f1,f2,label
0,100,0
1,101,0
2,102,0
3,103,0
4,104,1
5,105,0
6,106,0
7,107,0
8,108,1
9,109,0
10,110,0
前两列是来自两个不同渠道的特征列，最后一列包含标签。我需要使用例如四行数据的序列作为输入模型的时间，而批量大小例如为三，因此输入形状就像三批四行，其中每行包含两个值。 我想我需要使用某种重塑功能，但不知道如何使用。 有人可以告诉我如何解决这个问题吗？
在第一次尝试解决问题时，我更改了代码以首先组合通道来构建特征列，然后制作特征列的序列。这将输入的形状从 [batch_size、channel_num、sequence_length] 更改为 [batch_size、sequence_length、channel_num] 并添加了维度到模型所期望的标签。 这是新代码：
import tensorflow as tf
import numpy as np
BATCH_SIZE = 3
SEQ_LENGTH = 4
NUM_CLASSES = 2
LSTM_UNITS = 64
NUM_SHARDS = 4
NUM_CHANNELS = 2
tf.enable_eager_execution()
def parse_values(f1, f2, label):
features = [f1,f2]
return features, label
def keras_model():
inputs = tf.keras.layers.Input(shape=(SEQ_LENGTH,NUM_CHANNELS))
x = tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(LSTM_UNITS, return_sequences=True))(inputs)
outputs = tf.keras.layers.TimeDistributed(tf.keras.layers.Dense(NUM_CLASSES, activation='relu'))(x)
return tf.keras.Model(inputs, outputs)
dataset = tf.data.experimental.CsvDataset(filenames='../../input/aFile.csv', header=True,record_defaults=[tf.int64] * 3, select_cols=[0,1,2])
dataset= dataset.map(parse_values).window(size=SEQ_LENGTH, shift=1, drop_remainder=True).flat_map(lambda features, label:
tf.data.Dataset.zip((features.batch(SEQ_LENGTH), label.batch(SEQ_LENGTH))))
dataset = dataset.batch(BATCH_SIZE, drop_remainder=True)
train_iterator = dataset.make_one_shot_iterator()
train_features, train_labels = train_iterator.get_next()
print(train_features)
#train_labels = train_labels[:,SEQ_LENGTH-1] # output => [0 1 0]
#print(train_labels)
train_labels = np.expand_dims(train_labels, axis=2)
print(train_labels)
model = keras_model()
model.summary()
model.compile(optimizer='adam',loss='sparse_categorical_crossentropy',metrics=['accuracy'])
model.fit(x=train_features,y=train_labels, batch_size=BATCH_SIZE,epochs=1, steps_per_epoch=10)
下面是输出：
...tf.Tensor(
[[[ 0 100]
[ 1 101]
[ 2 102]
[ 3 103]]
[[ 1 101]
[ 2 102]
[ 3 103]
[ 4 104]]
[[ 2 102]
[ 3 103]
[ 4 104]
[ 5 105]]], shape=(3, 4, 2), dtype=int64)
[[[0]
[0]
[0]
[0]]
[[0]
[0]
[0]
[1]]
[[0]
[0]
[1]
[0]]]
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_1 (InputLayer) (None, 4, 2) 0
_________________________________________________________________
bidirectional (Bidirectional (None, 4, 128) 34304
_________________________________________________________________
time_distributed (TimeDistri (None, 4, 2) 258
=================================================================
Total params: 34,562
Trainable params: 34,562
Non-trainable params: 0
_________________________________________________________________
1/10 [==>...........................] - ETA: 8s - loss: 13.3860 - acc: 0.1667
10/10 [==============================] - 1s 101ms/step - loss: 12.9909 - acc: 0.1667
Process finished with exit code 0
每个序列只有一个标签来确定序列是属于类别 0 还是类别 1 对我来说更有意义（在我的例子中，每个批次有 3 个值，因为批次大小为 3）。我试图通过添加一行代码（如下所示）来做到这一点，因为它导致异常 'Incompatible shapes: [3] vs. [3,4]'
我后来不得不注释掉它train_labels = train_labels[:,SEQ_LENGTH-1] # output => [0 1 0]
我无法弄清楚如何修复该错误，因此正如您在输出中看到的那样，我将序列中包含的所有行的标签提供给了模型。 后来我想出了一个技巧，可以为序列中的所有项目设置相同的标签。我决定将序列中的所有标签设置为序列的最后一个标签。例如 [0 0 0 1] 会变成 [1 1 1 1] 而 [0 0 1 0] 会变成 [0 0 0 0]。我还将损失函数更改为 'binary_crossentropy'，因为这里的问题是二元分类。下面是代码：
import tensorflow as tf
import numpy as np
BATCH_SIZE = 3
SEQ_LENGTH = 4
NUM_CLASSES = 1
LSTM_UNITS = 64
NUM_SHARDS = 4
NUM_CHANNELS = 2
tf.enable_eager_execution()
def parse_values(f1, f2, label):
features = [f1,f2]
return features, label
def map_label(features, label):
sequence_label1 = tf.fill([SEQ_LENGTH],label[SEQ_LENGTH-1])
return features, sequence_label1
def keras_model():
inputs = tf.keras.layers.Input(shape=(SEQ_LENGTH,NUM_CHANNELS),batch_size=BATCH_SIZE)
x = tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(LSTM_UNITS, return_sequences=True))(inputs)
outputs = tf.keras.layers.TimeDistributed(tf.keras.layers.Dense(NUM_CLASSES, activation='sigmoid'))(x)
return tf.keras.Model(inputs, outputs)
dataset = tf.data.experimental.CsvDataset(filenames='../../input/aFile.csv', header=True,record_defaults=[tf.int64] * 3, select_cols=[0,1,2])
dataset= dataset.map(parse_values).window(size=SEQ_LENGTH, shift=1, drop_remainder=True).flat_map(lambda features, label:
tf.data.Dataset.zip((features.batch(SEQ_LENGTH), label.batch(SEQ_LENGTH)))).map(map_label)
dataset = dataset.batch(BATCH_SIZE, drop_remainder=True)
train_iterator = dataset.make_one_shot_iterator()
train_features, train_labels = train_iterator.get_next()
print(train_features)
train_labels = np.expand_dims(train_labels, axis=2)
print(train_labels)
model = keras_model()
model.summary()
model.compile(optimizer='adam',loss='binary_crossentropy',metrics=['accuracy'])
model.fit(x=train_features,y=train_labels, batch_size=BATCH_SIZE,epochs=1, steps_per_epoch=10)
下面是输出：
...tf.Tensor(
[[[ 0 100]
[ 1 101]
[ 2 102]
[ 3 103]]
[[ 1 101]
[ 2 102]
[ 3 103]
[ 4 104]]
[[ 2 102]
[ 3 103]
[ 4 104]
[ 5 105]]], shape=(3, 4, 2), dtype=int64)
[[[0]
[0]
[0]
[0]]
[[1]
[1]
[1]
[1]]
[[0]
[0]
[0]
[0]]]
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_1 (InputLayer) (3, 4, 2) 0
_________________________________________________________________
bidirectional (Bidirectional (3, 4, 128) 34304
_________________________________________________________________
time_distributed (TimeDistri (3, 4, 1) 129
=================================================================
Total params: 34,433
Trainable params: 34,433
Non-trainable params: 0
_________________________________________________________________
...
1/10 [==>...........................] - ETA: 10s - loss: 0.6866 - acc: 0.5833
10/10 [==============================] - 1s 124ms/step - loss: 0.6571 - acc: 0.6500
Process finished with exit code 0
我希望这对面临类似问题的任何人有所帮助。
我今天再次调查了这个问题，我认为你可以通过像这样修改解析函数来解决这个问题：
def parse_values(f1, f2, label):
features = tf.stack([f1, f2], 0)
return features, label