TensorflowJs conv2d - 张量形状

Question

我想为音频文件创建机器学习模型。我将音频文件转换为（频谱图）张量。我的特征张量（音频文件）具有以下形状 [119, 241, 125]（119 个文件，241 个 samples/file，125 个 frequencies/sample）。通过样本，我定义了我在一个时间跨度内采集的样本，例如16 毫秒。我的输出形状将是 [119, numOptions].

我在音频识别方面遵循了这个tutorial from Tensorflow.js。他们构建了这个模型：

我将特征张量重塑为 4D： this.features = this.features.reshape([this.features.shape[0],this.features.shape[1],this.features.shape[2],1])对于 2Dconv。

  buildModel() {
        const inputShape1 = [this.features.shape[1], this.features.shape[2],this.features.shape[3]];
        this.model = tfNode.sequential();
        // filter to the image => feature extractor, edge detector, sharpener (depends on the models understanding)
        this.model.add(tfNode.layers.conv2d(
            {filters: 8, kernelSize: [4, 2], activation: 'relu', inputShape: inputShape1}
        ));

        // see the image at a higher level, generalize it more, prevent overfit
        this.model.add(tfNode.layers.maxPooling2d(
            {poolSize: [2, 2], strides: [2, 2]}
        ));

        // filter to the image => feature extractor, edge detector, sharpener (depends on the models understanding)
        const inputShape2 = [119,62,8];
        this.model.add(tfNode.layers.conv2d(
            {filters: 32, kernelSize: [4, 2], activation: 'relu', inputShape: inputShape2}
        ));

        // see the image at a higher level, generalize it more, prevent overfit
        this.model.add(tfNode.layers.maxPooling2d(
            {poolSize: [2, 2], strides: [2, 2]}
        ));

        // filter to the image => feature extractor, edge detector, sharpener (depends on the models understanding)
        const inputShape3 = [58,30,32];
        this.model.add(tfNode.layers.conv2d(
            {filters: 32, kernelSize: [4, 2], activation: 'relu', inputShape: inputShape3}
        ));

        // see the image at a higher level, generalize it more, prevent overfit
        this.model.add(tfNode.layers.maxPooling2d(
            {poolSize: [2, 2], strides: [2, 2]}
        ));

        // 1D output, => final output score of labels
        this.model.add(tfNode.layers.flatten({}));

        // prevents overfitting, randomly set 0
        this.model.add(tfNode.layers.dropout({rate: 0.25}));

        // learn anything linear, non linear comb. from conv. and soft pool
        this.model.add(tfNode.layers.dense({units: 2000, activation: 'relu'}));

        this.model.add(tfNode.layers.dropout({rate: 0.25}));

        // give probability for each label
        this.model.add(tfNode.layers.dense({units: this.labels.shape[1], activation: 'softmax'}));

        this.model.summary();

        // compile the model
        this.model.compile({loss: 'meanSquaredError', optimizer: 'adam'});
        this.model.summary()
    };

模型摘要：

_________________________________________________________________
Layer (type)                 Output shape              Param #   
=================================================================
conv2d_Conv2D1 (Conv2D)      [null,238,124,8]          72        
_________________________________________________________________
max_pooling2d_MaxPooling2D1  [null,119,62,8]           0         
_________________________________________________________________
conv2d_Conv2D2 (Conv2D)      [null,116,61,32]          2080      
_________________________________________________________________
max_pooling2d_MaxPooling2D2  [null,58,30,32]           0         
_________________________________________________________________
conv2d_Conv2D3 (Conv2D)      [null,55,29,32]           8224      
_________________________________________________________________
max_pooling2d_MaxPooling2D3  [null,27,14,32]           0         
_________________________________________________________________
flatten_Flatten1 (Flatten)   [null,12096]              0         
_________________________________________________________________
dropout_Dropout1 (Dropout)   [null,12096]              0         
_________________________________________________________________
dense_Dense1 (Dense)         [null,2000]               24194000  
_________________________________________________________________
dropout_Dropout2 (Dropout)   [null,2000]               0         
_________________________________________________________________
dense_Dense2 (Dense)         [null,2]                  4002      
=================================================================
Total params: 24208378
Trainable params: 24208378
Non-trainable params: 0
_________________________________________________________________
    Epoch 1 / 10
eta=0.0 ======================================>----------------------------------------------------------------------------- loss=0.515 0.51476
eta=0.8 ============================================================================>--------------------------------------- loss=0.442 0.44186
eta=0.0 ===================================================================================================================> 
3449ms 32236us/step - loss=0.485 val_loss=0.958 
Epoch 2 / 10
eta=0.0 ======================================>----------------------------------------------------------------------------- loss=0.422 0.42188
eta=0.9 ============================================================================>--------------------------------------- loss=0.395 0.39535
eta=0.0 ===================================================================================================================> 
3643ms 34043us/step - loss=0.411 val_loss=0.958 
Epoch 3 / 10

1) 第一个输入尺寸是我的特征张量形状。另外两个 inputShapes (inputShape2, inputShape3) 由我收到的错误消息定义。如何预先确定以下两种输入尺寸？

Answer 1

inputShape是如何计算的？

计算出来的不是inputShape。它是传递给必须匹配 inputShape 的模型的数据集。在定义模型时，inputShape 是 3D 的。但是查看模型摘要，还有一个值为 null 的第四个维度，即 batchshape。因此，训练数据应该是 4D 的。第一个维度或 batchshape 可以是任何东西——重要的是特征和标签具有相同的 batchshape。有更详细的回答here

图层形状是如何计算的？

这取决于使用的图层。 dropout、activation 等层不会更改输入形状。

根据步长内核，卷积层会改变输入形状。详细说明了它是如何计算的。
展平层将简单地将 inputShape 重塑为一维。在模型摘要中，有输入形状 [null,27,14,32]，展平层的形状为 [null, 12096] (12096 = 27 * 14 *32)
密集层也会改变输入形状。密集层的形状取决于该层的单元数。

TensorflowJs conv2d - 张量形状

TensorflowJs conv2d - Tensor Shapes

tensorflow

machine-learning

tensorflow.js

模型摘要：