使用扩张卷积在语义分割中进行上采样

Question

我正在研究一个使用扩张（atrous）卷积网络进行语义分割的项目。我正在使用咖啡框架。我的输入数据和标签大小是：

data (1 3 1158 1544 )
label (1 1 1158 1544)

我正在使用带损失的 softmax 进行评估。

网络工作正常，直到 Softmax 层要求输入的 blob 应该具有相同的维度。通常在这个网络中，数据的大小会变小，我需要在将它提供给 Softmax 层之前调整它的大小。

我需要一些想法如何在将数据提供给 softmax 层之前调整数据的大小。我知道我可以将反卷积与双线性插值结合使用并进行一些裁剪，但我错过了如何进行的想法。

任何帮助将不胜感激。

日志的最后一部分如下：

I0413 11:44:45.734990 18455 net.cpp:84] Creating Layer ct_final
I0413 11:44:45.734992 18455 net.cpp:406] ct_final <- ct_fc1
I0413 11:44:45.734997 18455 net.cpp:380] ct_final -> score_fc1
I0413 11:44:45.736572 18455 net.cpp:122] Setting up ct_final
I0413 11:44:45.736582 18455 net.cpp:129] Top shape: 1 32 33 81 (85536)
I0413 11:44:45.736583 18455 net.cpp:137] Memory required for data: 5731224640
I0413 11:44:45.736588 18455 layer_factory.hpp:77] Creating layer deconv1_bilinear
I0413 11:44:45.736598 18455 net.cpp:84] Creating Layer deconv1_bilinear
I0413 11:44:45.736599 18455 net.cpp:406] deconv1_bilinear <- score_fc1
I0413 11:44:45.736604 18455 net.cpp:380] deconv1_bilinear -> score_deconv1
I0413 11:44:45.740128 18455 net.cpp:122] Setting up deconv1_bilinear
I0413 11:44:45.740137 18455 net.cpp:129] Top shape: 1 32 136 328 (1427456)
I0413 11:44:45.740139 18455 net.cpp:137] Memory required for data: 5736934464
I0413 11:44:45.740144 18455 layer_factory.hpp:77] Creating layer deconv2_bilinear
I0413 11:44:45.740151 18455 net.cpp:84] Creating Layer deconv2_bilinear
I0413 11:44:45.740154 18455 net.cpp:406] deconv2_bilinear <- score_deconv1
I0413 11:44:45.740160 18455 net.cpp:380] deconv2_bilinear -> score
I0413 11:44:45.743669 18455 net.cpp:122] Setting up deconv2_bilinear
I0413 11:44:45.743695 18455 net.cpp:129] Top shape: 1 32 548 1316 (23077376)
I0413 11:44:45.743697 18455 net.cpp:137] Memory required for data: 5829243968
I0413 11:44:45.743700 18455 layer_factory.hpp:77] Creating layer loss
I0413 11:44:45.743721 18455 net.cpp:84] Creating Layer loss
I0413 11:44:45.743723 18455 net.cpp:406] loss <- score
I0413 11:44:45.743727 18455 net.cpp:406] loss <- label
I0413 11:44:45.743731 18455 net.cpp:380] loss -> loss
I0413 11:44:45.743762 18455 layer_factory.hpp:77] Creating layer loss
F0413 11:44:45.778823 18455 softmax_loss_layer.cpp:47] Check failed:outer_num_ * inner_num_ == bottom[1]->count() (721168 vs. 1787952) Number of labels must match number of predictions; e.g., if softmax axis == 1 and prediction shape is (N, C, H, W), label count (number of labels) must be N*H*W, with integer values in {0, 1, ..., C-1}.
*** Check failure stack trace: ***
    @     0x7f4ecd1215cd  google::LogMessage::Fail()
    @     0x7f4ecd123433  google::LogMessage::SendToLog()
    @     0x7f4ecd12115b  google::LogMessage::Flush()
    @     0x7f4ecd123e1e  google::LogMessageFatal::~LogMessageFatal()
    @     0x7f4ecd7bceb8  caffe::SoftmaxWithLossLayer<>::Reshape()
    @     0x7f4ecd7b10d7  caffe::Net<>::Init()
    @     0x7f4ecd7b380e  caffe::Net<>::Net()
    @     0x7f4ecd7414da  caffe::Solver<>::InitTrainNet()
    @     0x7f4ecd7429a5  caffe::Solver<>::Init()
    @     0x7f4ecd742cbf  caffe::Solver<>::Solver()
    @     0x7f4ecd786f11  caffe::Creator_AdamSolver<>()
    @           0x40a7d8  train()
    @           0x407568  main
    @     0x7f4ecb8b7830  __libc_start_main
    @           0x407e39  _start
    @              (nil)  (unknown)

如果你们需要 train.prototxt 请告诉我。

谢谢！！

Answer 1

反卷积确实是在卷积步骤之后恢复到原始输入大小的常用方法：这就是语义分割神经网络的基本架构：全卷积网络 (FCN)。

如果您需要有关如何使用它的示例，请查看此 repository: it contains many examples of FCNs in Caffe, and they use Deconvolution Caffe layers. In addition, you can read the related paper 以获得对反卷积的更深入解释。

请注意，Deconvolution 并不总是双线性插值。它通常通过插值进行初始化，但随后在训练（反向传播）期间学习。

使用扩张卷积在语义分割中进行上采样

Upsampling in Semantic Segmentation using dilated Convolution

c++

caffe

semantic-segmentation