为多标签分类创建lmdb

Question

我正在尝试创建 2 个 lmdb。一个用于我的图像，一个用于我的标签。我想确定图片的角度，为此我试图估计水平和垂直角度。我有类像：0-10 度水平 10-20 度水平等等。垂直角度也一样。现在我不知道如何创建标签数据库，因为标签必须如何在 lmdb 中格式化。我有一个 .txt 列表文件，其中包含： /path/pic.png 1 32 个条目，其中 1 表示 10-20 度，32 表示 320-330 度。我的代码如下所示：

for line in fileinput.input(data):
    entries = re.split(' ', line.strip())
    images.append(entries[0])
    labels.append(entries[1:])
.... 
for in_idx, (image, label) in enumerate(inputs):
    im = cv2.imread(image)
    im = im[:,:,::-1]
    im = im.transpose((2,0,1))

    im_dat = caffe.io.array_to_datum(im)
    print im_dat

    images_txn.put('{:0>10d}'.format(in_idx), im_dat.SerializeToString())

    label = np.array(label).astype(int).reshape(1, 1, len(label))

    label_dat = caffe.io.array_to_datum(label)
    labels_txn.put('{:0>10d}'.format(in_idx), label_dat.SerializeToString())

但这似乎不正确，因为我在尝试训练网络时遇到以下错误：

Check failed: outer_num_ * inner_num_ == bottom[1]->count() (10 vs. 20) Number of labels must match number of predictions; e.g., if softmax axis == 1 and prediction shape is (N, C, H, W), label count (number of labels) must be NHW, with integer values in {0, 1, ..., C-1}.

我的数据层如下所示：

name: "LearnAngle"
layer {
  name: "data"
  type: "Data"
  top: "images"
  include {
    phase: TRAIN
  }
  transform_param {
    mirror: true
    crop_size: 256
    mean_file: "/path/mean_train.binaryproto"
  }
  data_param {
    source: "path/train2_images_lmdb"
    batch_size: 10
    backend: LMDB
  }
}
layer {
  name: "data_label"
  type: "Data"
  top: "labels"
  include {
    phase: TRAIN
  }
  data_param {
    source: "path/train2_labels_lmdb"
    batch_size: 10
    backend: LMDB
  }
}

我最后一层看起来像

layer {
  name: "fc8"
  type: "InnerProduct"
  bottom: "fc7"
  top: "fc8"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  inner_product_param {
    num_output: 36
    weight_filler {
      type: "gaussian"
      std: 0.01
    }
    bias_filler {
      type: "constant"
      value: 0
    }
  }
}

layer {
  name: "loss"
  type: "SoftmaxWithLoss"
  bottom: "fc8"
  bottom: "labels"
  top: "loss"
}

Answer 1

问题是：你的 top: "labels" 在第 2 data 层包含 2 种水平和垂直角度的标签，而你只使用了 1 个 SoftmaxWithLoss 层作为标签.

事实上，要在一个网络中训练 2 个分类任务，您可以为 2 个任务的标签分别创建 2 lmdb 个数据库，并使用 2 data 层将它们解析为 2 SoftmaxWithLoss层。喜欢下面的内容：

为 2 任务分类创建 lmdb 的代码：

for line in fileinput.input(data):
    entries = re.split(' ', line.strip())
    images.append(entries[0])
    horizontal_labels.append(entries[1])
    vertical_labels.append(entries[2])

...
for in_idx, (image, label) in enumerate(inputs):
    im = cv2.imread(image)
    im = im[:,:,::-1]
    im = im.transpose((2,0,1))

    im_dat = caffe.io.array_to_datum(im)
    print im_dat

    images_txn.put('{:0>10d}'.format(in_idx), im_dat.SerializeToString())
    horizontal_label = [label[0]]
    vertical_label = [label[1]]

    horizontal_label = np.array(horizontal_label).astype(int).reshape(1, 1, 1)
    vertical_label = np.array(vertical_label).astype(int).reshape(1, 1, 1)

    horizontal_label_dat = caffe.io.array_to_datum(horizontal_label)
    vertical_label_dat = caffe.io.array_to_datum(vertical_label)

    horizontal_labels_txn.put('{:0>10d}'.format(in_idx), horizontal_label_dat.SerializeToString())
    vertical_labels_txn.put('{:0>10d}'.format(in_idx), vertical_label_dat.SerializeToString())

train_val.prototxt:

name: "LearnAngle"
layer {
  name: "data"
  type: "Data"
  top: "images"
  include {
    phase: TRAIN
  }
  ...
}
layer {
  name: "horizontal_label"
  type: "Data"
  top: "horizontal_label"
  include {
    phase: TRAIN
  }
  data_param {
    source: "path/horizontal_labels_lmdb" #created using above python code
    ...
  }
}
layer {
  name: "vertical_label"
  type: "Data"
  top: "vertical_label"
  include {
    phase: TRAIN
  }
  data_param {
    source: "path/vertical_labels_lmdb" #created using above python code
    ...
  }
}
... #follow layers
# branch for horizontal label classification
layer {
  name: "fc_horizon"
  type: "InnerProduct"
  bottom: "fc7"
  top: "fc_horizon"
  ...
  inner_product_param {
    num_output: 36
    ...
  }
}

layer {
  name: "loss_horizon"
  type: "SoftmaxWithLoss"
  bottom: "fc_horizon"
  bottom: "horizontal_label"
  top: "loss_horizon"
}
# branch for vertical label classification
layer {
  name: "fc_vertical"
  type: "InnerProduct"
  bottom: "fc7"
  top: "fc_vertical"
  ...
  inner_product_param {
    num_output: 36
    ...
  }
}

layer {
  name: "loss_vertical"
  type: "SoftmaxWithLoss"
  bottom: "fc_vertical"
  bottom: "vertical_label"
  top: "loss_vertical"
}

为多标签分类创建lmdb

Create lmdb for multi label classification

multilabel-classification

deep-learning

caffe