未能找到 HDF5 数据集数据

Question

我对咖啡和深度学习还很陌生。我想使用从 here 下载的数据集来训练我的模型。

我的火车数据是 hdf5 格式。它还具有以下参数。

{
    "debug": false, 
    "git_revision": "60c477dae59f3d1378568e2ebea054a135683e2f", 
    "height": 128, 
    "no_train_mirrors": false, 
    "output_dir": "/tmp/parse27k_crops_64x128", 
    "output_mode": "hdf5", 
    "padding": 32, 
    "padding_mode": "edge", 
    "parse_path": "/fast_work/sudowe/parse27k", 
    "single_threaded": false, 
    "verbose": false, 
    "width": 64
}

我的火车模型中有以下数据层。

layer {
  name: "data"
  type: "HDF5Data"
  top: "data"
  top: "label"
  hdf5_data_param {
    source: "path_to_caffe/caffe/examples/hdf5_classification/data/train.txt"
    batch_size: 10
  }
   include {
    phase: TRAIN
  }
}

当我尝试训练 train.txt 文件中提到的 train.hdf5 数据时，我收到以下错误消息。

I1031 11:52:10.185920  8670 layer_factory.hpp:77] Creating layer data
I1031 11:52:10.185933  8670 net.cpp:100] Creating Layer data
I1031 11:52:10.185940  8670 net.cpp:408] data -> data
I1031 11:52:10.185957  8670 net.cpp:408] data -> label
I1031 11:52:10.185971  8670 hdf5_data_layer.cpp:79] Loading list of HDF5 filenames from: path_to_caffe/caffe/examples/hdf5_classification/data/train.txt
I1031 11:52:10.186003  8670 hdf5_data_layer.cpp:93] Number of HDF5 files: 2
F1031 11:52:10.186825  8670 hdf5.cpp:14] Check failed: H5LTfind_dataset(file_id, dataset_name_) Failed to find HDF5 dataset data
*** Check failure stack trace: ***
    @     0x7f231a6a1daa  (unknown)
    @     0x7f231a6a1ce4  (unknown)
    @     0x7f231a6a16e6  (unknown)
    @     0x7f231a6a4687  (unknown)
    @     0x7f231acca607  caffe::hdf5_load_nd_dataset_helper<>()
    @     0x7f231acc93d5  caffe::hdf5_load_nd_dataset<>()
    @     0x7f231ad5172e  caffe::HDF5DataLayer<>::LoadHDF5FileData()
    @     0x7f231ad50548  caffe::HDF5DataLayer<>::LayerSetUp()
    @     0x7f231acaf3ac  caffe::Net<>::Init()
    @     0x7f231acb0235  caffe::Net<>::Net()
    @     0x7f231ae0332a  caffe::Solver<>::InitTrainNet()
    @     0x7f231ae0442c  caffe::Solver<>::Init()
    @     0x7f231ae0475a  caffe::Solver<>::Solver()
    @     0x7f231adf8453  caffe::Creator_SGDSolver<>()
    @           0x40f0fe  caffe::SolverRegistry<>::CreateSolver()
    @           0x408134  train()
    @           0x405b3c  main
    @     0x7f23196adf45  (unknown)
    @           0x4063ab  (unknown)
    @              (nil)  (unknown)

我们将不胜感激任何形式的帮助或建议。

Answer 1

在 caffe 输入数据层输出 blob 只能以 .hdf5 文件中的数据集名称命名。我的数据集具有以下结构

crops       Dataset {27482, 3, 128, 192}
labels      Dataset {27482, 12}
mean        Dataset {3, 128, 192}
pids        Dataset {27482}

在@Shai 的帮助下我是这样解决的:

layer {
  name: "data"
  type: "HDF5Data"
  top: "crops"
  top: "labels"
  include {
    phase: TRAIN
  }
  hdf5_data_param {
    source: "path_to_caffe/examples/hdf5_classification/data/train.txt"
    batch_size: 64
  }
}

未能找到 HDF5 数据集数据

Failed to find HDF5 dataset data

hdf5

deep-learning

hdf

caffe