caffe

Question

我想训练一个网络来识别图像中的一些 RGB。（输入：256X256 图像和一些 RGB 值）。

我写了一个脚本来为浮动多标签创建 HDF5 文件：

import h5py, os
import caffe
import numpy as np

SIZE = 256 # images size
with open( '/home/path/images ):
    lines = T.readlines()

X = np.zeros( (len(lines), SIZE, SIZE, 3), dtype='f4' )
r = np.zeros( (len(lines),1), dtype='f4' )
g = np.zeros( (len(lines),1), dtype='f4' )
b = np.zeros( (len(lines),1), dtype='f4' )

for i,l in enumerate(lines):
    sp = l.split(' ')
    img = caffe.io.load_image( sp[0] )
#    img = caffe.io.resize( img, (3, SIZE, SIZE) ) # resize to fixed $
    print img
    X[i] = img
#    print X[i]
    r[i] = float(sp[1])
    g[i] = float(sp[2])
    b[i] = float(sp[3])
    print "R" + str(r[i]) + "G" + str(g[i]) + "B" + str(b[i])
with h5py.File('/home/path/train.h5','$
    H.create_dataset('X', data=X)
    H.create_dataset('r', data=r)
    H.create_dataset('g', data=g)
    H.create_dataset('b', data=b) 

with open('/home/path/train_h5_list.tx$
    L.write( 'train.h5' ) # list all h5 files

我正在使用多标签回归网络。当我运行使用我的数据集 (HDF5) 在这个网络上训练时，我得到了这个错误：

name: "FKPReg"
state {
  phase: TRAIN
}
layer {
  name: "fkp"
  type: "HDF5Data"
  top: "data"
  top: "label"
  include {
    phase: TRAIN
  }
  hdf5_data_param {
    source: "/home/path/train_h5_list.txt"
    batch_size: 64
  }
}
layer {
  name: "conv1"
  type: "Convolution"
  bottom: "data"
  top: "conv1"
  convolution_param {
    num_output: 32
    kernel_size: 11
    stride: 2
    bias_filler {
      type: "constant"
      value: 0.1
    }
  }
}
layer {
  name: "relu2"
  type: "ReLU"
  bottom: "conv1"
  top: "conv1"
}
layer {
  name: "pool1"
  type: "Pooling"
  bottom: "conv1"
  top: "pool1"
  pooling_param {
    pool: MAX
    kernel_size: 2
    stride: 2
  }
}
layer {
  name: "conv2"
  type: "Convolution"
  bottom: "pool1"
  top: "conv2"
  convolution_param {
    num_output: 64
    pad: 2
    kernel_size: 7
    group: 2
    bias_filler {
      type: "constant"
      value: 0.1
    }
  }
}
layer {
  name: "relu2"
  type: "ReLU"
  bottom: "conv2"
  top: "conv2"
}
layer {
  name: "pool2"
  type: "Pooling"
  bottom: "conv2"
  top: "pool2"
  pooling_param {
    pool: MAX
    kernel_size: 2
    stride: 2
  }
}
layer {
  name: "norm2"
  type: "LRN"
  bottom: "pool2"
  top: "norm2"
  lrn_param {
    local_size: 3
    alpha: 5e-05
    beta: 0.75
    norm_region: WITHIN_CHANNEL
  }
}
layer {
  name: "conv3"
  type: "Convolution"
  bottom: "norm2"
  top: "conv3"
  convolution_param {
    num_output: 32
    pad: 1
    kernel_size: 5
    bias_filler {
      type: "constant"
      value: 0.1
    }
  }
}
layer {
  name: "relu3"
  type: "ReLU"
  bottom: "conv3"
  top: "conv3"
}
layer {
  name: "conv4"
  type: "Convolution"
  bottom: "conv3"
  top: "conv4"
  convolution_param {
    num_output: 64
    pad: 1
    kernel_size: 5
    bias_filler {
      type: "constant"
      value: 0.1
    }
  }
}
layer {
  name: "relu4"
  type: "ReLU"
  bottom: "conv4"
  top: "conv4"
}
layer {
  name: "conv5"
  type: "Convolution"
  bottom: "conv4"
  top: "conv5"
  convolution_param {
    num_output: 32
    pad: 1
    kernel_size: 5
    bias_filler {
      type: "constant"
      value: 0.1
    }
  }
}
layer {
  name: "relu5"
  type: "ReLU"
  bottom: "conv5"
  top: "conv5"
}
layer {
  name: "pool5"
  type: "Pooling"
  bottom: "conv5"
  top: "pool5"
  pooling_param {
    pool: MAX
    kernel_size: 4
    stride: 2
  }
}
layer {
  name: "drop0"
  type: "Dropout"
  bottom: "pool5"
  top: "pool5"
  dropout_param {
    dropout_ratio: 0.5
  }
}
layer {
  name: "ip1"
  type: "InnerProduct"
  bottom: "pool5"
  top: "ip1"
  inner_product_param {
    num_output: 100
    bias_filler {
      type: "constant"
      value: 0.1
    }
  }
}
layer {
  name: "relu4"
  type: "ReLU"
  bottom: "ip1"
  top: "ip1"
}
layer {
  name: "drop1"
  type: "Dropout"
  bottom: "ip1"
  top: "ip1"
  dropout_param {
    dropout_ratio: 0.5
  }
}
layer {
  name: "ip2"
  type: "InnerProduct"
  bottom: "ip1"
  top: "ip2"
  inner_product_param {
    num_output: 3
    bias_filler {
      type: "constant"
      value: 0.1
    }
  }
}
layer {
  name: "relu22"
  type: "ReLU"
  bottom: "ip2"
  top: "ip2"
}
layer {
  name: "loss"
  type: "EuclideanLoss"
  bottom: "ip2"
  bottom: "label"
  top: "loss"
}
I1106 11:47:52.235343 28083 layer_factory.hpp:74] Creating layer fkp
I1106 11:47:52.235384 28083 net.cpp:90] Creating Layer fkp
I1106 11:47:52.235410 28083 net.cpp:368] fkp -> data
I1106 11:47:52.235443 28083 net.cpp:368] fkp -> label
I1106 11:47:52.235481 28083 net.cpp:120] Setting up fkp
I1106 11:47:52.235496 28083 hdf5_data_layer.cpp:80] Loading list of HDF5 filenames from: /home/path/train_h5_list.txt
I1106 11:47:52.235568 28083 hdf5_data_layer.cpp:94] Number of HDF5 files: 1
HDF5-DIAG: Error detected in HDF5 (1.8.11) thread 140703305845312:
  #000: ../../../src/H5F.c line 1586 in H5Fopen(): unable to open file
    major: File accessibilty
    minor: Unable to open file
  #001: ../../../src/H5F.c line 1275 in H5F_open(): unable to open file: time = Sun Nov  6 11:47:52 2016
, name = 'train.h5', tent_flags = 0
    major: File accessibilty
    minor: Unable to open file
  #002: ../../../src/H5FD.c line 987 in H5FD_open(): open failed
    major: Virtual File Layer
    minor: Unable to initialize object
  #003: ../../../src/H5FDsec2.c line 343 in H5FD_sec2_open(): unable to open file: name = 'train.h5', errno = 2, error message = 'No such file or directory', flags = 0, o_flags = 0
    major: File accessibilty
    minor: Unable to open file
F1106 11:47:52.236398 28083 hdf5_data_layer.cpp:32] Failed opening HDF5 file: train.h5
*** Check failure stack trace: ***
    @     0x7ff809dfcdaa  (unknown)
    @     0x7ff809dfcce4  (unknown)
    @     0x7ff809dfc6e6  (unknown)
    @     0x7ff809dff687  (unknown)
    @     0x7ff80a194406  caffe::HDF5DataLayer<>::LoadHDF5FileData()
    @     0x7ff80a192c98  caffe::HDF5DataLayer<>::LayerSetUp()
    @     0x7ff80a173be3  caffe::Net<>::Init()
    @     0x7ff80a175952  caffe::Net<>::Net()
    @     0x7ff80a15bbf0  caffe::Solver<>::InitTrainNet()
    @     0x7ff80a15cbc3  caffe::Solver<>::Init()
    @     0x7ff80a15cd96  caffe::Solver<>::Solver()
    @           0x40c5d0  caffe::GetSolver<>()
    @           0x406611  train()
    @           0x404bb1  main
    @     0x7ff80930ef45  (unknown)
    @           0x40515d  (unknown)
    @              (nil)  (unknown)
Aborted (core dumped)

我做错了什么？谢谢

Answer 1

一些评论

caffe.io.resize( img, (3, SIZE, SIZE) ) - 这是错误的。
您需要将大小调整为 (SIZE, SIZE)，将 transpose 调整为 (3, SIZE, SIZE)。 resize 应该影响仅图像的空间维度，transpose 应该负责在高度和宽度之前安排 channel 维度。
因此，X 的 shape 应该是 (len(lines), 3, SIZE, SIZE)。
如果您的 HDF5 文件包含数据集 X、r、g 和 b，那么您的 "HDF5Data"层可以有"top"s"X"、"r"、"g"and/or"b"。您不能将 "data" 或 "label" 作为 "top"，因为在输入 hdf5 文件中没有这样的数据集。
您收到的错误消息指出（非常清楚）

error message = 'No such file or directory'

这通常意味着 train.h5 不在搜索路径中。
尝试将 完整路径 写入 /home/path/train_h5_list.txt.

caffe - 打开 HDF5 文件失败：train.h5

caffe - Failed opening HDF5 file: train.h5

python

hdf5

neural-network

deep-learning