如何使用 caffe 从自己的数据集生成 coorectlly lmdb 文件？

Question

我正在尝试生成 lmdbs 以使用 Caffe 训练我的 CNN。

我使用 Stanford Dogs 作为数据集 Dataset.After 下载数据集，我准备了训练数据集和测试数据 set.I 已将配置保存到文件中 train.txt和 val.txt.

然后，我尝试在我的控制台上通过以下两个命令生成 lmdb 文件：

GLOG_logtostderr=1 /home/bme/caffe/build/tools/convert_imageset \
    --resize_height=200 --resize_width=200 --shuffle  \
    /home/bme/caffe/data/dogs/data/ \
    /home/bme/caffe/data/dogs/dogs_data/train.txt \
    /home/bme/caffe/examples/dogs/dogs_train_lmdb

GLOG_logtostderr=1 /home/bme/caffe/build/tools/convert_imageset \
    --resize_height=200 --resize_width=200 --shuffle  \
    /home/bme/caffe/data/dogs/data/ \
    /home/bme/caffe/data/dogs/dogs_data/val.txt \
    /home/bme/caffe/examples/dogs/dogs_val_lmdb

但是，它为第一个命令产生了这些错误：

I0331 18:13:39.625823  8810 convert_imageset.cpp:86] Shuffling data
I0331 18:13:39.627516  8810 convert_imageset.cpp:89] A total of 17493 images.
I0331 18:13:39.627708  8810 db_lmdb.cpp:35] Opened lmdb /home/bme/caffe/examples/dogs/dogs_train_lmdb
E0331 18:13:39.627734  8810 io.cpp:80] Could not open or find file /home/bme/caffe/data/dogs/data/108_n02111889-Samoyed17.jpg 108   
E0331 18:13:39.627743  8810 io.cpp:80] Could not open or find file /home/bme/caffe/data/dogs/data/12_n02104365-schipperke145.jpg 12   
E0331 18:13:39.627753  8810 io.cpp:80] Could not open or find file /home/bme/caffe/data/dogs/data/17_n02113712-miniature_poodle107.jpg 17   
E0331 18:13:39.627760  8810 io.cpp:80] Could not open or find file /home/bme/caffe/data/dogs/data/12_n02104365-schipperke63.jpg 12   
E0331 18:13:39.627768  8810 io.cpp:80] Could not open or find file /home/bme/caffe/data/dogs/data/97_n02105056-groenendael102.jpg 97   
E0331 18:13:39.627774  8810 io.cpp:80] Could not open or find file /home/bme/caffe/data/dogs/data/74_n02096585-Boston_bull38.jpg 74   
E0331 18:13:39.627781  8810 io.cpp:80] Could not open or find file /home/bme/caffe/data/dogs/data/4_n02113799-standard_poodle145.jpg 4   
E0331 18:13:39.627789  8810 io.cpp:80] Could not open or find file /home/bme/caffe/data/dogs/data/22_n02097047-miniature_schnauzer27.jpg 22   
E0331 18:13:39.627797  8810 io.cpp:80] Could not open or find file /home/bme/caffe/data/dogs/data/80_n02107312-miniature_pinscher167.jpg 80   
E0331 18:13:39.627805  8810 io.cpp:80] Could not open or find file /home/bme/caffe/data/dogs/data/95_n02091134-whippet42.jpg 95

第二个命令的错误是：

I0331 18:13:39.625823  8810 convert_imageset.cpp:86] Shuffling data
I0331 18:13:39.627516  8810 convert_imageset.cpp:89] A total of 17493 images.
I0331 18:13:39.627708  8810 db_lmdb.cpp:35] Opened lmdb /home/bme/caffe/examples/dogs/dogs_train_lmdb
E0331 18:13:39.627734  8810 io.cpp:80] Could not open or find file /home/bme/caffe/data/dogs/data/108_n02111889-Samoyed17.jpg 108   
E0331 18:13:39.627743  8810 io.cpp:80] Could not open or find file /home/bme/caffe/data/dogs/data/12_n02104365-schipperke145.jpg 12   
E0331 18:13:39.627753  8810 io.cpp:80] Could not open or find file /home/bme/caffe/data/dogs/data/17_n02113712-miniature_poodle107.jpg 17   
E0331 18:13:39.627760  8810 io.cpp:80] Could not open or find file /home/bme/caffe/data/dogs/data/12_n02104365-schipperke63.jpg 12   
E0331 18:13:39.627768  8810 io.cpp:80] Could not open or find file /home/bme/caffe/data/dogs/data/97_n02105056-groenendael102.jpg 97   
E0331 18:13:39.627774  8810 io.cpp:80] Could not open or find file /home/bme/caffe/data/dogs/data/74_n02096585-Boston_bull38.jpg 74   
E0331 18:13:39.627781  8810 io.cpp:80] Could not open or find file /home/bme/caffe/data/dogs/data/4_n02113799-standard_poodle145.jpg 4   
E0331 18:13:39.627789  8810 io.cpp:80] Could not open or find file /home/bme/caffe/data/dogs/data/22_n02097047-miniature_schnauzer27.jpg 22   
E0331 18:13:39.627797  8810 io.cpp:80] Could not open or find file /home/bme/caffe/data/dogs/data/80_n02107312-miniature_pinscher167.jpg 80   
E0331 18:13:39.627805  8810 io.cpp:80] Could not open or find file /home/bme/caffe/data/dogs/data/95_n02091134-whippet42.jpg 95   
E0331 18:13:39.627812  8810 io.cpp:80] Could not open or find file /home/bme/caffe/data/dogs/data/56_n02101556-clumber86.jpg 56   
E0331 18:13:39.627820  8810 io.cpp:80] Could not open or find file /home/bme/caffe/data/dogs/data/26_n02085782-Japanese_spaniel36.jpg 26

我试图通过更改文件 train.txt 和 val.txt 的权限来解决问题。但同样的错误仍然存在。所以，任何解决这个问题的建议。

谢谢

Answer 1

您发布的输出都是针对训练集生成的（它们是相同的）。似乎在 17,493 张图像中有 12 个损坏的文件是 caffe 无法处理的 find/open.
检查这些文件是否存在，以及它们是否未损坏。

如何使用 caffe 从自己的数据集生成 coorectlly lmdb 文件？

How to generate coorectlly lmdb files from own dataset using caffe ?

machine-learning

ubuntu-14.04

deep-learning

caffe

lmdb