如何在 iOS 上将 caffemodel 与 OpenCV 结合使用？

Question

我正在尝试在 iOS 设备上使用 .caffemodel 和 OpenCV。我找到了这个 github repository，但它只能用 Xcode 6 构建。我正在使用 Xcode 7，但我也下载了 Xcode 6，但仍然没有成功建造它。

如何在 iOS 9 上使用带有 OpenCV 的 caffemodel？

PS：替代方案是 this，但它是用 swift 和金属编写的，我需要能够将它与 OpenCV 一起使用。

Answer 1

您可以使用 OpenCV DNN contrib module。

您需要先使用 contrib 模块构建 OpenCV，您可以找到步骤 here。

然后就可以导入使用.caffemodel下面的this tutorial了。

这是教程的更新版本，因为它不能按原样运行：

#include <opencv2/dnn.hpp>
#include <opencv2/imgproc.hpp>
#include <opencv2/highgui.hpp>
using namespace cv;
using namespace cv::dnn;
#include <fstream>
#include <iostream>
#include <cstdlib>
using namespace std;
/* Find best class for the blob (i. e. class with maximal probability) */
void getMaxClass(dnn::Blob &probBlob, int *classId, double *classProb)
{
    Mat probMat = probBlob.matRefConst().reshape(1, 1); //reshape the blob to 1x1000 matrix
    Point classNumber;
    minMaxLoc(probMat, NULL, classProb, NULL, &classNumber);
    *classId = classNumber.x;
}
std::vector<String> readClassNames(const char *filename = "synset_words.txt")
{
    std::vector<String> classNames;
    std::ifstream fp(filename);
    if (!fp.is_open())
    {
        std::cerr << "File with classes labels not found: " << filename << std::endl;
        exit(-1);
    }
    std::string name;
    while (!fp.eof())
    {
        std::getline(fp, name);
        if (name.length())
            classNames.push_back( name.substr(name.find(' ')+1) );
    }
    fp.close();
    return classNames;
}
int main(int argc, char **argv)
{
    cv::dnn::initModule();        

    String modelTxt = "bvlc_googlenet.prototxt";
    String modelBin = "bvlc_googlenet.caffemodel";
    String imageFile = (argc > 1) ? argv[1] : "space_shuttle.jpg";
    Ptr<dnn::Importer> importer;
    try                                     //Try to import Caffe GoogleNet model
    {
        importer = dnn::createCaffeImporter(modelTxt, modelBin);
    }
    catch (const cv::Exception &err)        //Importer can throw errors, we will catch them
    {
        std::cerr << err.msg << std::endl;
    }
    if (!importer)
    {
        std::cerr << "Can't load network by using the following files: " << std::endl;
        std::cerr << "prototxt:   " << modelTxt << std::endl;
        std::cerr << "caffemodel: " << modelBin << std::endl;
        std::cerr << "bvlc_googlenet.caffemodel can be downloaded here:" << std::endl;
        std::cerr << "http://dl.caffe.berkeleyvision.org/bvlc_googlenet.caffemodel" << std::endl;
        exit(-1);
    }
    dnn::Net net;
    importer->populateNet(net);
    importer.release();                     //We don't need importer anymore
    Mat img = imread(imageFile);
    if (img.empty())
    {
        std::cerr << "Can't read image from the file: " << imageFile << std::endl;
        exit(-1);
    }
    resize(img, img, Size(224, 224));                   //GoogLeNet accepts only 224x224 RGB-images
    dnn::Blob inputBlob = dnn::Blob(img);   //Convert Mat to dnn::Blob batch of images
    net.setBlob(".data", inputBlob);        //set the network input
    net.forward();                          //compute output
    dnn::Blob prob = net.getBlob("prob");   //gather output of "prob" layer
    int classId;
    double classProb;
    getMaxClass(prob, &classId, &classProb);//find the best class
    std::vector<String> classNames = readClassNames();
    std::cout << "Best class: #" << classId << " '" << classNames.at(classId) << "'" << std::endl;
    std::cout << "Probability: " << classProb * 100 << "%" << std::endl;
    return 0;
} //main

Answer 2

我会 post 另一个答案，因为最近的版本有一些不同。

首先，现在 dnn 已经在标准 OpenCV 库中，因此您不必从 contrib_modules.

构建它

加载网络的函数是readNetFromCaffe。

例如，以下代码加载 NN：

  std::string modelName = "path/to/mymodel.caffemodel";

  std::string protoName = "path/to/deploy.prototxt";
  cv::dnn::Net net;
  try
  {
    net = cv::dnn::readNetFromCaffe(protoName, modelName);
  }
  catch (cv::Exception& e)
  {
    std::cerr << "Exception: " << e.what() << std::endl;
    if (net.empty())
    {
      std::cerr << "Can't load network by using the following files: " << std::endl;
      std::cerr << "prototxt:   " << protoName << std::endl;
      std::cerr << "caffemodel: " << modelName << std::endl;
      std::cerr << "bvlc_googlenet.caffemodel can be downloaded here:" << std::endl;
      std::cerr << "http://dl.caffe.berkeleyvision.org/bvlc_googlenet.caffemodel" << std::endl;
      exit(-1);
    }
  }

然后你可以运行 NN:

  cv::Mat res_mat;
  float res;
  cv::Mat inputBlob = cv::dnn::blobFromImage(roi, 1.0f, cv::Size(227, 227),
                                  cv::Scalar(0, 0, 0), false);
  net.setInput(inputBlob);
  //During the forward pass output of each network layer is computed,
  //but in this example we need output from "prob" layer only.
  res_mat = net.forward("score");
  std::cout<<res_mat<<std::endl;
  res_mat = res_mat.reshape(1, 1); //reshape the blob to 1x2 matrix
  return res_mat.at<float>(0);

函数 cv::dnn::blobFromImage 将图像的大小调整为第三个参数指定的输入网络大小（在我的例子中是 cv::Size(227, 227)）。参数cv::Scalar(0, 0, 0)是三个BGR通道减去均值

score是我用的神经网络中输出层的名字。您可以在 prototxt 文件中看到此信息。

如何在 iOS 上将 caffemodel 与 OpenCV 结合使用？

How to use caffemodel with OpenCV on iOS?

c++

opencv

ios

caffe