如何使用 CImg 库捕获和处理图像的每一帧？
How do I capture and Process each and every frame of an image using CImg library?
我正在使用 Raspberrypi 中的 CImg 库进行基于实时图像处理的项目。
当我使用内置的 Raspicam 命令（例如
时，我需要以更高的帧速率（比如至少 30 fps）捕获图像
sudo raspistill -o -img_%d.jpg -tl 5 -t 1000 -a 512
/* -tl：以毫秒为单位的延时持续时间
-t：总持续时间（1000 毫秒 = 1 秒）
-a ：显示帧数
*/
使用此命令虽然它每秒显示 34 帧，但我最多只能捕获 4 frames/images（其余帧被跳过）
sudo raspistill -o -img_%d.jpg -tl 5 -tl 1000 -q 5 -md 7 -w 640 -h 480 -a 512
通过上面的命令，我每秒最多可以捕获 7-8 张图像，但会降低图像的分辨率和质量。
但我不想在图像质量上妥协，因为我将捕获图像，立即处理它并删除图像以节省内存。
后来尝试用V4L2(Video for Linux)驱动来发挥相机的最佳性能，但是网上关于V4l2和cimg的教程很少，没找到一.
我一直在使用以下命令
# Capture a JPEG image
v4l2-ctl --set-fmt-video=width=2592,height=1944,pixelformat=3
v4l2-ctl --stream-mmap=3 --stream-count=1 –stream-to=somefile.jpg
（来源：http://www.geeetech.com/wiki/index.php/Raspberry_Pi_Camera_Module）
但我无法获得有关这些参数的足够信息，例如 (stream-mmap & stream-count) 它到底是做什么的，以及这些命令如何帮助我每秒捕获 30 frames/images？
条件：
最重要的是我不想使用 OPENCV、MATLAB 或任何其他图像处理软件，因为我的图像处理任务非常简单（即检测 LED 灯闪烁）也是我的 objective是以更高的性能为代价拥有一个轻量级的工具来执行这些操作。
而且我的编程代码应该是 C 或 C++，而不是 python 或 Java（因为处理速度很重要！）
请注意，我的目的不是录制视频，而是捕捉尽可能多的帧并处理每张图片。
为了在 Cimg 中使用，我从参考手册中搜索了一些文档，但我无法清楚地理解如何将其用于我的目的。
class cimg_library::CImgList 表示 cimg_library::CImg 个图像列表。例如，它可用于存储图像序列的不同帧。
（来源：http://cimg.eu/reference/group__cimg__overview.html）
- 我找到了以下示例，但我不太确定它是否适合我的任务
从 YUV 图像序列文件加载列表。
CImg<T>& load_yuv
(
const char *const
filename,
const unsigned int
size_x,
const unsigned int
size_y,
const unsigned int
first_frame = 0,
const unsigned int
last_frame = ~0U,
const unsigned int
step_frame = 1,
const bool
yuv2rgb = true
参数
文件名
从中读取数据的文件名。
size_x
图片的宽度。
size_y
图片的高度。
first_frame
要读取的第一个图像帧的索引。
last_frame
要读取的最后一个图像帧的索引。
step_frame
在每帧之间应用的步骤。
yuv2rgb
在读取过程中应用 YUV 到 RGB 变换。
但是在这里，我需要直接来自图像帧的 rgb 值而不压缩。
现在我在 OpenCv 中有以下代码来执行我的任务，但我请求你帮助我使用 CImg 库（在 C++ 中）或任何其他轻量级库或 v4l2
#include <iostream>
#include <opencv2/opencv.hpp>
using namespace std;
using namespace cv;
int main (){
VideoCapture capture (0); //Since you have your device at /dev/video0
/* You can edit the capture properties with "capture.set (property, value);" or in the driver with "v4l2-ctl --set-ctrl=auto_exposure=1"*/
waitKey (200); //Wait 200 ms to ensure the device is open
Mat frame; // create Matrix where the new frame will be stored
if (capture.isOpened()){
while (true){
capture >> frame; //Put the new image in the Matrix
imshow ("Image", frame); //function to show the image in the screen
}
}
}
- 本人是编程初学者Raspberry pi，以上问题陈述如有错误请多多包涵
"With some of your recommendations, I slighthly modified the raspicam c++ api code and combined with CIMG image processing functionality "
#include "CImg.h"
#include <iostream>
#include <cstdlib>
#include <fstream>
#include <sstream>
#include <sys/timeb.h>
#include "raspicam.h"
using namespace std;
using namespace cimg_library;
bool doTestSpeedOnly=false;
size_t nFramesCaptured=100;
//parse command line
//returns the index of a command line param in argv. If not found, return -1
int findParam ( string param,int argc,char **argv ) {
int idx=-1;
for ( int i=0; i<argc && idx==-1; i++ )
if ( string ( argv[i] ) ==param ) idx=i;
return idx;
}
//parse command line
//returns the value of a command line param. If not found, defvalue is returned
float getParamVal ( string param,int argc,char **argv,float defvalue=-1 ) {
int idx=-1;
for ( int i=0; i<argc && idx==-1; i++ )
if ( string ( argv[i] ) ==param ) idx=i;
if ( idx==-1 ) return defvalue;
else return atof ( argv[ idx+1] );
}
raspicam::RASPICAM_EXPOSURE getExposureFromString ( string str ) {
if ( str=="OFF" ) return raspicam::RASPICAM_EXPOSURE_OFF;
if ( str=="AUTO" ) return raspicam::RASPICAM_EXPOSURE_AUTO;
if ( str=="NIGHT" ) return raspicam::RASPICAM_EXPOSURE_NIGHT;
if ( str=="NIGHTPREVIEW" ) return raspicam::RASPICAM_EXPOSURE_NIGHTPREVIEW;
if ( str=="BACKLIGHT" ) return raspicam::RASPICAM_EXPOSURE_BACKLIGHT;
if ( str=="SPOTLIGHT" ) return raspicam::RASPICAM_EXPOSURE_SPOTLIGHT;
if ( str=="SPORTS" ) return raspicam::RASPICAM_EXPOSURE_SPORTS;
if ( str=="SNOW" ) return raspicam::RASPICAM_EXPOSURE_SNOW;
if ( str=="BEACH" ) return raspicam::RASPICAM_EXPOSURE_BEACH;
if ( str=="VERYLONG" ) return raspicam::RASPICAM_EXPOSURE_VERYLONG;
if ( str=="FIXEDFPS" ) return raspicam::RASPICAM_EXPOSURE_FIXEDFPS;
if ( str=="ANTISHAKE" ) return raspicam::RASPICAM_EXPOSURE_ANTISHAKE;
if ( str=="FIREWORKS" ) return raspicam::RASPICAM_EXPOSURE_FIREWORKS;
return raspicam::RASPICAM_EXPOSURE_AUTO;
}
raspicam::RASPICAM_AWB getAwbFromString ( string str ) {
if ( str=="OFF" ) return raspicam::RASPICAM_AWB_OFF;
if ( str=="AUTO" ) return raspicam::RASPICAM_AWB_AUTO;
if ( str=="SUNLIGHT" ) return raspicam::RASPICAM_AWB_SUNLIGHT;
if ( str=="CLOUDY" ) return raspicam::RASPICAM_AWB_CLOUDY;
if ( str=="SHADE" ) return raspicam::RASPICAM_AWB_SHADE;
if ( str=="TUNGSTEN" ) return raspicam::RASPICAM_AWB_TUNGSTEN;
if ( str=="FLUORESCENT" ) return raspicam::RASPICAM_AWB_FLUORESCENT;
if ( str=="INCANDESCENT" ) return raspicam::RASPICAM_AWB_INCANDESCENT;
if ( str=="FLASH" ) return raspicam::RASPICAM_AWB_FLASH;
if ( str=="HORIZON" ) return raspicam::RASPICAM_AWB_HORIZON;
return raspicam::RASPICAM_AWB_AUTO;
}
void processCommandLine ( int argc,char **argv,raspicam::RaspiCam &Camera ) {
Camera.setWidth ( getParamVal ( "-w",argc,argv,640 ) );
Camera.setHeight ( getParamVal ( "-h",argc,argv,480 ) );
Camera.setBrightness ( getParamVal ( "-br",argc,argv,50 ) );
Camera.setSharpness ( getParamVal ( "-sh",argc,argv,0 ) );
Camera.setContrast ( getParamVal ( "-co",argc,argv,0 ) );
Camera.setSaturation ( getParamVal ( "-sa",argc,argv,0 ) );
Camera.setShutterSpeed( getParamVal ( "-ss",argc,argv,0 ) );
Camera.setISO ( getParamVal ( "-iso",argc,argv ,400 ) );
if ( findParam ( "-vs",argc,argv ) !=-1 )
Camera.setVideoStabilization ( true );
Camera.setExposureCompensation ( getParamVal ( "-ec",argc,argv ,0 ) );
if ( findParam ( "-gr",argc,argv ) !=-1 )
Camera.setFormat(raspicam::RASPICAM_FORMAT_GRAY);
if ( findParam ( "-yuv",argc,argv ) !=-1 )
Camera.setFormat(raspicam::RASPICAM_FORMAT_YUV420);
if ( findParam ( "-test_speed",argc,argv ) !=-1 )
doTestSpeedOnly=true;
int idx;
if ( ( idx=findParam ( "-ex",argc,argv ) ) !=-1 )
Camera.setExposure ( getExposureFromString ( argv[idx+1] ) );
if ( ( idx=findParam ( "-awb",argc,argv ) ) !=-1 )
Camera.setAWB( getAwbFromString ( argv[idx+1] ) );
nFramesCaptured=getParamVal("-nframes",argc,argv,100);
Camera.setAWB_RB(getParamVal("-awb_b",argc,argv ,1), getParamVal("-awb_g",argc,argv ,1));
}
//timer functions
#include <sys/time.h>
#include <unistd.h>
class Timer{
private:
struct timeval _start, _end;
public:
Timer(){}
void start(){
gettimeofday(&_start, NULL);
}
void end(){
gettimeofday(&_end, NULL);
}
double getSecs(){
return double(((_end.tv_sec - _start.tv_sec) * 1000 + (_end.tv_usec - _start.tv_usec)/1000.0) + 0.5)/1000.;
}
};
void saveImage ( string filepath,unsigned char *data,raspicam::RaspiCam &Camera ) {
std::ofstream outFile ( filepath.c_str(),std::ios::binary );
if ( Camera.getFormat()==raspicam::RASPICAM_FORMAT_BGR || Camera.getFormat()==raspicam::RASPICAM_FORMAT_RGB ) {
outFile<<"P6\n";
} else if ( Camera.getFormat()==raspicam::RASPICAM_FORMAT_GRAY ) {
outFile<<"P5\n";
} else if ( Camera.getFormat()==raspicam::RASPICAM_FORMAT_YUV420 ) { //made up format
outFile<<"P7\n";
}
outFile<<Camera.getWidth() <<" "<<Camera.getHeight() <<" 255\n";
outFile.write ( ( char* ) data,Camera.getImageBufferSize() );
}
int main ( int argc,char **argv ) {
int a=1,b=0,c;
int x=444,y=129; //pixel coordinates
raspicam::RaspiCam Camera;
processCommandLine ( argc,argv,Camera );
cout<<"Connecting to camera"<<endl;
if ( !Camera.open() ) {
cerr<<"Error opening camera"<<endl;
return -1;
}
// cout<<"Connected to camera ="<<Camera.getId() <<" bufs="<<Camera.getImageBufferSize( )<<endl;
unsigned char *data=new unsigned char[ Camera.getImageBufferSize( )];
Timer timer;
// cout<<"Capturing...."<<endl;
// size_t i=0;
timer.start();
for (int i=0;i<=nFramesCaptured;i++)
{
Camera.grab();
Camera.retrieve ( data );
std::stringstream fn;
fn<<"image.jpg";
saveImage ( fn.str(),data,Camera );
// cerr<<"Saving "<<fn.str()<<endl;
CImg<float> Img("/run/shm/image.jpg");
//Img.display("Window Title");
// 9 PIXELS MATRIX GRAYSCALE VALUES
float pixvalR1 = Img(x-1,y-1);
float pixvalR2 = Img(x,y-1);
float pixvalR3 = Img(x+1,y-1);
float pixvalR4 = Img(x-1,y);
float pixvalR5 = Img(x,y);
float pixvalR6 = Img(x+1,y);
float pixvalR7 = Img(x-1,y+1);
float pixvalR8 = Img(x,y+1);
float pixvalR9 = Img(x+1,y+1);
// std::cout<<"coordinate value :"<<pixvalR5 << endl;
// MEAN VALUES OF RGB PIXELS
float light = (pixvalR1+pixvalR2+pixvalR3+pixvalR4+pixvalR5+pixvalR6+pixvalR7+pixvalR8+pixvalR9)/9 ;
// DISPLAYING MEAN RGB VALUES OF 9 PIXELS
// std::cout<<"Lightness value :"<<light << endl;
// THRESHOLDING CONDITION
c = (light > 130 ) ? a : b;
// cout<<"Data is " << c <<endl;
ofstream fout("c.txt", ios::app);
fout<<c;
fout.close();
}
timer.end();
cerr<< timer.getSecs()<< " seconds for "<< nFramesCaptured << " frames : FPS " << ( ( float ) ( nFramesCaptured ) / timer.getSecs() ) <<endl;
Camera.release();
std::cin.ignore();
}
- 从这段代码中，我想知道我们如何直接从 camera.retrieve(data) 中获取数据，而不将其存储为图像文件并从图像缓冲区访问数据，以进行处理图像并将其进一步删除。
根据 Mark Setchell 的建议，我对代码做了一些小的改动，我得到了很好的结果，但是，有没有什么方法可以提高处理性能以获得更高的帧率？使用此代码，我最多可以达到 10 FPS。
#include <ctime>
#include <fstream>
#include <iostream>
#include <thread>
#include <mutex>
#include <raspicam/raspicam.h>
// Don't want any X11 display by CImg
#define cimg_display 0
#include <CImg.h>
using namespace cimg_library;
using namespace std;
#define NFRAMES 1000
#define NTHREADS 2
#define WIDTH 640
#define HEIGHT 480
// Commands/status for the worker threads
#define WAIT 0
#define GO 1
#define GOING 2
#define EXIT 3
#define EXITED 4
volatile int command[NTHREADS];
// Serialize access to cout
std::mutex cout_mutex;
// CImg initialisation
// Create a 1280x960 greyscale (Y channel of YUV) image
// Create a globally-accessible CImg for main and workers to access
CImg<unsigned char> img(WIDTH,HEIGHT,1,1,128);
////////////////////////////////////////////////////////////////////////////////
// worker thread - There will 2 or more of these running in parallel with the
// main thread. Do any image processing in here.
////////////////////////////////////////////////////////////////////////////////
void worker (int id) {
// If you need a "results" image of type CImg, create it here before entering
// ... the main processing loop below - you don't want to do malloc()s in the
// ... high-speed loop
// CImg results...
int wakeups=0;
// Create a white for annotating
unsigned char white[] = { 255,255,255 };
while(true){
// Busy wait with 500us sleep - at worst we only miss 50us of processing time per frame
while((command[id]!=GO)&&(command[id]!=EXIT)){
std::this_thread::sleep_for(std::chrono::microseconds(500));
}
if(command[id]==EXIT){command[id]=EXITED;break;}
wakeups++;
// Process frame of data - access CImg structure here
command[id]=GOING;
// You need to add your processing in HERE - everything from
// ... 9 PIXELS MATRIX GRAYSCALE VALUES to
// ... THRESHOLDING CONDITION
int a=1,b=0,c;
int x=330,y=84;
// CImg<float> Img("/run/shm/result.png");
float pixvalR1 = img(x-1,y-1);
float pixvalR2 = img(x,y-1);
float pixvalR3 = img(x+1,y-1);
float pixvalR4 = img(x-1,y);
float pixvalR5 = img(x,y);
float pixvalR6 = img(x+1,y);
float pixvalR7 = img(x-1,y+1);
float pixvalR8 = img(x,y+1);
float pixvalR9 = img(x+1,y+1);
// MEAN VALUES OF RGB PIXELS
float light = (pixvalR1+pixvalR2+pixvalR3+pixvalR4+pixvalR5+pixvalR6+pixvalR7+pixvalR8+pixvalR9)/9 ;
// DISPLAYING MEAN RGB VALUES OF 9 PIXELS
// std::cout<<"Lightness value :"<<light << endl;
// THRESHOLDING CONDITION
c = (light > 130 ) ? a : b;
// cout<<"Data is " << c <<endl;
ofstream fout("c.txt", ios::app);
fout<<c;
fout.close();
// Pretend to do some processing.
// You need to delete the following "sleep_for" and "if(id==0...){...}"
// std::this_thread::sleep_for(std::chrono::milliseconds(2));
/* if((id==0)&&(wakeups==NFRAMES)){
// Annotate final image and save as PNG
img.draw_text(100,100,"Hello World",white);
img.save_png("result.png");
} */
}
cout_mutex.lock();
std::cout << "Thread[" << id << "]: Received " << wakeups << " wakeups" << std::endl;
cout_mutex.unlock();
}
//timer functions
#include <sys/time.h>
#include <unistd.h>
class Timer{
private:
struct timeval _start, _end;
public:
Timer(){}
void start(){
gettimeofday(&_start, NULL);
}
void end(){
gettimeofday(&_end, NULL);
}
double getSecs(){
return double(((_end.tv_sec - _start.tv_sec) * 1000 + (_end.tv_usec - _start.tv_usec)/1000.0) + 0.5)/1000.;
}
};
int main ( int argc,char **argv ) {
Timer timer;
raspicam::RaspiCam Camera;
// Allowable values: RASPICAM_FORMAT_GRAY,RASPICAM_FORMAT_RGB,RASPICAM_FORMAT_BGR,RASPICAM_FORMAT_YUV420
Camera.setFormat(raspicam::RASPICAM_FORMAT_YUV420);
// Allowable widths: 320, 640, 1280
// Allowable heights: 240, 480, 960
// setCaptureSize(width,height)
Camera.setCaptureSize(WIDTH,HEIGHT);
std::cout << "Main: Starting" << std::endl;
std::cout << "Main: NTHREADS:" << NTHREADS << std::endl;
std::cout << "Main: NFRAMES:" << NFRAMES << std::endl;
std::cout << "Main: Width: " << Camera.getWidth() << std::endl;
std::cout << "Main: Height: " << Camera.getHeight() << std::endl;
// Spawn worker threads - making sure they are initially in WAIT state
std::thread threads[NTHREADS];
for(int i=0; i<NTHREADS; ++i){
command[i]=WAIT;
threads[i] = std::thread(worker,i);
}
// Open camera
cout<<"Opening Camera..."<<endl;
if ( !Camera.open()) {cerr<<"Error opening camera"<<endl;return -1;}
// Wait until camera stabilizes
std::cout<<"Sleeping for 3 secs"<<endl;
std::this_thread::sleep_for(std::chrono::seconds(3));
timer.start();
for(int frame=0;frame<NFRAMES;frame++){
// Capture frame
Camera.grab();
// Copy just the Y component to our mono CImg
std::memcpy(img._data,Camera.getImageBufferData(),WIDTH*HEIGHT);
// Notify worker threads that data is ready for processing
for(int i=0; i<NTHREADS; ++i){
command[i]=GO;
}
}
timer.end();
cerr<< timer.getSecs()<< " seconds for "<< NFRAMES << " frames : FPS " << ( ( float ) ( NFRAMES ) / timer.getSecs() ) << endl;
// Let workers process final frame, then tell to exit
// std::this_thread::sleep_for(std::chrono::milliseconds(50));
// Notify worker threads to exit
for(int i=0; i<NTHREADS; ++i){
command[i]=EXIT;
}
// Wait for all threads to finish
for(auto& th : threads) th.join();
}
执行代码的编译命令：
g++ -std=c++11 /home/pi/raspicam/src/raspicimgthread.cpp -o threadraspicimg -I. -I/usr/local/include -L /opt/vc/lib -L /usr/local/lib -lraspicam -lmmal -lmmal_core -lmmal_util -O2 -L/usr/X11R6/lib -lm -lpthread -lX11
**RESULTS :**
Main: Starting
Main: NTHREADS:2
Main: NFRAMES:1000
Main: Width: 640
Main: Height: 480
Opening Camera...
Sleeping for 3 secs
99.9194 seconds for 1000 frames : FPS 10.0081
Thread[1]: Received 1000 wakeups
Thread[0]: Received 1000 wakeups
real 1m43.198s
user 0m2.060s
sys 0m5.850s
还有一个问题是，当我使用普通的 Raspicam c++ API 代码执行相同的任务（我之前提到的代码）时，我得到了几乎相同的结果，但在性能（当然我的帧速率从 9.4 FPS 增加到 10 FPS）。
但是在代码1中：
我一直在将图像保存在 ram 磁盘中进行处理，然后我正在删除。
我没有使用任何线程进行并行处理。
在代码2中：
我们没有在磁盘中保存任何图像，而是直接从缓冲区中处理它。而且我们也在使用线程来提高处理速度。
不幸的是，尽管我们对代码 1 的代码 2 进行了一些更改，但我无法获得所需的结果（将以 30 FPS 执行）
等待您提出好的建议，非常感谢您的帮助。
提前致谢
最好的问候
BLV Lohith Kumar
更新答案
我在这里更新了我的原始答案，以展示如何将获取的数据复制到 CImg 结构中，还展示了 2 个工作线程可以在主线程继续获取帧的同时处理图像全速。它达到每秒 60 帧。
我没有在工作线程内部做任何处理，因为我不知道你想做什么。我所做的只是将最后一帧保存到磁盘，以表明采集到 CImg 的工作正常。你可以有 3 个工作线程。您可以在 round-robin 的基础上将一帧传递给每个线程，或者您可以让 2 个线程中的每一个在每次迭代时处理一半的帧。或者 3 个线程中的每一个处理一帧的三分之一。您可以更改轮询唤醒以使用条件变量。
#include <ctime>
#include <fstream>
#include <iostream>
#include <thread>
#include <mutex>
#include <raspicam/raspicam.h>
// Don't want any X11 display by CImg
#define cimg_display 0
#include <CImg.h>
using namespace cimg_library;
using namespace std;
#define NFRAMES 1000
#define NTHREADS 2
#define WIDTH 1280
#define HEIGHT 960
// Commands/status for the worker threads
#define WAIT 0
#define GO 1
#define GOING 2
#define EXIT 3
#define EXITED 4
volatile int command[NTHREADS];
// Serialize access to cout
std::mutex cout_mutex;
// CImg initialisation
// Create a 1280x960 greyscale (Y channel of YUV) image
// Create a globally-accessible CImg for main and workers to access
CImg<unsigned char> img(WIDTH,HEIGHT,1,1,128);
////////////////////////////////////////////////////////////////////////////////
// worker thread - There will 2 or more of these running in parallel with the
// main thread. Do any image processing in here.
////////////////////////////////////////////////////////////////////////////////
void worker (int id) {
// If you need a "results" image of type CImg, create it here before entering
// ... the main processing loop below - you don't want to do malloc()s in the
// ... high-speed loop
// CImg results...
int wakeups=0;
// Create a white for annotating
unsigned char white[] = { 255,255,255 };
while(true){
// Busy wait with 500us sleep - at worst we only miss 50us of processing time per frame
while((command[id]!=GO)&&(command[id]!=EXIT)){
std::this_thread::sleep_for(std::chrono::microseconds(500));
}
if(command[id]==EXIT){command[id]=EXITED;break;}
wakeups++;
// Process frame of data - access CImg structure here
command[id]=GOING;
// You need to add your processing in HERE - everything from
// ... 9 PIXELS MATRIX GRAYSCALE VALUES to
// ... THRESHOLDING CONDITION
// Pretend to do some processing.
// You need to delete the following "sleep_for" and "if(id==0...){...}"
std::this_thread::sleep_for(std::chrono::milliseconds(2));
if((id==0)&&(wakeups==NFRAMES)){
// Annotate final image and save as PNG
img.draw_text(100,100,"Hello World",white);
img.save_png("result.png");
}
}
cout_mutex.lock();
std::cout << "Thread[" << id << "]: Received " << wakeups << " wakeups" << std::endl;
cout_mutex.unlock();
}
int main ( int argc,char **argv ) {
raspicam::RaspiCam Camera;
// Allowable values: RASPICAM_FORMAT_GRAY,RASPICAM_FORMAT_RGB,RASPICAM_FORMAT_BGR,RASPICAM_FORMAT_YUV420
Camera.setFormat(raspicam::RASPICAM_FORMAT_YUV420);
// Allowable widths: 320, 640, 1280
// Allowable heights: 240, 480, 960
// setCaptureSize(width,height)
Camera.setCaptureSize(WIDTH,HEIGHT);
std::cout << "Main: Starting" << std::endl;
std::cout << "Main: NTHREADS:" << NTHREADS << std::endl;
std::cout << "Main: NFRAMES:" << NFRAMES << std::endl;
std::cout << "Main: Width: " << Camera.getWidth() << std::endl;
std::cout << "Main: Height: " << Camera.getHeight() << std::endl;
// Spawn worker threads - making sure they are initially in WAIT state
std::thread threads[NTHREADS];
for(int i=0; i<NTHREADS; ++i){
command[i]=WAIT;
threads[i] = std::thread(worker,i);
}
// Open camera
cout<<"Opening Camera..."<<endl;
if ( !Camera.open()) {cerr<<"Error opening camera"<<endl;return -1;}
// Wait until camera stabilizes
std::cout<<"Sleeping for 3 secs"<<endl;
std::this_thread::sleep_for(std::chrono::seconds(3));
for(int frame=0;frame<NFRAMES;frame++){
// Capture frame
Camera.grab();
// Copy just the Y component to our mono CImg
std::memcpy(img._data,Camera.getImageBufferData(),WIDTH*HEIGHT);
// Notify worker threads that data is ready for processing
for(int i=0; i<NTHREADS; ++i){
command[i]=GO;
}
}
// Let workers process final frame, then tell to exit
std::this_thread::sleep_for(std::chrono::milliseconds(50));
// Notify worker threads to exit
for(int i=0; i<NTHREADS; ++i){
command[i]=EXIT;
}
// Wait for all threads to finish
for(auto& th : threads) th.join();
}
时间注意事项
你可以这样计时：
#include <chrono>
typedef std::chrono::high_resolution_clock hrclock;
hrclock::time_point t1,t2;
t1 = hrclock::now();
// do something that needs timing
t2 = hrclock::now();
std::chrono::nanoseconds elapsed = t2-t1;
long long nanoseconds=elapsed.count();
原答案
我一直在用 Raspicam 做一些实验。我从 SourceForge 下载了他们的代码并稍作修改以进行一些简单的 capture-only 测试。我最终使用的代码如下所示：
#include <ctime>
#include <fstream>
#include <iostream>
#include <raspicam/raspicam.h>
#include <unistd.h> // for usleep()
using namespace std;
#define NFRAMES 1000
int main ( int argc,char **argv ) {
raspicam::RaspiCam Camera;
// Allowable values: RASPICAM_FORMAT_GRAY,RASPICAM_FORMAT_RGB,RASPICAM_FORMAT_BGR,RASPICAM_FORMAT_YUV420
Camera.setFormat(raspicam::RASPICAM_FORMAT_YUV420);
// Allowable widths: 320, 640, 1280
// Allowable heights: 240, 480, 960
// setCaptureSize(width,height)
Camera.setCaptureSize(1280,960);
// Open camera
cout<<"Opening Camera..."<<endl;
if ( !Camera.open()) {cerr<<"Error opening camera"<<endl;return -1;}
// Wait until camera stabilizes
cout<<"Sleeping for 3 secs"<<endl;
usleep(3000000);
cout << "Grabbing " << NFRAMES << " frames" << endl;
// Allocate memory
unsigned long bytes=Camera.getImageBufferSize();
cout << "Width: " << Camera.getWidth() << endl;
cout << "Height: " << Camera.getHeight() << endl;
cout << "ImageBufferSize: " << bytes << endl;;
unsigned char *data=new unsigned char[bytes];
for(int frame=0;frame<NFRAMES;frame++){
// Capture frame
Camera.grab();
// Extract the image
Camera.retrieve ( data,raspicam::RASPICAM_FORMAT_IGNORE );
// Wake up a thread here to process the frame with CImg
}
return 0;
}
我不喜欢cmake所以我这样编译：
g++ -std=c++11 simpletest.c -o simpletest -I. -I/usr/local/include -L /opt/vc/lib -L /usr/local/lib -lraspicam -lmmal -lmmal_core -lmmal_util
我发现，无论图像的尺寸如何，或多或少与编码（RGB、BGR、GRAY）无关，它都能达到 30 fps（每秒帧数）。
唯一能让我变得更好的方法是进行以下更改：
在上面的代码中，使用RASPICAM_FORMAT_YUV420而不是其他任何东西
编辑文件 private_impl.cpp 并更改第 71 行以将帧速率设置为 90。
如果我这样做，我可以达到 66 fps。
由于 Raspberry Pi 只有 900MHz CPU 但是有 4 个核心，我猜你会想在循环外的开始处启动 1-3 个额外的线程然后唤醒一个或多个我在处理数据的代码中注明的地方。他们要做的第一件事是在下一帧开始之前将数据从采集缓冲区中复制出来——或者有多个缓冲区并以 round-robin 方式使用它们。
线程注意事项
下图中，绿色代表获取图像的Camera.grab()，红色代表获取图像后进行的处理。目前，您正在获取数据（绿色），然后在获取下一帧之前对其进行处理（红色）。请注意，您的 4 个 CPU 中有 3 个什么都不做。
我的建议是将处理（红色）卸载给另一个 CPUs/threads 并尽可能快地获取新数据（绿色）。像这样：
现在您看到每秒获得更多帧（绿色）。
我正在使用 Raspberrypi 中的 CImg 库进行基于实时图像处理的项目。
当我使用内置的 Raspicam 命令（例如
时，我需要以更高的帧速率（比如至少 30 fps）捕获图像sudo raspistill -o -img_%d.jpg -tl 5 -t 1000 -a 512
/* -tl：以毫秒为单位的延时持续时间 -t：总持续时间（1000 毫秒 = 1 秒） -a ：显示帧数 */
使用此命令虽然它每秒显示 34 帧，但我最多只能捕获 4 frames/images（其余帧被跳过）
sudo raspistill -o -img_%d.jpg -tl 5 -tl 1000 -q 5 -md 7 -w 640 -h 480 -a 512
通过上面的命令，我每秒最多可以捕获 7-8 张图像，但会降低图像的分辨率和质量。
但我不想在图像质量上妥协，因为我将捕获图像，立即处理它并删除图像以节省内存。
后来尝试用V4L2(Video for Linux)驱动来发挥相机的最佳性能，但是网上关于V4l2和cimg的教程很少，没找到一.
我一直在使用以下命令
# Capture a JPEG image
v4l2-ctl --set-fmt-video=width=2592,height=1944,pixelformat=3
v4l2-ctl --stream-mmap=3 --stream-count=1 –stream-to=somefile.jpg
（来源：http://www.geeetech.com/wiki/index.php/Raspberry_Pi_Camera_Module）
但我无法获得有关这些参数的足够信息，例如 (stream-mmap & stream-count) 它到底是做什么的，以及这些命令如何帮助我每秒捕获 30 frames/images？
条件：
最重要的是我不想使用 OPENCV、MATLAB 或任何其他图像处理软件，因为我的图像处理任务非常简单（即检测 LED 灯闪烁）也是我的 objective是以更高的性能为代价拥有一个轻量级的工具来执行这些操作。
而且我的编程代码应该是 C 或 C++，而不是 python 或 Java（因为处理速度很重要！）
请注意，我的目的不是录制视频，而是捕捉尽可能多的帧并处理每张图片。
为了在 Cimg 中使用，我从参考手册中搜索了一些文档，但我无法清楚地理解如何将其用于我的目的。
class cimg_library::CImgList 表示 cimg_library::CImg 个图像列表。例如，它可用于存储图像序列的不同帧。 （来源：http://cimg.eu/reference/group__cimg__overview.html）
- 我找到了以下示例，但我不太确定它是否适合我的任务
从 YUV 图像序列文件加载列表。
CImg<T>& load_yuv
(
const char *const
filename,
const unsigned int
size_x,
const unsigned int
size_y,
const unsigned int
first_frame = 0,
const unsigned int
last_frame = ~0U,
const unsigned int
step_frame = 1,
const bool
yuv2rgb = true
参数 文件名 从中读取数据的文件名。 size_x 图片的宽度。 size_y 图片的高度。 first_frame 要读取的第一个图像帧的索引。 last_frame 要读取的最后一个图像帧的索引。 step_frame 在每帧之间应用的步骤。 yuv2rgb 在读取过程中应用 YUV 到 RGB 变换。
但是在这里，我需要直接来自图像帧的 rgb 值而不压缩。
现在我在 OpenCv 中有以下代码来执行我的任务，但我请求你帮助我使用 CImg 库（在 C++ 中）或任何其他轻量级库或 v4l2
#include <iostream>
#include <opencv2/opencv.hpp>
using namespace std;
using namespace cv;
int main (){
VideoCapture capture (0); //Since you have your device at /dev/video0
/* You can edit the capture properties with "capture.set (property, value);" or in the driver with "v4l2-ctl --set-ctrl=auto_exposure=1"*/
waitKey (200); //Wait 200 ms to ensure the device is open
Mat frame; // create Matrix where the new frame will be stored
if (capture.isOpened()){
while (true){
capture >> frame; //Put the new image in the Matrix
imshow ("Image", frame); //function to show the image in the screen
}
}
}
- 本人是编程初学者Raspberry pi，以上问题陈述如有错误请多多包涵
"With some of your recommendations, I slighthly modified the raspicam c++ api code and combined with CIMG image processing functionality "
#include "CImg.h"
#include <iostream>
#include <cstdlib>
#include <fstream>
#include <sstream>
#include <sys/timeb.h>
#include "raspicam.h"
using namespace std;
using namespace cimg_library;
bool doTestSpeedOnly=false;
size_t nFramesCaptured=100;
//parse command line
//returns the index of a command line param in argv. If not found, return -1
int findParam ( string param,int argc,char **argv ) {
int idx=-1;
for ( int i=0; i<argc && idx==-1; i++ )
if ( string ( argv[i] ) ==param ) idx=i;
return idx;
}
//parse command line
//returns the value of a command line param. If not found, defvalue is returned
float getParamVal ( string param,int argc,char **argv,float defvalue=-1 ) {
int idx=-1;
for ( int i=0; i<argc && idx==-1; i++ )
if ( string ( argv[i] ) ==param ) idx=i;
if ( idx==-1 ) return defvalue;
else return atof ( argv[ idx+1] );
}
raspicam::RASPICAM_EXPOSURE getExposureFromString ( string str ) {
if ( str=="OFF" ) return raspicam::RASPICAM_EXPOSURE_OFF;
if ( str=="AUTO" ) return raspicam::RASPICAM_EXPOSURE_AUTO;
if ( str=="NIGHT" ) return raspicam::RASPICAM_EXPOSURE_NIGHT;
if ( str=="NIGHTPREVIEW" ) return raspicam::RASPICAM_EXPOSURE_NIGHTPREVIEW;
if ( str=="BACKLIGHT" ) return raspicam::RASPICAM_EXPOSURE_BACKLIGHT;
if ( str=="SPOTLIGHT" ) return raspicam::RASPICAM_EXPOSURE_SPOTLIGHT;
if ( str=="SPORTS" ) return raspicam::RASPICAM_EXPOSURE_SPORTS;
if ( str=="SNOW" ) return raspicam::RASPICAM_EXPOSURE_SNOW;
if ( str=="BEACH" ) return raspicam::RASPICAM_EXPOSURE_BEACH;
if ( str=="VERYLONG" ) return raspicam::RASPICAM_EXPOSURE_VERYLONG;
if ( str=="FIXEDFPS" ) return raspicam::RASPICAM_EXPOSURE_FIXEDFPS;
if ( str=="ANTISHAKE" ) return raspicam::RASPICAM_EXPOSURE_ANTISHAKE;
if ( str=="FIREWORKS" ) return raspicam::RASPICAM_EXPOSURE_FIREWORKS;
return raspicam::RASPICAM_EXPOSURE_AUTO;
}
raspicam::RASPICAM_AWB getAwbFromString ( string str ) {
if ( str=="OFF" ) return raspicam::RASPICAM_AWB_OFF;
if ( str=="AUTO" ) return raspicam::RASPICAM_AWB_AUTO;
if ( str=="SUNLIGHT" ) return raspicam::RASPICAM_AWB_SUNLIGHT;
if ( str=="CLOUDY" ) return raspicam::RASPICAM_AWB_CLOUDY;
if ( str=="SHADE" ) return raspicam::RASPICAM_AWB_SHADE;
if ( str=="TUNGSTEN" ) return raspicam::RASPICAM_AWB_TUNGSTEN;
if ( str=="FLUORESCENT" ) return raspicam::RASPICAM_AWB_FLUORESCENT;
if ( str=="INCANDESCENT" ) return raspicam::RASPICAM_AWB_INCANDESCENT;
if ( str=="FLASH" ) return raspicam::RASPICAM_AWB_FLASH;
if ( str=="HORIZON" ) return raspicam::RASPICAM_AWB_HORIZON;
return raspicam::RASPICAM_AWB_AUTO;
}
void processCommandLine ( int argc,char **argv,raspicam::RaspiCam &Camera ) {
Camera.setWidth ( getParamVal ( "-w",argc,argv,640 ) );
Camera.setHeight ( getParamVal ( "-h",argc,argv,480 ) );
Camera.setBrightness ( getParamVal ( "-br",argc,argv,50 ) );
Camera.setSharpness ( getParamVal ( "-sh",argc,argv,0 ) );
Camera.setContrast ( getParamVal ( "-co",argc,argv,0 ) );
Camera.setSaturation ( getParamVal ( "-sa",argc,argv,0 ) );
Camera.setShutterSpeed( getParamVal ( "-ss",argc,argv,0 ) );
Camera.setISO ( getParamVal ( "-iso",argc,argv ,400 ) );
if ( findParam ( "-vs",argc,argv ) !=-1 )
Camera.setVideoStabilization ( true );
Camera.setExposureCompensation ( getParamVal ( "-ec",argc,argv ,0 ) );
if ( findParam ( "-gr",argc,argv ) !=-1 )
Camera.setFormat(raspicam::RASPICAM_FORMAT_GRAY);
if ( findParam ( "-yuv",argc,argv ) !=-1 )
Camera.setFormat(raspicam::RASPICAM_FORMAT_YUV420);
if ( findParam ( "-test_speed",argc,argv ) !=-1 )
doTestSpeedOnly=true;
int idx;
if ( ( idx=findParam ( "-ex",argc,argv ) ) !=-1 )
Camera.setExposure ( getExposureFromString ( argv[idx+1] ) );
if ( ( idx=findParam ( "-awb",argc,argv ) ) !=-1 )
Camera.setAWB( getAwbFromString ( argv[idx+1] ) );
nFramesCaptured=getParamVal("-nframes",argc,argv,100);
Camera.setAWB_RB(getParamVal("-awb_b",argc,argv ,1), getParamVal("-awb_g",argc,argv ,1));
}
//timer functions
#include <sys/time.h>
#include <unistd.h>
class Timer{
private:
struct timeval _start, _end;
public:
Timer(){}
void start(){
gettimeofday(&_start, NULL);
}
void end(){
gettimeofday(&_end, NULL);
}
double getSecs(){
return double(((_end.tv_sec - _start.tv_sec) * 1000 + (_end.tv_usec - _start.tv_usec)/1000.0) + 0.5)/1000.;
}
};
void saveImage ( string filepath,unsigned char *data,raspicam::RaspiCam &Camera ) {
std::ofstream outFile ( filepath.c_str(),std::ios::binary );
if ( Camera.getFormat()==raspicam::RASPICAM_FORMAT_BGR || Camera.getFormat()==raspicam::RASPICAM_FORMAT_RGB ) {
outFile<<"P6\n";
} else if ( Camera.getFormat()==raspicam::RASPICAM_FORMAT_GRAY ) {
outFile<<"P5\n";
} else if ( Camera.getFormat()==raspicam::RASPICAM_FORMAT_YUV420 ) { //made up format
outFile<<"P7\n";
}
outFile<<Camera.getWidth() <<" "<<Camera.getHeight() <<" 255\n";
outFile.write ( ( char* ) data,Camera.getImageBufferSize() );
}
int main ( int argc,char **argv ) {
int a=1,b=0,c;
int x=444,y=129; //pixel coordinates
raspicam::RaspiCam Camera;
processCommandLine ( argc,argv,Camera );
cout<<"Connecting to camera"<<endl;
if ( !Camera.open() ) {
cerr<<"Error opening camera"<<endl;
return -1;
}
// cout<<"Connected to camera ="<<Camera.getId() <<" bufs="<<Camera.getImageBufferSize( )<<endl;
unsigned char *data=new unsigned char[ Camera.getImageBufferSize( )];
Timer timer;
// cout<<"Capturing...."<<endl;
// size_t i=0;
timer.start();
for (int i=0;i<=nFramesCaptured;i++)
{
Camera.grab();
Camera.retrieve ( data );
std::stringstream fn;
fn<<"image.jpg";
saveImage ( fn.str(),data,Camera );
// cerr<<"Saving "<<fn.str()<<endl;
CImg<float> Img("/run/shm/image.jpg");
//Img.display("Window Title");
// 9 PIXELS MATRIX GRAYSCALE VALUES
float pixvalR1 = Img(x-1,y-1);
float pixvalR2 = Img(x,y-1);
float pixvalR3 = Img(x+1,y-1);
float pixvalR4 = Img(x-1,y);
float pixvalR5 = Img(x,y);
float pixvalR6 = Img(x+1,y);
float pixvalR7 = Img(x-1,y+1);
float pixvalR8 = Img(x,y+1);
float pixvalR9 = Img(x+1,y+1);
// std::cout<<"coordinate value :"<<pixvalR5 << endl;
// MEAN VALUES OF RGB PIXELS
float light = (pixvalR1+pixvalR2+pixvalR3+pixvalR4+pixvalR5+pixvalR6+pixvalR7+pixvalR8+pixvalR9)/9 ;
// DISPLAYING MEAN RGB VALUES OF 9 PIXELS
// std::cout<<"Lightness value :"<<light << endl;
// THRESHOLDING CONDITION
c = (light > 130 ) ? a : b;
// cout<<"Data is " << c <<endl;
ofstream fout("c.txt", ios::app);
fout<<c;
fout.close();
}
timer.end();
cerr<< timer.getSecs()<< " seconds for "<< nFramesCaptured << " frames : FPS " << ( ( float ) ( nFramesCaptured ) / timer.getSecs() ) <<endl;
Camera.release();
std::cin.ignore();
}
- 从这段代码中，我想知道我们如何直接从 camera.retrieve(data) 中获取数据，而不将其存储为图像文件并从图像缓冲区访问数据，以进行处理图像并将其进一步删除。
根据 Mark Setchell 的建议，我对代码做了一些小的改动，我得到了很好的结果，但是，有没有什么方法可以提高处理性能以获得更高的帧率？使用此代码，我最多可以达到 10 FPS。
#include <ctime>
#include <fstream>
#include <iostream>
#include <thread>
#include <mutex>
#include <raspicam/raspicam.h>
// Don't want any X11 display by CImg
#define cimg_display 0
#include <CImg.h>
using namespace cimg_library;
using namespace std;
#define NFRAMES 1000
#define NTHREADS 2
#define WIDTH 640
#define HEIGHT 480
// Commands/status for the worker threads
#define WAIT 0
#define GO 1
#define GOING 2
#define EXIT 3
#define EXITED 4
volatile int command[NTHREADS];
// Serialize access to cout
std::mutex cout_mutex;
// CImg initialisation
// Create a 1280x960 greyscale (Y channel of YUV) image
// Create a globally-accessible CImg for main and workers to access
CImg<unsigned char> img(WIDTH,HEIGHT,1,1,128);
////////////////////////////////////////////////////////////////////////////////
// worker thread - There will 2 or more of these running in parallel with the
// main thread. Do any image processing in here.
////////////////////////////////////////////////////////////////////////////////
void worker (int id) {
// If you need a "results" image of type CImg, create it here before entering
// ... the main processing loop below - you don't want to do malloc()s in the
// ... high-speed loop
// CImg results...
int wakeups=0;
// Create a white for annotating
unsigned char white[] = { 255,255,255 };
while(true){
// Busy wait with 500us sleep - at worst we only miss 50us of processing time per frame
while((command[id]!=GO)&&(command[id]!=EXIT)){
std::this_thread::sleep_for(std::chrono::microseconds(500));
}
if(command[id]==EXIT){command[id]=EXITED;break;}
wakeups++;
// Process frame of data - access CImg structure here
command[id]=GOING;
// You need to add your processing in HERE - everything from
// ... 9 PIXELS MATRIX GRAYSCALE VALUES to
// ... THRESHOLDING CONDITION
int a=1,b=0,c;
int x=330,y=84;
// CImg<float> Img("/run/shm/result.png");
float pixvalR1 = img(x-1,y-1);
float pixvalR2 = img(x,y-1);
float pixvalR3 = img(x+1,y-1);
float pixvalR4 = img(x-1,y);
float pixvalR5 = img(x,y);
float pixvalR6 = img(x+1,y);
float pixvalR7 = img(x-1,y+1);
float pixvalR8 = img(x,y+1);
float pixvalR9 = img(x+1,y+1);
// MEAN VALUES OF RGB PIXELS
float light = (pixvalR1+pixvalR2+pixvalR3+pixvalR4+pixvalR5+pixvalR6+pixvalR7+pixvalR8+pixvalR9)/9 ;
// DISPLAYING MEAN RGB VALUES OF 9 PIXELS
// std::cout<<"Lightness value :"<<light << endl;
// THRESHOLDING CONDITION
c = (light > 130 ) ? a : b;
// cout<<"Data is " << c <<endl;
ofstream fout("c.txt", ios::app);
fout<<c;
fout.close();
// Pretend to do some processing.
// You need to delete the following "sleep_for" and "if(id==0...){...}"
// std::this_thread::sleep_for(std::chrono::milliseconds(2));
/* if((id==0)&&(wakeups==NFRAMES)){
// Annotate final image and save as PNG
img.draw_text(100,100,"Hello World",white);
img.save_png("result.png");
} */
}
cout_mutex.lock();
std::cout << "Thread[" << id << "]: Received " << wakeups << " wakeups" << std::endl;
cout_mutex.unlock();
}
//timer functions
#include <sys/time.h>
#include <unistd.h>
class Timer{
private:
struct timeval _start, _end;
public:
Timer(){}
void start(){
gettimeofday(&_start, NULL);
}
void end(){
gettimeofday(&_end, NULL);
}
double getSecs(){
return double(((_end.tv_sec - _start.tv_sec) * 1000 + (_end.tv_usec - _start.tv_usec)/1000.0) + 0.5)/1000.;
}
};
int main ( int argc,char **argv ) {
Timer timer;
raspicam::RaspiCam Camera;
// Allowable values: RASPICAM_FORMAT_GRAY,RASPICAM_FORMAT_RGB,RASPICAM_FORMAT_BGR,RASPICAM_FORMAT_YUV420
Camera.setFormat(raspicam::RASPICAM_FORMAT_YUV420);
// Allowable widths: 320, 640, 1280
// Allowable heights: 240, 480, 960
// setCaptureSize(width,height)
Camera.setCaptureSize(WIDTH,HEIGHT);
std::cout << "Main: Starting" << std::endl;
std::cout << "Main: NTHREADS:" << NTHREADS << std::endl;
std::cout << "Main: NFRAMES:" << NFRAMES << std::endl;
std::cout << "Main: Width: " << Camera.getWidth() << std::endl;
std::cout << "Main: Height: " << Camera.getHeight() << std::endl;
// Spawn worker threads - making sure they are initially in WAIT state
std::thread threads[NTHREADS];
for(int i=0; i<NTHREADS; ++i){
command[i]=WAIT;
threads[i] = std::thread(worker,i);
}
// Open camera
cout<<"Opening Camera..."<<endl;
if ( !Camera.open()) {cerr<<"Error opening camera"<<endl;return -1;}
// Wait until camera stabilizes
std::cout<<"Sleeping for 3 secs"<<endl;
std::this_thread::sleep_for(std::chrono::seconds(3));
timer.start();
for(int frame=0;frame<NFRAMES;frame++){
// Capture frame
Camera.grab();
// Copy just the Y component to our mono CImg
std::memcpy(img._data,Camera.getImageBufferData(),WIDTH*HEIGHT);
// Notify worker threads that data is ready for processing
for(int i=0; i<NTHREADS; ++i){
command[i]=GO;
}
}
timer.end();
cerr<< timer.getSecs()<< " seconds for "<< NFRAMES << " frames : FPS " << ( ( float ) ( NFRAMES ) / timer.getSecs() ) << endl;
// Let workers process final frame, then tell to exit
// std::this_thread::sleep_for(std::chrono::milliseconds(50));
// Notify worker threads to exit
for(int i=0; i<NTHREADS; ++i){
command[i]=EXIT;
}
// Wait for all threads to finish
for(auto& th : threads) th.join();
}
执行代码的编译命令：
g++ -std=c++11 /home/pi/raspicam/src/raspicimgthread.cpp -o threadraspicimg -I. -I/usr/local/include -L /opt/vc/lib -L /usr/local/lib -lraspicam -lmmal -lmmal_core -lmmal_util -O2 -L/usr/X11R6/lib -lm -lpthread -lX11
**RESULTS :**
Main: Starting
Main: NTHREADS:2
Main: NFRAMES:1000
Main: Width: 640
Main: Height: 480
Opening Camera...
Sleeping for 3 secs
99.9194 seconds for 1000 frames : FPS 10.0081
Thread[1]: Received 1000 wakeups
Thread[0]: Received 1000 wakeups
real 1m43.198s
user 0m2.060s
sys 0m5.850s
还有一个问题是，当我使用普通的 Raspicam c++ API 代码执行相同的任务（我之前提到的代码）时，我得到了几乎相同的结果，但在性能（当然我的帧速率从 9.4 FPS 增加到 10 FPS）。
但是在代码1中：
我一直在将图像保存在 ram 磁盘中进行处理，然后我正在删除。 我没有使用任何线程进行并行处理。
在代码2中：
我们没有在磁盘中保存任何图像，而是直接从缓冲区中处理它。而且我们也在使用线程来提高处理速度。
不幸的是，尽管我们对代码 1 的代码 2 进行了一些更改，但我无法获得所需的结果（将以 30 FPS 执行）
等待您提出好的建议，非常感谢您的帮助。
提前致谢
最好的问候 BLV Lohith Kumar
更新答案
我在这里更新了我的原始答案，以展示如何将获取的数据复制到 CImg 结构中，还展示了 2 个工作线程可以在主线程继续获取帧的同时处理图像全速。它达到每秒 60 帧。
我没有在工作线程内部做任何处理，因为我不知道你想做什么。我所做的只是将最后一帧保存到磁盘，以表明采集到 CImg 的工作正常。你可以有 3 个工作线程。您可以在 round-robin 的基础上将一帧传递给每个线程，或者您可以让 2 个线程中的每一个在每次迭代时处理一半的帧。或者 3 个线程中的每一个处理一帧的三分之一。您可以更改轮询唤醒以使用条件变量。
#include <ctime>
#include <fstream>
#include <iostream>
#include <thread>
#include <mutex>
#include <raspicam/raspicam.h>
// Don't want any X11 display by CImg
#define cimg_display 0
#include <CImg.h>
using namespace cimg_library;
using namespace std;
#define NFRAMES 1000
#define NTHREADS 2
#define WIDTH 1280
#define HEIGHT 960
// Commands/status for the worker threads
#define WAIT 0
#define GO 1
#define GOING 2
#define EXIT 3
#define EXITED 4
volatile int command[NTHREADS];
// Serialize access to cout
std::mutex cout_mutex;
// CImg initialisation
// Create a 1280x960 greyscale (Y channel of YUV) image
// Create a globally-accessible CImg for main and workers to access
CImg<unsigned char> img(WIDTH,HEIGHT,1,1,128);
////////////////////////////////////////////////////////////////////////////////
// worker thread - There will 2 or more of these running in parallel with the
// main thread. Do any image processing in here.
////////////////////////////////////////////////////////////////////////////////
void worker (int id) {
// If you need a "results" image of type CImg, create it here before entering
// ... the main processing loop below - you don't want to do malloc()s in the
// ... high-speed loop
// CImg results...
int wakeups=0;
// Create a white for annotating
unsigned char white[] = { 255,255,255 };
while(true){
// Busy wait with 500us sleep - at worst we only miss 50us of processing time per frame
while((command[id]!=GO)&&(command[id]!=EXIT)){
std::this_thread::sleep_for(std::chrono::microseconds(500));
}
if(command[id]==EXIT){command[id]=EXITED;break;}
wakeups++;
// Process frame of data - access CImg structure here
command[id]=GOING;
// You need to add your processing in HERE - everything from
// ... 9 PIXELS MATRIX GRAYSCALE VALUES to
// ... THRESHOLDING CONDITION
// Pretend to do some processing.
// You need to delete the following "sleep_for" and "if(id==0...){...}"
std::this_thread::sleep_for(std::chrono::milliseconds(2));
if((id==0)&&(wakeups==NFRAMES)){
// Annotate final image and save as PNG
img.draw_text(100,100,"Hello World",white);
img.save_png("result.png");
}
}
cout_mutex.lock();
std::cout << "Thread[" << id << "]: Received " << wakeups << " wakeups" << std::endl;
cout_mutex.unlock();
}
int main ( int argc,char **argv ) {
raspicam::RaspiCam Camera;
// Allowable values: RASPICAM_FORMAT_GRAY,RASPICAM_FORMAT_RGB,RASPICAM_FORMAT_BGR,RASPICAM_FORMAT_YUV420
Camera.setFormat(raspicam::RASPICAM_FORMAT_YUV420);
// Allowable widths: 320, 640, 1280
// Allowable heights: 240, 480, 960
// setCaptureSize(width,height)
Camera.setCaptureSize(WIDTH,HEIGHT);
std::cout << "Main: Starting" << std::endl;
std::cout << "Main: NTHREADS:" << NTHREADS << std::endl;
std::cout << "Main: NFRAMES:" << NFRAMES << std::endl;
std::cout << "Main: Width: " << Camera.getWidth() << std::endl;
std::cout << "Main: Height: " << Camera.getHeight() << std::endl;
// Spawn worker threads - making sure they are initially in WAIT state
std::thread threads[NTHREADS];
for(int i=0; i<NTHREADS; ++i){
command[i]=WAIT;
threads[i] = std::thread(worker,i);
}
// Open camera
cout<<"Opening Camera..."<<endl;
if ( !Camera.open()) {cerr<<"Error opening camera"<<endl;return -1;}
// Wait until camera stabilizes
std::cout<<"Sleeping for 3 secs"<<endl;
std::this_thread::sleep_for(std::chrono::seconds(3));
for(int frame=0;frame<NFRAMES;frame++){
// Capture frame
Camera.grab();
// Copy just the Y component to our mono CImg
std::memcpy(img._data,Camera.getImageBufferData(),WIDTH*HEIGHT);
// Notify worker threads that data is ready for processing
for(int i=0; i<NTHREADS; ++i){
command[i]=GO;
}
}
// Let workers process final frame, then tell to exit
std::this_thread::sleep_for(std::chrono::milliseconds(50));
// Notify worker threads to exit
for(int i=0; i<NTHREADS; ++i){
command[i]=EXIT;
}
// Wait for all threads to finish
for(auto& th : threads) th.join();
}
时间注意事项
你可以这样计时：
#include <chrono>
typedef std::chrono::high_resolution_clock hrclock;
hrclock::time_point t1,t2;
t1 = hrclock::now();
// do something that needs timing
t2 = hrclock::now();
std::chrono::nanoseconds elapsed = t2-t1;
long long nanoseconds=elapsed.count();
原答案
我一直在用 Raspicam 做一些实验。我从 SourceForge 下载了他们的代码并稍作修改以进行一些简单的 capture-only 测试。我最终使用的代码如下所示：
#include <ctime>
#include <fstream>
#include <iostream>
#include <raspicam/raspicam.h>
#include <unistd.h> // for usleep()
using namespace std;
#define NFRAMES 1000
int main ( int argc,char **argv ) {
raspicam::RaspiCam Camera;
// Allowable values: RASPICAM_FORMAT_GRAY,RASPICAM_FORMAT_RGB,RASPICAM_FORMAT_BGR,RASPICAM_FORMAT_YUV420
Camera.setFormat(raspicam::RASPICAM_FORMAT_YUV420);
// Allowable widths: 320, 640, 1280
// Allowable heights: 240, 480, 960
// setCaptureSize(width,height)
Camera.setCaptureSize(1280,960);
// Open camera
cout<<"Opening Camera..."<<endl;
if ( !Camera.open()) {cerr<<"Error opening camera"<<endl;return -1;}
// Wait until camera stabilizes
cout<<"Sleeping for 3 secs"<<endl;
usleep(3000000);
cout << "Grabbing " << NFRAMES << " frames" << endl;
// Allocate memory
unsigned long bytes=Camera.getImageBufferSize();
cout << "Width: " << Camera.getWidth() << endl;
cout << "Height: " << Camera.getHeight() << endl;
cout << "ImageBufferSize: " << bytes << endl;;
unsigned char *data=new unsigned char[bytes];
for(int frame=0;frame<NFRAMES;frame++){
// Capture frame
Camera.grab();
// Extract the image
Camera.retrieve ( data,raspicam::RASPICAM_FORMAT_IGNORE );
// Wake up a thread here to process the frame with CImg
}
return 0;
}
我不喜欢cmake所以我这样编译：
g++ -std=c++11 simpletest.c -o simpletest -I. -I/usr/local/include -L /opt/vc/lib -L /usr/local/lib -lraspicam -lmmal -lmmal_core -lmmal_util
我发现，无论图像的尺寸如何，或多或少与编码（RGB、BGR、GRAY）无关，它都能达到 30 fps（每秒帧数）。
唯一能让我变得更好的方法是进行以下更改：
在上面的代码中，使用RASPICAM_FORMAT_YUV420而不是其他任何东西
编辑文件
private_impl.cpp并更改第 71 行以将帧速率设置为 90。
如果我这样做，我可以达到 66 fps。
由于 Raspberry Pi 只有 900MHz CPU 但是有 4 个核心，我猜你会想在循环外的开始处启动 1-3 个额外的线程然后唤醒一个或多个我在处理数据的代码中注明的地方。他们要做的第一件事是在下一帧开始之前将数据从采集缓冲区中复制出来——或者有多个缓冲区并以 round-robin 方式使用它们。
线程注意事项
下图中，绿色代表获取图像的Camera.grab()，红色代表获取图像后进行的处理。目前，您正在获取数据（绿色），然后在获取下一帧之前对其进行处理（红色）。请注意，您的 4 个 CPU 中有 3 个什么都不做。
我的建议是将处理（红色）卸载给另一个 CPUs/threads 并尽可能快地获取新数据（绿色）。像这样：
现在您看到每秒获得更多帧（绿色）。