在处理中从相机获取两个连续帧

Question

我一直在尝试使用 Processing 实现此目的的几种方法，但每次都无法获得完全连续的方法。有人知道这样做的 "right way" 吗？

提前致谢！

Answer 1

我看了一下科泰相机sources and issue reports, especially this one。不幸的是，此代码并非用于提供实时相机帧。

您可以尝试AndroidCapture project as a starter, and modify the native Android class来实现您的目标。

Answer 2

理论上，听 captureEvent() 获取新帧并跟踪第一帧是否先前已录制，如果是则随后录制第二帧。

这是一个基本的注释草图来说明这一点（按任意键抓取另一对帧）：

import processing.video.*;

Capture camera;

PImage firstFrame;
PImage secondFrame;

void setup(){
  size(1920,480);

  camera = new Capture(this,640,480);
  camera.start();
}
void draw(){
  image(camera,0,0);
  if(firstFrame != null){
    image(firstFrame,640,0);
  }
  if(secondFrame != null){
    image(secondFrame,1280,0);
  }
}
//this is the callback from the video library when a new camera frame is available
void captureEvent(Capture c){
  //read a new frame
  c.read();
  //if the first frame wasn't recorded yet, record(copy) it's pixels
  if(firstFrame == null){
    firstFrame = c.get();
  }
  //same for the second frame, but check if the first frame has been recorded first
  if(firstFrame != null && secondFrame == null){
    secondFrame = c.get();
  }
}

void keyPressed(){
  //reset consecutive frames on keypress
  firstFrame = secondFrame = null;
}

理论上（正如您在 Processing Video Library's source code 中看到的那样），仅当新相机样本准备就绪时才会触发 captureEvent。在实践中，您会发现两个连续的帧可能看起来相同（即使它们在时间上可能相隔一秒钟），甚至如您在评论中指出的那样有噪声。

感觉你要的是一个连续的帧，但是又和之前的有很大区别。如果是这种情况，您可以使用 FrameDifferencing example (Processing > Examples > Libraries > Video > Capture > FrameDifferencing)

这是上面草图的修改版本，使用 Golan Levin 的 FrameDifferencing 代码只抓取第二帧，如果它有一点不同的话：

import processing.video.*;

Capture camera;

PImage firstFrame;
PImage secondFrame;
PImage diff;

void setup(){
  size(1920,960);

  camera = new Capture(this,640,480);
  camera.start();

  diff = createImage(640,480,RGB);
}
void draw(){
  image(camera,0,0);
  if(firstFrame != null){
    image(firstFrame,640,0);
  }
  if(secondFrame != null){
    image(secondFrame,1280,0);
  }
  image(diff,0,480);
}
//this is the callback from the video library when a new camera frame is available
void captureEvent(Capture c){
  //read a new frame
  c.read();
  //if the first frame wasn't recorded yet, record(copy) it's pixels
  if(firstFrame == null){
    firstFrame = c.get();
    println("recorded first frame at",new java.util.Date());
  }
  //same for the second frame, but check if the first frame has been recorded first
  if(firstFrame != null && secondFrame == null){
    //if the difference between the first frame cand the current frame is even ever so slightly off, record the second frame
    if(difference(firstFrame,camera) > 100){
      secondFrame = c.get();
    }
  }

}

int difference(PImage first,PImage second){
  final int numPixels = 640*480;
  camera.loadPixels();
  int movementSum = 0; // Amount of movement in the frame
  for (int i = 0; i < numPixels; i++) { // For each pixel in the video frame...
    color currColor = first.pixels[i];
    color prevColor = second.pixels[i];
    // Extract the red, green, and blue components from current pixel
    int currR = (currColor >> 16) & 0xFF; // Like red(), but faster
    int currG = (currColor >> 8) & 0xFF;
    int currB = currColor & 0xFF;
    // Extract red, green, and blue components from previous pixel
    int prevR = (prevColor >> 16) & 0xFF;
    int prevG = (prevColor >> 8) & 0xFF;
    int prevB = prevColor & 0xFF;
    // Compute the difference of the red, green, and blue values
    int diffR = abs(currR - prevR);
    int diffG = abs(currG - prevG);
    int diffB = abs(currB - prevB);
    // Render the difference image to the screen
    diff.pixels[i] = color(diffR, diffG, diffB);
    // Add these differences to the running tally
    movementSum += diffR + diffG + diffB;
  }
  diff.updatePixels();
  return movementSum;
}

void keyPressed(){
  //reset consecutive frames on keypress
  firstFrame = secondFrame = null;
}

在上面的例子中，100 是一个任意值。最大值为 255*3*640*480（每个通道 0-255 * 通道数 * 宽度 * 高度）

在处理中从相机获取两个连续帧

Getting two consecutive frames from camera in Processing

image

video

camera

processing

frames