为什么从 Project Tango 平板电脑中提取的图像显示为灰色？

Question

我正在使用 Project Tango C API

我正在将 TangoImageBuffer.data 指向的整个缓冲区写入文件。

// open a file
std::fstream f(app.getNextFileName(), std::ios::out | std::ios::binary);

// write the entire stride * height * 4 buffer into the file
f.write((const char *) tango_image_buffer->data, 
    tango_image_buffer->stride * buffer->height * 4 * sizeof(uint8_t)
);

f.close();

然后我将文件导出到我的电脑并使用 python-opencv 和 numpy 将其可视化：

import sys

import cv2
import numpy as np

# dimensions of the image
stride    = 1280
width     = 1280
height    = 720

# I am using 3 channels so that the resulting image is not
# transluscent
channels  = 3

input_filename  = sys.argv[1]
output_filename = sys.argv[2]

# load the image buffer into a list
data = np.fromfile(input_filename, dtype=np.uint8)

# create a height x width x channels matrix with the datatype uint8 
# and all elements set to zero
img = np.zeros((height, width, channels), dtype=np.uint8);

# map elements in array to image matrix
for y in range(0, height):    
    for x in range(0, width):
        img[y, x, 0] = data[y * stride + x + 2] #blue
        img[y, x, 1] = data[y * stride + x + 1] #green
        img[y, x, 2] = data[y * stride + x + 0] #red

# display and save the resulting image
cv2.namedWindow("tango-rgba")
cv2.imshow("tango-rgba", img)
cv2.waitKey(0)
cv2.destroyAllWindows()
cv2.imwrite(output_filename, img)

很遗憾，生成的图像looks gray
只有图像显示是灰色的。仔细观察像素，你会发现像素的红绿蓝分量为different.
编辑：我已更改此 post 以指定我使用的缓冲区为 RGBA8888 格式。从项目探戈参考，"Only RGBA 8888 is provided"
编辑：实际上似乎没有使用 RGBA8888 格式。 rhashimoto 建议格式为 YV12.
编辑：图像格式实际上是 NV21（见下面的答案）。

Answer 1

似乎在撰写本文时 post，Project Tango 参考确实不正确。

图片格式为NV21.

char format_str[100] = {0};

switch (tango_image_buffer->format)
{
  case TANGO_HAL_PIXEL_FORMAT_RGBA_8888:
        sprintf(format_str, "%s", "RGBA8888");
        break;
  case TANGO_HAL_PIXEL_FORMAT_YV12:
        sprintf(format_str, "%s", "YV12");
        break;
  case TANGO_HAL_PIXEL_FORMAT_YCrCb_420_SP:
        sprintf(format_str, "%s", "NV21");
        break;
  default:
        break;

}
__android_log_print(ANDROID_LOG_VERBOSE, "Capture Info",
        "stride: %u, width: %u, height: %u, timestamp: %lf, frame: %llu, format: %s",
        tango_image_buffer->stride, 
        tango_image_buffer->width, 
        tango_image_buffer->height,
        tango_image_buffer->timestamp, 
        tango_image_buffer->frame_number, 
        format_str);

上面这段代码的输出是：

   V/Capture Info﹕ stride: 1280, width: 1280, height: 720, timestamp: 22157.368703, frame: 0, format: NV21

这是我在 python 中使用 opencv 和 numpy 的新可视化代码：

import sys

import cv2
import numpy as np

input_filename  = sys.argv[1]
output_filename = sys.argv[2]


# dimensions of the image
stride    = 1280
width     = 1280
height    = 720
channels  = 4

# load file into buffer
data = np.fromfile(input_filename, dtype=np.uint8)

# create yuv image
yuv = np.ndarray((height + height / 2, width), dtype=np.uint8, buffer=data)

# create a height x width x channels matrix with the datatype uint8 for rgb image
img = np.zeros((height, width, channels), dtype=np.uint8);

# convert yuv image to rgb image
cv2.cvtColor(yuv, cv2.COLOR_YUV2BGRA_NV21, img, channels)

# display and save the resulting image
cv2.namedWindow("tango-rgba")
cv2.imshow("tango-rgba", img)
cv2.waitKey(0)
cv2.destroyAllWindows()
cv2.imwrite(output_filename, img)

和 resulting image.

TLDR：图像格式为 NV21。 OpenCV 提供了将 NV21 转换为 RGB 的功能。 result 看起来很棒。

为什么从 Project Tango 平板电脑中提取的图像显示为灰色？

Why does the image extracted from the Project Tango Tablet appear gray?

c++

c

google-project-tango