OpenGL视频帧适配

Question

我正在做一个项目，我必须在纵向模式下将分辨率为 640x480 的相机的数据投影到 4K 屏幕上。

摄像头是 Kinect V1，但我会切换到分辨率更好 (1920x1080) 的版本 2。

我的问题是如何更改要显示的纹理比例以获得正确的结果。目前，我已经设法在整个屏幕上显示，但图像的宽度变平了。理想的做法是保持比例并在图像的每一侧切割一个 X 宽度。

我正在使用带有 OpenGL 的 SDL，这里是代码的相关部分：

// window creation
auto window = SDL_CreateWindow("Imagine",
                                   x,
                                   y,
                                   0,
                                   0,
                                   SDL_WINDOW_OPENGL | SDL_WINDOW_SHOWN | SDL_WINDOW_FULLSCREEN_DESKTOP | SDL_WINDOW_ALLOW_HIGHDPI | SDL_WINDOW_BORDERLESS);

// GL initialization and texture creation
void SdlNuitrackRenderHandler::initTexture(int width, int height)
{
    glEnable(GL_TEXTURE_2D);
    glEnableClientState(GL_VERTEX_ARRAY);
    glDisableClientState(GL_COLOR_ARRAY);
    glOrtho(0, _width, _height, 0, -1.0, 1.0);
    glMatrixMode(GL_PROJECTION);
    glPushMatrix();
    glLoadIdentity();

    glGenTextures(1, &_textureID);

    width = power2(width);
    height = power2(height);

    if (_textureBuffer != 0)
        delete[] _textureBuffer;

    _textureBuffer = new uint8_t[width * height * 3];
    memset(_textureBuffer, 0, sizeof(uint8_t) * width * height * 3);

    glBindTexture(GL_TEXTURE_2D, _textureID);
    glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, width, height, 0, GL_RGB, GL_UNSIGNED_BYTE, NULL);

    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);

    // Set texture coordinates [0, 1] and vertexes position
    {
        _textureCoords[0] = (float) _width / width;
        _textureCoords[1] = (float) _height / height;
        _textureCoords[2] = (float) _width / width;
        _textureCoords[3] = 0.0;
        _textureCoords[4] = 0.0;
        _textureCoords[5] = 0.0;
        _textureCoords[6] = 0.0;
        _textureCoords[7] = (float) _height / height;

        _vertexes[0] = _width;
        _vertexes[1] = _height;
        _vertexes[2] = _width;
        _vertexes[3] = 0.0;
        _vertexes[4] = 0.0;
        _vertexes[5] = 0.0;
        _vertexes[6] = 0.0;
        _vertexes[7] = _height;
    }

// Texture rendering
// Render prepared background texture
void SdlNuitrackRenderHandler::renderTexture()
{
    glClearColor(1, 1, 1, 1);
    glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);

    glEnable(GL_TEXTURE_2D);
    glColor4f(1, 1, 1, 1);

    glBindTexture(GL_TEXTURE_2D, _textureID);
    glTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, _width, _height, GL_RGB, GL_UNSIGNED_BYTE, _textureBuffer);

    glEnableClientState(GL_VERTEX_ARRAY);
    glEnableClientState(GL_TEXTURE_COORD_ARRAY);

    glVertexPointer(2, GL_FLOAT, 0, _vertexes);
    glTexCoordPointer(2, GL_FLOAT, 0, _textureCoords);

    glDrawArrays(GL_TRIANGLE_FAN, 0, 4);

    glDisableClientState(GL_VERTEX_ARRAY);
    glDisableClientState(GL_TEXTURE_COORD_ARRAY);

    glDisable(GL_TEXTURE_2D);
}

Answer 1

虽然我同意评论中关于这是过时的 OpenGL 代码的内容，但这个问题的核心与 OpenGL 无关。您想要在另一个具有不同纵横比的矩形内绘制 1 个具有正确纵横比的矩形。您只需要知道在哪里放置顶点。

通常使用 TEXTURE_2D 纹理目标，您希望纹理坐标在两个方向上都为 0-1，除非您打算裁剪输入图像。曾经有一段时间，纹理的宽度和高度必须是 2 的幂。很长一段时间以来都不是这样了。所以删除这两行：

width = power2(width);
height = power2(height);

所以第一件事就是正确设置它们：

    _textureCoords[0] = 1.0;
    _textureCoords[1] = 1.0;
    _textureCoords[2] = 1.0;
    _textureCoords[3] = 0.0;
    _textureCoords[4] = 0.0;
    _textureCoords[5] = 0.0;
    _textureCoords[6] = 0.0;
    _textureCoords[7] = 1.0;

（因此，该代码确实难以阅读并且维护起来会很痛苦。您应该使纹理坐标（和顶点坐标）成为 struct 具有 x 和 y 值，所以它是有道理的。现在还不明显，它是 4 组 2D 坐标，它们是 (max, max), (max, min), (min, min ), (min, max)。但我离题了。）

接下来，要弄清楚纹理坐标，您需要知道视频是要缩放以适应宽度还是高度。为此，您可以计算出

double widthScaleRatio = displayWidth / imageWidth; // <- using this scale will guarantee the width of the new image is the same as the display's width, but might crop the height
double heightScaleRatio = displayHeight / imageHeight; // <- using this scale will guarantee the height of the new image is the same as the display's height but might crop the width
double  scale = 1.0;
// If scaling by the widthScaleRatio makes the height too big, use the heightScaleRatio
// Otherwise use the widthScaleRatio
if (imageHeight * widthScaleRatio > displayHeight)
{
    scale = heightScaleRatio;
}
else
{
    scale = widthScaleRatio;
}

现在按 scale:

缩放宽度和高度

double newWidth  = imageWidth * scale;
double newHeight = imageHeight * scale;

并基于此设置顶点：

    _vertexes[0] = newWidth;
    _vertexes[1] = newHeight;
    _vertexes[2] = newWidth;
    _vertexes[3] = 0.0;
    _vertexes[4] = 0.0;
    _vertexes[5] = 0.0;
    _vertexes[6] = 0.0;
    _vertexes[7] = newHeight;

同样的警告也适用于使此代码与纹理坐标一样更易于阅读。

编辑：这是一个简单的程序来展示它是如何工作的：

int main(){
    double  displayWidth    = 2160;
    double  displayHeight   = 4096;
    double  imageWidth  =  640;
    double  imageHeight = 480;
    double widthScaleRatio = displayWidth / imageWidth; // <- using this scale will guarantee the width of the new image is the same as the display's width, but might crop the height
    double heightScaleRatio = displayHeight / imageHeight; // <- using this scale will guarantee the height of the new image is the same as the display's height but might crop the width
    double  scale = 1.0;
    // If scaling by the widthScaleRatio makes the height too big, use the heightScaleRatio
    // Otherwise use the widthScaleRatio
    if (imageHeight * widthScaleRatio > displayHeight)
    {
        scale = heightScaleRatio;
    }
    else
    {
        scale = widthScaleRatio;
    }

    double newWidth  = imageWidth * scale;
    double newHeight = imageHeight * scale;

    std::cout << "New size = (" << newWidth << ", " << newHeight  << ")\n";
}

当我运行它时，我得到：

New size = (2160, 1620)

OpenGL视频帧适配

OpenGL video frame fitting

c++

opengl

sdl

kinect

sdl-2