glReadPixels 的快照。然后翻转像素内容。但是图像质量下降了
snapshot by glReadPixels. Then flip the pixels content. But the image quality has been decreased
全部。
现在我正在使用unity3D开发游戏。我想通过 AVFoundation 将每一帧的内容保存为 mp4 文件。但是我在处理快照时遇到了一些问题。在我使用 glReadPixels 获取保存在渲染缓冲区中的数据后，顶点着色器和片段着色器用于帮助我将像素内容的更新面朝下。但是，在翻转每一帧之后，我发现每一帧的质量都下降了很多。所以，任何人以前都遇到过这种情况。
这是相关的代码。
快照部分，
- (void *)snapshot
{
// NSLog(@"snapshot used here");
GLint backingWidth1, backingHeight1;
glBindRenderbufferOES(GL_RENDERBUFFER_OES, mainDisplaySurface->systemColorRB);
// Get the size of the backing CAEAGLLayer
glGetRenderbufferParameterivOES(GL_RENDERBUFFER_OES, GL_RENDERBUFFER_WIDTH_OES, &backingWidth1);
glGetRenderbufferParameterivOES(GL_RENDERBUFFER_OES, GL_RENDERBUFFER_HEIGHT_OES, &backingHeight1);
NSInteger x = 0, y = 0, width = backingWidth1, height = backingHeight1;
NSInteger dataLength = width * height * 4;
GLubyte *data = (GLubyte*)malloc(dataLength * sizeof(GLubyte));
// Read pixel data from the framebuffer
glPixelStorei(GL_PACK_ALIGNMENT, 4);
glReadPixels(x, y, width, height, GL_RGBA, GL_UNSIGNED_BYTE, data);
if (transformData == NULL)
{
NSLog(@"transformData initial");
transformData = new loadDataFromeX();
transformData->setupOGL(backingWidth1, backingHeight1);
}
NSLog(@"data %d, %d, %d", (int)data[0], (int)data[1], (int)data[2]);
transformData->drawingOGL(data);
return data;
}
在这里，transformData 是一个 c++ class 来帮助我进行翻转工作。
在函数 setOGL() 中，所有的纹理和帧缓冲区都已构建。
在函数drawingOGL()中，已经通过顶点着色器和片段着色器完成了翻转工作。下面列出了此功能的详细信息，
int loadDataFromeX::drawingOGL(unsigned char* data)
{
//load data to the texture;
glDisable(GL_DEPTH_TEST);
glBindFramebuffer(GL_FRAMEBUFFER ,transFBO.frameBuffer);
glClear(GL_COLOR_BUFFER_BIT);
glClearColor(1., 0., 0., 1.);
glViewport(0, 0, imageWidth, imageHeight);
GLfloat vertex_postions[] = {
-1.0f, -1.0f, -10.0f,
1.0f, -1.0f, -10.0f,
-1.0f, 1.0f, -10.0f,
1.0f, 1.0f, -10.0f
};
GLfloat texture_coords[] = { //left up corner is (0.0)
0.0f, 1.0f,
1.0f, 1.0f,
0.0f, 0.0f,
1.0f, 0.0f,
};
glUseProgram(gl_program_id);
glVertexAttribPointer(gl_attribute_position, 3, GL_FLOAT, GL_FALSE, 0,vertex_postions);
glEnableVertexAttribArray(gl_attribute_position);
glVertexAttribPointer(gl_attribute_texture_coordinate, 2, GL_FLOAT, GL_FALSE, 0,texture_coords);
glEnableVertexAttribArray(gl_attribute_texture_coordinate);
// Load textures
glActiveTexture(GL_TEXTURE0);
glBindTexture(GL_TEXTURE_2D, texture);
if(flag)
{
flag = false;
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, imageWidth, imageHeight, 0, GL_RGBA, GL_UNSIGNED_BYTE, data);
}
else
{
glTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, imageWidth, imageHeight, GL_RGBA, GL_UNSIGNED_BYTE, data);
}
glUniform1i(glGetUniformLocation(gl_program_id, "inputImageTexture"), 0);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glUniformMatrix4fv(mvpLocation, 1, 0, gComputeMVP);
glDrawArrays(GL_TRIANGLE_STRIP ,0 ,4);
glPixelStorei(GL_PACK_ALIGNMENT, 4);
glReadPixels(0, 0, imageWidth, imageHeight,GL_RGBA, GL_UNSIGNED_BYTE, data);
glFinish();
cout<<"data: "<<(int)data[0]<<"; "<<(int)data[1]<<", "<<(int)data[2]<<endl;
return 1;
}
下面已经提供了顶点着色器和片段着色器，
static float l = -1.f, r = 1.f;
static float b = -1.f, t = 1.f;
static float n = 0.1f, f = 100.f;
static float gComputeMVP[16] = {
2.0f/(r-l), 0.0f, 0.0f, 0.0f,
0.0f, 2.0f/(t-b), 0.0f, 0.0f,
0.0f, 0.0f, -2.0f/(f-n), 0.0f,
-(r+l)/(r-l), -(t+b)/(t-b), -(f+n)/(f-n), 1.0f
};
// Shader sources
const GLchar* vertex_shader_str =
"attribute vec4 position;\n"
"attribute vec4 inputTextureCoordinate;\n"
"varying mediump vec2 textureCoordinate;\n"
"uniform mat4 mvpMatrix;\n"
"void main()\n"
"{\n"
" gl_Position = position;\n"
" gl_Position = mvpMatrix * position;\n"
" textureCoordinate = inputTextureCoordinate.xy;\n"
"}";
const char* fragment_shader_str = ""
" varying mediump vec2 textureCoordinate;\n"
"\n"
" uniform sampler2D inputImageTexture;\n"
" \n"
" void main()\n"
" {\n"
" mediump vec4 Color = texture2D(inputImageTexture, textureCoordinate);\n"
" gl_FragColor = vec4(Color.z, Color.y, Color.x, Color.w);\n"
" }";
我不知道为什么每一帧的质量都下降了。而且，当我比较使用drawingOGL之前和之后变量，数据的输出时，如下所示的这两行，
cout<<"data: "<<(int)data[0]<<"; "<<(int)data[1]<<", "<<(int)data[2]<<endl;
NSLog(@"data %d, %d, %d", (int)data[0], (int)data[1], (int)data[2]);
第一行给出了正确的像素值。但是，第二行总是给出零。真的很奇怪，对吧？
我找到了这个奇怪问题的原因。是unity3D的context造成的。我不熟悉 unity3d。所以，也许，只有像我这样的人才会做出这种蠢事。上下文中有一些与OpenGL ES相关的特殊设置，属于unity3d。因此，为了完成快照中的任务，必须建立一个新的上下文，并且只有在快照工作时才激活它。
为了解决这个问题，我像这样为快照任务构建了一个单独的上下文（EAGLContext*），
- (void *)snapshot
{
// NSLog(@"snapshot used here");
GLint backingWidth1, backingHeight1;
glBindRenderbufferOES(GL_RENDERBUFFER_OES, mainDisplaySurface->systemColorRB);
// Get the size of the backing CAEAGLLayer
glGetRenderbufferParameterivOES(GL_RENDERBUFFER_OES, GL_RENDERBUFFER_WIDTH_OES, &backingWidth1);
glGetRenderbufferParameterivOES(GL_RENDERBUFFER_OES, GL_RENDERBUFFER_HEIGHT_OES, &backingHeight1);
NSInteger x = 0, y = 0, width = backingWidth1, height = backingHeight1;
NSInteger dataLength = width * height * 4;
GLubyte *data = (GLubyte*)malloc(dataLength * sizeof(GLubyte));
// Read pixel data from the framebuffer
glPixelStorei(GL_PACK_ALIGNMENT, 4);
glReadPixels(x, y, width, height, GL_BGRA, GL_UNSIGNED_BYTE, data);
NSLog(@"data %d, %d, %d", (int)data[0], (int)data[1], (int)data[2]);
NSLog(@"backingWidth1 : %d, backingHeight1: %d", backingWidth1, backingHeight1);
if (transformData == NULL)
{
mycontext = [[EAGLContext alloc] initWithAPI:kEAGLRenderingAPIOpenGLES2];
[EAGLContext setCurrentContext: mycontext];
NSLog(@"transformData initial");
transformData = new loadDataFromeX();
transformData->setupOGL(backingWidth1, backingHeight1);
[EAGLContext setCurrentContext: mainDisplaySurface->context];
}
{
[EAGLContext setCurrentContext: mycontext];
transformData->drawingOGL(data);
[EAGLContext setCurrentContext: mainDisplaySurface->context];
}
}
当快照使用的资源被释放时，代码是这样写的，
if (transformData != NULL)
{
{
[EAGLContext setCurrentContext: mycontext];
transformData->destroy();
delete transformData;
transformData = NULL;
[EAGLContext setCurrentContext: mainDisplaySurface->context];
}
[mycontext release];
mycontext = nil;
}
全部。 现在我正在使用unity3D开发游戏。我想通过 AVFoundation 将每一帧的内容保存为 mp4 文件。但是我在处理快照时遇到了一些问题。在我使用 glReadPixels 获取保存在渲染缓冲区中的数据后，顶点着色器和片段着色器用于帮助我将像素内容的更新面朝下。但是，在翻转每一帧之后，我发现每一帧的质量都下降了很多。所以，任何人以前都遇到过这种情况。 这是相关的代码。 快照部分，
- (void *)snapshot
{
// NSLog(@"snapshot used here");
GLint backingWidth1, backingHeight1;
glBindRenderbufferOES(GL_RENDERBUFFER_OES, mainDisplaySurface->systemColorRB);
// Get the size of the backing CAEAGLLayer
glGetRenderbufferParameterivOES(GL_RENDERBUFFER_OES, GL_RENDERBUFFER_WIDTH_OES, &backingWidth1);
glGetRenderbufferParameterivOES(GL_RENDERBUFFER_OES, GL_RENDERBUFFER_HEIGHT_OES, &backingHeight1);
NSInteger x = 0, y = 0, width = backingWidth1, height = backingHeight1;
NSInteger dataLength = width * height * 4;
GLubyte *data = (GLubyte*)malloc(dataLength * sizeof(GLubyte));
// Read pixel data from the framebuffer
glPixelStorei(GL_PACK_ALIGNMENT, 4);
glReadPixels(x, y, width, height, GL_RGBA, GL_UNSIGNED_BYTE, data);
if (transformData == NULL)
{
NSLog(@"transformData initial");
transformData = new loadDataFromeX();
transformData->setupOGL(backingWidth1, backingHeight1);
}
NSLog(@"data %d, %d, %d", (int)data[0], (int)data[1], (int)data[2]);
transformData->drawingOGL(data);
return data;
}
在这里，transformData 是一个 c++ class 来帮助我进行翻转工作。 在函数 setOGL() 中，所有的纹理和帧缓冲区都已构建。 在函数drawingOGL()中，已经通过顶点着色器和片段着色器完成了翻转工作。下面列出了此功能的详细信息，
int loadDataFromeX::drawingOGL(unsigned char* data)
{
//load data to the texture;
glDisable(GL_DEPTH_TEST);
glBindFramebuffer(GL_FRAMEBUFFER ,transFBO.frameBuffer);
glClear(GL_COLOR_BUFFER_BIT);
glClearColor(1., 0., 0., 1.);
glViewport(0, 0, imageWidth, imageHeight);
GLfloat vertex_postions[] = {
-1.0f, -1.0f, -10.0f,
1.0f, -1.0f, -10.0f,
-1.0f, 1.0f, -10.0f,
1.0f, 1.0f, -10.0f
};
GLfloat texture_coords[] = { //left up corner is (0.0)
0.0f, 1.0f,
1.0f, 1.0f,
0.0f, 0.0f,
1.0f, 0.0f,
};
glUseProgram(gl_program_id);
glVertexAttribPointer(gl_attribute_position, 3, GL_FLOAT, GL_FALSE, 0,vertex_postions);
glEnableVertexAttribArray(gl_attribute_position);
glVertexAttribPointer(gl_attribute_texture_coordinate, 2, GL_FLOAT, GL_FALSE, 0,texture_coords);
glEnableVertexAttribArray(gl_attribute_texture_coordinate);
// Load textures
glActiveTexture(GL_TEXTURE0);
glBindTexture(GL_TEXTURE_2D, texture);
if(flag)
{
flag = false;
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, imageWidth, imageHeight, 0, GL_RGBA, GL_UNSIGNED_BYTE, data);
}
else
{
glTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, imageWidth, imageHeight, GL_RGBA, GL_UNSIGNED_BYTE, data);
}
glUniform1i(glGetUniformLocation(gl_program_id, "inputImageTexture"), 0);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glUniformMatrix4fv(mvpLocation, 1, 0, gComputeMVP);
glDrawArrays(GL_TRIANGLE_STRIP ,0 ,4);
glPixelStorei(GL_PACK_ALIGNMENT, 4);
glReadPixels(0, 0, imageWidth, imageHeight,GL_RGBA, GL_UNSIGNED_BYTE, data);
glFinish();
cout<<"data: "<<(int)data[0]<<"; "<<(int)data[1]<<", "<<(int)data[2]<<endl;
return 1;
}
下面已经提供了顶点着色器和片段着色器，
static float l = -1.f, r = 1.f;
static float b = -1.f, t = 1.f;
static float n = 0.1f, f = 100.f;
static float gComputeMVP[16] = {
2.0f/(r-l), 0.0f, 0.0f, 0.0f,
0.0f, 2.0f/(t-b), 0.0f, 0.0f,
0.0f, 0.0f, -2.0f/(f-n), 0.0f,
-(r+l)/(r-l), -(t+b)/(t-b), -(f+n)/(f-n), 1.0f
};
// Shader sources
const GLchar* vertex_shader_str =
"attribute vec4 position;\n"
"attribute vec4 inputTextureCoordinate;\n"
"varying mediump vec2 textureCoordinate;\n"
"uniform mat4 mvpMatrix;\n"
"void main()\n"
"{\n"
" gl_Position = position;\n"
" gl_Position = mvpMatrix * position;\n"
" textureCoordinate = inputTextureCoordinate.xy;\n"
"}";
const char* fragment_shader_str = ""
" varying mediump vec2 textureCoordinate;\n"
"\n"
" uniform sampler2D inputImageTexture;\n"
" \n"
" void main()\n"
" {\n"
" mediump vec4 Color = texture2D(inputImageTexture, textureCoordinate);\n"
" gl_FragColor = vec4(Color.z, Color.y, Color.x, Color.w);\n"
" }";
我不知道为什么每一帧的质量都下降了。而且，当我比较使用drawingOGL之前和之后变量，数据的输出时，如下所示的这两行，
cout<<"data: "<<(int)data[0]<<"; "<<(int)data[1]<<", "<<(int)data[2]<<endl;
NSLog(@"data %d, %d, %d", (int)data[0], (int)data[1], (int)data[2]);
第一行给出了正确的像素值。但是，第二行总是给出零。真的很奇怪，对吧？
我找到了这个奇怪问题的原因。是unity3D的context造成的。我不熟悉 unity3d。所以，也许，只有像我这样的人才会做出这种蠢事。上下文中有一些与OpenGL ES相关的特殊设置，属于unity3d。因此，为了完成快照中的任务，必须建立一个新的上下文，并且只有在快照工作时才激活它。
为了解决这个问题，我像这样为快照任务构建了一个单独的上下文（EAGLContext*），
- (void *)snapshot
{
// NSLog(@"snapshot used here");
GLint backingWidth1, backingHeight1;
glBindRenderbufferOES(GL_RENDERBUFFER_OES, mainDisplaySurface->systemColorRB);
// Get the size of the backing CAEAGLLayer
glGetRenderbufferParameterivOES(GL_RENDERBUFFER_OES, GL_RENDERBUFFER_WIDTH_OES, &backingWidth1);
glGetRenderbufferParameterivOES(GL_RENDERBUFFER_OES, GL_RENDERBUFFER_HEIGHT_OES, &backingHeight1);
NSInteger x = 0, y = 0, width = backingWidth1, height = backingHeight1;
NSInteger dataLength = width * height * 4;
GLubyte *data = (GLubyte*)malloc(dataLength * sizeof(GLubyte));
// Read pixel data from the framebuffer
glPixelStorei(GL_PACK_ALIGNMENT, 4);
glReadPixels(x, y, width, height, GL_BGRA, GL_UNSIGNED_BYTE, data);
NSLog(@"data %d, %d, %d", (int)data[0], (int)data[1], (int)data[2]);
NSLog(@"backingWidth1 : %d, backingHeight1: %d", backingWidth1, backingHeight1);
if (transformData == NULL)
{
mycontext = [[EAGLContext alloc] initWithAPI:kEAGLRenderingAPIOpenGLES2];
[EAGLContext setCurrentContext: mycontext];
NSLog(@"transformData initial");
transformData = new loadDataFromeX();
transformData->setupOGL(backingWidth1, backingHeight1);
[EAGLContext setCurrentContext: mainDisplaySurface->context];
}
{
[EAGLContext setCurrentContext: mycontext];
transformData->drawingOGL(data);
[EAGLContext setCurrentContext: mainDisplaySurface->context];
}
}
当快照使用的资源被释放时，代码是这样写的，
if (transformData != NULL)
{
{
[EAGLContext setCurrentContext: mycontext];
transformData->destroy();
delete transformData;
transformData = NULL;
[EAGLContext setCurrentContext: mainDisplaySurface->context];
}
[mycontext release];
mycontext = nil;
}