iOS

Question

我一直在尝试将 .h264 视频解码为 opengl，但我显示的帧是黑色的。我没有看到任何错误，如果我将帧从 CMSampleBufferRef 导出到相机胶卷，它们就很好。

那可能是在 openGL 方面？但是当我显示图像而不是视频时，它工作正常，所以不知道从哪里看。

这是初始化视频解码器的代码：

- (void)initVideo {

    glActiveTexture(GL_TEXTURE0);
    glGenTextures(1, &_glTextureHook);

    NSURL * url = [NSURL URLWithString:[self.videoMedia getFilePath]];
    self.videoAsset = [[AVURLAsset alloc] initWithURL:url options:NULL];

    dispatch_semaphore_t sema = dispatch_semaphore_create(0);

    [self.videoAsset loadValuesAsynchronouslyForKeys:@[@"tracks"] completionHandler:^{

        self.videoTrack = [self.videoAsset tracksWithMediaType:AVMediaTypeVideo][0];

        NSString *key = (NSString *) kCVPixelBufferPixelFormatTypeKey;
        NSNumber *value = @(kCVPixelFormatType_32BGRA);
        NSDictionary *settings = @{key : value};

        self.outputVideoTrackOuput = [[AVAssetReaderTrackOutput alloc]
                                      initWithTrack:self.videoTrack outputSettings:settings];

        self.assetReader = [[AVAssetReader alloc] initWithAsset:self.videoAsset error:nil];
        [self.assetReader addOutput:self.outputVideoTrackOuput];
        [self.assetReader startReading];

        dispatch_semaphore_signal(sema);
    }];

    dispatch_semaphore_wait(sema, DISPATCH_TIME_FOREVER);
}

以及检索 opengl 纹理的代码，视频的每一帧：

- (void)play {

    if (self.assetReader.status == AVAssetReaderStatusReading) {
        CMSampleBufferRef sampleBuffer = [self.outputVideoTrackOuput copyNextSampleBuffer];
        if(sampleBuffer) {
            CVImageBufferRef pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);

            if(pixelBuffer) {
                CVPixelBufferLockBaseAddress(pixelBuffer, 0);

                glBindTexture(GL_TEXTURE_2D, _glTextureHook);
                glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, 1920, 1080, 0, GL_BGRA_EXT, GL_UNSIGNED_BYTE, CVPixelBufferGetBaseAddress(pixelBuffer));
                CVPixelBufferUnlockBaseAddress(pixelBuffer, 0);
            }

            CFRelease(sampleBuffer);
        }
    }
}

图像和视频的顶点着色器和片段相同（适用于图像）。我看到的唯一区别是在 glTexImage2D 中，图像的两种格式都是 GL_RGBA。

我确保视频解码器的 _glTextureHook 已正确发送到着色器管理器，上下文已在线程上激活等。

这是片段着色器的代码（非常基础，顶点着色器也一样）：

precision lowp float;

uniform sampler2D Texture;

varying vec4 DestinationColor;
varying vec2 TexCoordOut;

void main() {
    gl_FragColor = DestinationColor * texture2D(Texture, TexCoordOut);
}

Answer 1

我只是在创建纹理后错过了那些线条：

glBindTexture(GL_TEXTURE_2D, _glTextureHook);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);

iOS - 使用 OpenGL ES 2.0 进行视频解码

iOS - Video decoding with OpenGL ES 2.0

video

opengl-es

objective-c

opengl-es-2.0