尝试合并多个 video/audio 剪辑时，AVAssetExportSession 导致音频视频不同步

Question

我正在尝试将一堆包含音频和视频的视频剪辑合并到一个较长的视频文件中。
AVMutableComposition *mainComposition = [[AVMutableComposition alloc] init]; AVMutableCompositionTrack *compositionVideoTrack = [mainComposition addMutableTrackWithMediaType:AVMediaTypeVideo preferredTrackID:kCMPersistentTrackID_Invalid];

AVMutableCompositionTrack *soundtrackTrack = [mainComposition addMutableTrackWithMediaType:AVMediaTypeAudio preferredTrackID:kCMPersistentTrackID_Invalid];
CMTime insertTime = kCMTimeZero;


NSArray *videoFileNames;
for (NSString  *videoPathString in videoFileNames) {

    AVAsset *videoAsset = [AVAsset assetWithURL:[NSURL fileURLWithPath:[NSString stringWithFormat:@"/Volumes/videoClips/CLIPS/%@",videoPathString]]];

                NSError *err1;
                NSError *err2;
                [compositionVideoTrack insertTimeRange:CMTimeRangeMake(kCMTimeZero, videoAsset.duration) ofTrack:[[videoAsset tracksWithMediaType:AVMediaTypeVideo] objectAtIndex:0] atTime:insertTime error:&err1];

                [soundtrackTrack insertTimeRange:CMTimeRangeMake(kCMTimeZero, videoAsset.duration) ofTrack:[[videoAsset tracksWithMediaType:AVMediaTypeAudio] objectAtIndex:0] atTime:insertTime error:&err2];
                insertTime = CMTimeAdd(insertTime, videoAsset.duration);
}


NSURL *outptVideoUrl = [NSURL fileURLWithPath:@"/Volumes/mergedVideo/finalizedMoviev.mp4"];
AVAssetExportSession *exporter = [[AVAssetExportSession alloc] initWithAsset:mainComposition presetName:AVAssetExportPreset1280x720];

exporter.outputURL= outptVideoUrl;
exporter.outputFileType = AVFileTypeMPEG4;   
exporter.shouldOptimizeForNetworkUse = NO;
[exporter exportAsynchronouslyWithCompletionHandler:^{
    dispatch_async(dispatch_get_main_queue(), ^{
        NSLog(@"Completed video export");
    });
}];`

这有效，但是在第一个剪辑之后音频和视频之间存在不同步。 Video if you watch the first clip the audio is perfectly synced then if you skip ahead towards the 15 minute mark you will notice a definite desync between video and audio. After a little bit of searching I was able to this postiOS AVFoundation audio/video out of sync

里面有一个 link 解释了压缩时音频查询空白样本是怎么回事。在压缩音频文件的正面和背面。技术说明 TN2258 AAC 音频 - 编码器延迟和同步

现在我想我能做的是解决这个问题，设置一个 AVAssetReader，通过音频和视频帧的每个 CMSamplebuffer 读取视频和音频输出，然后解压缩它们，然后将其放入 AVAssetWritter 中，并为每个重复此操作夹子。但我不确定的是如何始终保持音频和视频同步。我只是获取每个音频帧并将其按原样添加，还是我正在尝试 trim 删除空白启动和剩余数据包帧？

感谢您的帮助！

Answer 1

我的假设是正确的，通过单独查看每个帧并通过 avassetwriter 编写它们确实有效。但是，如果剪辑的音频采样率不同，您运行就会遇到麻烦，这就是我现在正在解决的新问题。

尝试合并多个 video/audio 剪辑时，AVAssetExportSession 导致音频视频不同步

AVAssetExportSession causes audio video to desync when trying to merge multiple video/audio clips

objective-c

avfoundation

avassetwriter

avassetexportsession

audio-video-sync