如何将 AVAsset 转换为 CMSampleBuffer 帧？

Question

我的应用导入用户通过系统文件选择器选择的视频，它作为 AVAsset:

@IBAction func handleImportVideoButton(_ sender: Any) {
    let documentPicker = UIDocumentPickerViewController(forOpeningContentTypes: [.movie], asCopy: true)
    documentPicker.delegate = self
    present(documentPicker, animated: true)
}

// UIDocumentPickerDelegate callback.
func documentPicker(_ controller: UIDocumentPickerViewController, didPickDocumentsAt urls: [URL]) {
    guard let url = urls.first else {
        return
    }
    model.recordedVideoSource = AVAsset(url: url)
}

然后如何将此 AVAsset 转换为 CMSampleBuffer 帧？最终目标是将 CMSampleBuffer 帧转换为 CGImage，这样我就可以考虑对每个图像帧执行机器学习分析。

Answer 1

这是未经测试的，但它应该给你如何去做的要点：

let asset = AVAsset()
let reader = AVAssetReader(asset: asset)
guard let track = asset.tracks(withMediaType: .video).last else {
    return
}
let trackOutput = AVAssetReaderTrackOutput(track: track, outputSettings: nil)
reader.add(trackOutput)
reader.startReading()

// Get first sample buffer
var sample = trackOutput.copyNextSampleBuffer()
while sample != nil {
    // iterate over all buffers
    // sample = trackOutput.copyNextSampleBuffer()
}

如何将 AVAsset 转换为 CMSampleBuffer 帧？

How do you convert an AVAsset into CMSampleBuffer frames?

core-foundation

avfoundation

ios

swift