同时使用 AVComposition 和 AVVideoComposition 时转换错误

Question

我正在创建一个 AVMutableComposition。我需要水平翻转资产，所以我设置合成轨道的变换是这样的：

compositionTrack.preferredTransform = assetTrack.preferredTransform.scaledBy(x: -1, y: 1)

如果我导出它（我使用 AVAssetPreset960x640 作为我的预设），它会按预期工作。

但是，我还需要添加一个 AVMutableVideoComposition 叠加层以使用此 copmosition 进行渲染。此叠加层不应水平翻转。我这样指定：

// Create video composition
let videoComposition = AVMutableVideoComposition()
videoComposition.renderSize = videoSize
videoComposition.frameDuration = CMTime(value: 1, timescale: 30)
videoComposition.animationTool = AVVideoCompositionCoreAnimationTool(
    postProcessingAsVideoLayer: videoLayer,
    in: outputLayer
)

let instruction = AVMutableVideoCompositionInstruction()
instruction.timeRange = CMTimeRange(
  start: .zero,
  duration: composition.duration
)
let layerInstruction = AVMutableVideoCompositionLayerInstruction(assetTrack: compositionTrack)
layerInstruction.setTransform(assetTrack.preferredTransform, at: .zero)
    
instruction.layerInstructions = [layerInstruction]
videoComposition.instructions = [instruction]

当我用这个导出视频时，它没有水平翻转。它停止执行应用于组合的 preferredTransform。大概是这条线导致了这个：

layerInstruction.setTransform(assetTrack.preferredTransform, at: .zero)

如果我将图层转换设置为 assetTrack.preferredTransform.scaledBy(x: -1, y: 1) 或 compositionTrack.preferredTransform，我在导出时会看到黑屏。

Apple these docs 解释了 AVVideoComposition 上的变换。如果我理解正确，他们说我应该只是设置图层指令的转换。如果我这样做 – 对层指令而不是合成应用变换 – 我仍然看到黑屏。

为什么会这样，我该如何解决？

Answer 1

编辑：有效的 t运行sform 是

CGAffineTransform(a: -1.0, b: 0.0, c: 0.0, d: 1.0, tx: videoSize.width, ty: 0.0)

所以我运行不久前关注这个问题，关注这个 RayWenderlich tutorial（您的代码看起来也非常熟悉！）。问题是 assetTrack.preferredTransform 不可靠，在某些视频上它可以工作，而在其他视频上它会黑屏。

而不是

let layerInstruction = AVMutableVideoCompositionLayerInstruction(assetTrack: compositionTrack)
layerInstruction.setTransform(assetTrack.preferredTransform, at: .zero)

试试这个：

func compositionLayerInstruction(for track: AVCompositionTrack, assetTrack: AVAssetTrack, orientation: UIImage.Orientation) -> AVMutableVideoCompositionLayerInstruction {

    let instruction = AVMutableVideoCompositionLayerInstruction(assetTrack: track)
    
    var transform = CGAffineTransform.identity
    let assetSize = assetTrack.naturalSize
    
    /// you should be able to play with these values to make a horizontal flip (try changing `a` and `d`)
    switch orientation {
    case .up:
        transform = CGAffineTransform(a: 1, b: 0, c: 0, d: 1, tx: 0, ty: 0)
    case .down:
        transform = CGAffineTransform(a: -1, b: 0, c: 0, d: -1, tx: assetSize.width, ty: assetSize.height)
    case .left:
        transform = CGAffineTransform(a: 0, b: -1, c: 1, d: 0, tx: 0, ty: assetSize.width)
    case .right:
        transform = CGAffineTransform(a: 0, b: 1, c: -1, d: 0, tx: assetSize.height, ty: 0)
     default:
        print("Unsupported orientation")
    }
    
    instruction.setTransform(transform, at: .zero)
    
    return instruction
}

let layerInstruction = compositionLayerInstruction(
    for: compositionTrack, assetTrack: assetTrack, orientation: videoInfo.orientation)
/// assetTrack is the original video

我基于 SO 上的另一个答案，但不记得在哪里。

这里有一个辅助函数，可以得到orientation

/// adjust the video orientation is the source has a different orientation
private func orientation(from transform: CGAffineTransform) -> (orientation: UIImage.Orientation, isPortrait: Bool) {
    var assetOrientation = UIImage.Orientation.up
    var isPortrait = false
    if transform.a == 0 && transform.b == 1.0 && transform.c == -1.0 && transform.d == 0 {
        assetOrientation = .right
        isPortrait = true
    } else if transform.a == 0 && transform.b == -1.0 && transform.c == 1.0 && transform.d == 0 {
        assetOrientation = .left
        isPortrait = true
    } else if transform.a == 1.0 && transform.b == 0 && transform.c == 0 && transform.d == 1.0 {
        assetOrientation = .up
    } else if transform.a == -1.0 && transform.b == 0 && transform.c == 0 && transform.d == -1.0 {
        assetOrientation = .down
    }
    return (assetOrientation, isPortrait)
}

完整用法：

/// get adjusted orientation of the video
let videoInfo = orientation(from: assetTrack.preferredTransform)
let videoSize: CGSize

if videoInfo.isPortrait {
    videoSize = CGSize(
    width: assetTrack.naturalSize.height,
    height: assetTrack.naturalSize.width)
} else {
    videoSize = assetTrack.naturalSize
}


 /// the video
let videoLayer = CALayer()
videoLayer.frame = CGRect(origin: .zero, size: videoSize)
        
/// overlay where you add graphics and other stuff
let overlayLayer = CALayer()
overlayLayer.frame = CGRect(origin: .zero, size: videoSize)

let outputLayer = CALayer()
outputLayer.frame = CGRect(origin: .zero, size: videoSize)
outputLayer.addSublayer(videoLayer)
outputLayer.addSublayer(overlayLayer)

let videoComposition = AVMutableVideoComposition()
videoComposition.renderSize = videoSize
videoComposition.frameDuration = CMTime(value: 1, timescale: 30)
videoComposition.animationTool = AVVideoCompositionCoreAnimationTool(
postProcessingAsVideoLayer: videoLayer,
in: outputLayer)

// MARK: - previous `let layerInstruction` code goes here...

instruction.layerInstructions = [layerInstruction]

/// export session code here...

同时使用 AVComposition 和 AVVideoComposition 时转换错误

Transform wrong when using both AVComposition and AVVideoComposition

avfoundation

ios

avcomposition

swift