带采样器的金属核图像内核

Question

我正在尝试使用带有采样器参数的 CIColorKernel 或 CIBlendKernel，但程序崩溃了。这是我编译成功的着色器代码。

extern "C" float4 wipeLinear(coreimage::sampler t1, coreimage::sampler t2, float time) {
    float2 coord1 = t1.coord();
    float2 coord2 = t2.coord();

    float4 innerRect = t2.extent();

    float minX = innerRect.x + time*innerRect.z;
    float minY = innerRect.y + time*innerRect.w;
    float cropWidth = (1 - time) * innerRect.w;
    float cropHeight = (1 - time) * innerRect.z;

    float4 s1 = t1.sample(coord1);
    float4 s2 = t2.sample(coord2);

   if ( coord1.x > minX && coord1.x < minX + cropWidth && coord1.y > minY && coord1.y <= minY + cropHeight) {
       return s1;
   } else {
      return s2;
   }
}

初始化时崩溃。

class CIWipeRenderer: CIFilter {
var backgroundImage:CIImage?
var foregroundImage:CIImage?
var  inputTime: Float = 0.0

static var kernel:CIColorKernel = { () -> CIColorKernel in

    let url = Bundle.main.url(forResource: "AppCIKernels", withExtension: "ci.metallib")!
    let data = try! Data(contentsOf: url)
    return try! CIColorKernel(functionName: "wipeLinear", fromMetalLibraryData: data) //Crashes here!!!!
    
}()

override var outputImage: CIImage? {
    guard let backgroundImage = backgroundImage else {
        return nil
    }
    
    guard let foregroundImage = foregroundImage else {
        return nil
    }
    
    return CIWipeRenderer.kernel.apply(extent: backgroundImage.extent, arguments: [backgroundImage, foregroundImage, inputTime])
}

}

它在 try 行中崩溃并出现以下错误：

 Fatal error: 'try!' expression unexpectedly raised an error: Foundation._GenericObjCError.nilError

如果我用下面的代码替换内核代码，它就像一个魅力：

  extern "C" float4 wipeLinear(coreimage::sample_t s1, coreimage::sample_t s2, float time)
{
     return mix(s1, s2, time);
}

所以代码中没有明显的错误，比如传递了错误的函数名等等。

Answer 1

是的，您不能在 CIColorKernel 或 CIBlendKernel 中使用采样器。这些内核针对从输入像素到输出像素具有 1:1 映射的用例进行了优化。这允许 Core Image 在一个命令缓冲区中执行多个这样的内核，因为它们不需要任何中间缓冲区写入。
sampler 将允许您在任意坐标处对输入进行采样，这在这种情况下是不允许的。

您可以简单地使用 CIKernel 代替。当您需要更自由地对输入进行采样时，可以使用它。

要初始化内核，您需要像这样修改代码：

static var kernel: CIKernel = {
    let url = Bundle.main.url(forResource: "AppCIKernels", withExtension: "ci.metallib")!
let data = try! Data(contentsOf: URL)
    return try! CIKernel(functionName: "wipeLinear", fromMetalLibraryData: data)
}()

调用内核时，你现在还需要提供一个ROI回调，像这样：

let roiCallback: CIKernelROICallback = { index, rect -> CGRect in
    return rect // you need the same region from the input as the output

}
// or even shorter
let roiCallback: CIKernelROICallback = {  }
return CIWipeRenderer.kernel.apply(extent: backgroundImage.extent, roiCallback: roiCallback, arguments: [backgroundImage, foregroundImage, inputTime])

Answer 2

对于您的用例，您实际上可以使用CIColorKernel。您只需要将渲染目标的范围也传递给内核，那么您就不需要采样器来访问它了。

内核看起来像这样：

extern "C" float4 wipeLinear(coreimage::sample_t t1, coreimage::sample_t t2, float4 destinationExtent, float time, coreimage::destination destination) {
    float minX = destinationExtent.x + time * destinationExtent.z;
    float minY = destinationExtent.y + time * destinationExtent.w;
    float cropWidth = (1.0 - time) * destinationExtent.w;
    float cropHeight = (1.0 - time) * destinationExtent.z;

    float2 destCoord = destination.coord();

   if ( destCoord.x > minX && destCoord.x < minX + cropWidth && destCoord.y > minY && destCoord.y <= minY + cropHeight) {
       return t1;
   } else {
      return t2;
   }
}

你这样称呼它：

let destinationExtent = CIVector(cgRect: backgroundImage.extent)
return CIWipeRenderer.kernel.apply(extent: backgroundImage.extent, arguments: [backgroundImage, foregroundImage, destinationExtent, inputTime])

请注意，内核中的最后一个destination参数是由Core Image自动传递的。您不需要使用 arguments.

传递它

Answer 3

奖励答案：

对于这种混合效果，您实际上根本不需要任何内核。您可以通过简单的裁剪和合成来实现所有这些：

class CIWipeRenderer: CIFilter {
    var backgroundImage:CIImage?
    var foregroundImage:CIImage?
    var inputTime: CGFloat = 0.0

    override var outputImage: CIImage? {
        guard let backgroundImage = backgroundImage else { return nil }
        guard let foregroundImage = foregroundImage else { return nil }

        // crop the foreground based on time
        var foregroundCrop = foregroundImage.extent
        foregroundCrop.size.width *= inputTime
        foregroundCrop.size.height *= inputTime

        return foregroundImage.cropped(to: foregroundCrop).composited(over: backgroundImage)
    }
}

带采样器的金属核图像内核

Metal core image kernel with sampler

core-image

ios

ciimage

metal

cikernel