如何使用 tensorflow/keras 中的预定内核列表初始化 Conv2D 层？

Question

我想使用 Conv2D 层来跨越输入图像和运行三个 2x2 内核。

这不是 tensorflow 的目的，但我真的想使用 tensorflow 作为后端引擎来有效地运行内核并在不同设备 GPU and/or CPU 之间分配工作负载。

我试过类似下面的代码。但是好像效果不是很好

import tensorflow as tf

class InitConvKernels(tf.keras.initializers.Initializer):

  def __init__(self, num_kernels, kernel_tensor):
    self.kernel_list= kernel_tensor
    self.index = -1
    self.num_kernels = num_kernels

  def __call__(self, shape, dtype=None):
    index += 1 
    assert(self.index <= self.num_kernels) # doesn't affect anything
    tf.print(shape) # doesn't work
    return self.kernel_list[index]

  def get_config(self):
    return {'kernel_list': self.kernel_list, 'num_kernels': self.num_kernels}

我正在调用自定义初始化程序，但返回的层是空的：

kernel_list = tf.constant([[[-1, -1],  [-1, -1]], [[1, 1],   [1, 1]],  [[-1, 1],  [1, -1]],])
layer = layers.Conv2D(
    filters=3,
    kernel_size=2,
    kernel_initializer=InitConvKernels(3,kernel_list),
    bias_initializer=initializers.Zeros()
)

layer.variables 为空 ([]) layer.layer.get_weights() 也为空 ([])

我的目标是在输入图像上评估 kernel_list 中的三个内核的卷积并汇总所有结果。

Answer 1

from PIL import Image
import requests
from io import BytesIO
import numpy as np
from keras.models import Sequential
from keras.layers import Conv2D


response = requests.get('https://upload.wikimedia.org/wikipedia/commons/thumb/0/02/Stack_Overflow_logo.svg/1280px-Stack_Overflow_logo.svg.png')
image = Image.open(BytesIO(response.content))

正在加载 image from url。

构建一个模型运行（运行更多内核使 kernel_init 生成器并在初始化时轻松调整过滤器的数量 Conv2D)

def kernel_init(shape, dtype=None, partition_info=None):
    kernel = np.zeros(shape)
    kernel[:,:,0,0] = np.array([[1,0,1],[-1,0,-1],[1,0,1]])
    return kernel

#Build Keras model
model = Sequential()
model.add(Conv2D(1, [3,3], kernel_initializer=kernel_init, 
                 input_shape=(251,1280,4), padding="valid"))
model.build()

# To apply existing filter, we use predict with no training
out = model.predict(image)

并可视化输出：

import matplotlib.pyplot as plt
plt.matshow(out[0,:,:,0])

编辑：值得一提的是 OpenAI's Triton，它可以帮助使用更高级的语言和框架（例如 pytorch）来运行高效的 GPU 代码：

Python-like programming language which enables researchers with no CUDA experience to write highly efficient GPU code—most of the time on par with what an expert would be able to produce.

如何使用 tensorflow/keras 中的预定内核列表初始化 Conv2D 层？

How to initialize a Conv2D layer with predetermined list of kernels in tensorflow/keras?

python

tensorflow

keras

convolution