python 如何在 Tensorflow 中创建我自己的预处理层？

Question

我有一个用于预处理数据的特定顺序模型，如下所示：

data_transformation = tf.keras.Sequential([layers.experimental.preprocessing.RandomContrast(factor=(0.7,0.9)),layers.GaussianNoise(stddev=tf.random.uniform(shape=(),minval=0, maxval=1)), layers.experimental.preprocessing.RandomRotation(factor=0.1, fill_mode='reflect', interpolation='bilinear', seed=None, name=None, fill_value=0.0), layers.experimental.preprocessing.RandomZoom(height_factor=(0.1,0.2), width_factor=(0.1,0.2), fill_mode='reflect', interpolation='bilinear', seed=None, name=None, fill_value=0.0),])

但是，我想添加我自己的预处理层，它由下面的 Python 函数定义：

import tensorflow as tf
import random

def my_random_contrast(image_to_be_transformed, contrast_factor):

    #build the contrast factor 
    selected_contrast_factor=random.uniform(1-contrast_factor, 1+contrast_factor)
    
    selected_contrast_factor_c1=selected_contrast_factor
    selected_contrast_factor_c2=selected_contrast_factor-0.01
    selected_contrast_factor_c3=selected_contrast_factor-0.02
    
    
    image_to_be_transformed=image_to_be_transformed.numpy()
    image_to_be_transformed[0,:,:]=((image_to_be_transformed[0,:,:]-tf.reduce_mean(image_to_be_transformed[0,:,:]))*selected_contrast_factor_c1)+tf.reduce_mean(image_to_be_transformed[0,:,:])
    image_to_be_transformed[1,:,:]=((image_to_be_transformed[1,:,:]-tf.reduce_mean(image_to_be_transformed[1,:,:]))*selected_contrast_factor_c2)+tf.reduce_mean(image_to_be_transformed[1,:,:])
    image_to_be_transformed[2,:,:]=((image_to_be_transformed[2,:,:]-tf.reduce_mean(image_to_be_transformed[2,:,:]))*selected_contrast_factor_c3)+tf.reduce_mean(image_to_be_transformed[2,:,:])
    
    image_to_be_transformed=tf.convert_to_tensor(image_to_be_transformed)
    return image_to_be_transformed


x=tf.random.uniform(shape=[3,224,224], minval=0, maxval=1, dtype=tf.float32)
y=my_random_contrast(x, 0.5)

我如何使用 TensorFlow 做到这一点？作为一个新的预处理层，它将接收来自其他层的输入和输出，我是否必须保证输入和输出是给定的类型？

Answer 1

您需要通过子class层class创建自定义层，可以在此处找到示例Making new Layers and Models via subclassing。网上可以找到很多例子。

Answer 2

如果您计划一次预处理多个图像，则需要确保您的方法适用于批量维度。此外，它应该能够在没有任何 numpy 操作的情况下工作，以便在图形模式下运行。然后，您可以编写自己的自定义图层或简单的 lambda 图层：

import tensorflow as tf
import matplotlib.pyplot as plt
import pathlib

dataset_url = "https://storage.googleapis.com/download.tensorflow.org/example_images/flower_photos.tgz"
data_dir = tf.keras.utils.get_file('flower_photos', origin=dataset_url, untar=True)
data_dir = pathlib.Path(data_dir)

ds = tf.keras.utils.image_dataset_from_directory(
  data_dir,
  image_size=(180, 180),
  batch_size=32, shuffle=False)

def my_random_contrast(image_to_be_transformed, contrast_factor):

    #build the contrast factor 
    selected_contrast_factor=tf.random.uniform((), minval=1 - contrast_factor, maxval=1 + contrast_factor)
    
    selected_contrast_factor_c1=selected_contrast_factor
    selected_contrast_factor_c2=selected_contrast_factor-0.01
    selected_contrast_factor_c3=selected_contrast_factor-0.02
    
    c0 = ((image_to_be_transformed[:,:,:, 0]-tf.reduce_mean(image_to_be_transformed[:,:,:, 0]))*selected_contrast_factor_c1)+tf.reduce_mean(image_to_be_transformed[:,:,:, 0])
    c1 = ((image_to_be_transformed[:,:,:, 1]-tf.reduce_mean(image_to_be_transformed[:,:,:, 1]))*selected_contrast_factor_c2)+tf.reduce_mean(image_to_be_transformed[:,:,:, 1])
    c2 = ((image_to_be_transformed[:,:,:, 2]-tf.reduce_mean(image_to_be_transformed[:,:,:, 2]))*selected_contrast_factor_c3)+tf.reduce_mean(image_to_be_transformed[:,:,:, 2])

    image_to_be_transformed = tf.concat([c0[..., tf.newaxis], image_to_be_transformed[:,:,:, 1:]], axis=-1)
    image_to_be_transformed = tf.concat([image_to_be_transformed[:,:,:, 0][..., tf.newaxis], c1[..., tf.newaxis], image_to_be_transformed[:,:,:, 2][..., tf.newaxis]], axis=-1)
    image_to_be_transformed = tf.concat([image_to_be_transformed[:,:,:, :2], c2[..., tf.newaxis]], axis=-1)

    return image_to_be_transformed


images, _ = next(iter(ds.take(1)))
image = images[2]
plt.figure()
f, axarr = plt.subplots(1,2) 
axarr[0].imshow(image / 255)

# After preprocessing dataset:
images, _ = next(iter(ds.map(lambda x, y: (my_random_contrast(x, 0.8), y)).take(1)))
image = images[2]
axarr[1].imshow(image / 255)

用法示例：

model = tf.keras.Sequential()
model.add(tf.keras.layers.Lambda(lambda x: my_random_contrast(x, 0.8)))
model.add(tf.keras.layers.Flatten())
model.add(tf.keras.layers.Dense(1))
model.compile(optimizer='adam', loss='mse')
model.fit(tf.random.normal((50, 64, 64, 3)), tf.random.normal((50, 1)))

python 如何在 Tensorflow 中创建我自己的预处理层？

How to create my own preprocessing layer in Tensorflow in python?

python

keras

tensorflow