如何使用 Tensorflow 概率中的 Gamma 函数作为回归模型中的对数似然损失

Question

我正在尝试使用 log_prob 方法在自定义 Keras 损失函数中使用来自 tfp 的 Gamma 函数，但训练开始时该函数总是 returns nan。

我测试了损失函数，似乎工作正常：

import tensorflow as tf
import tensorflow_probability as tfp

tf.enable_eager_execution()

def gamma_loss(y_true, alpha, beta):
    gamma_distr = tfp.distributions.Gamma(concentration=alpha, rate=beta)
    log_lik_gamma = gamma_distr.log_prob(y_true)
    return -tf.reduce_mean(log_lik_gamma)

gamma_loss(100, 2, 2).numpy()
# 194.00854

问题可能与我传递给函数的参数（alpha 和 beta）有关，这些参数是由我正在使用的模型的最终（自定义）层生成的. 这是完整的片段：

import tensorflow as tf
from tensorflow.keras import backend as K
from tensorflow.keras.layers import Input, Dense, Layer, Concatenate
from tensorflow.keras.models import Model
from tensorflow.keras.initializers import glorot_normal
import tensorflow_probability as tfp
from sklearn.datasets import make_regression


class GammaLayer(Layer):
    def __init__(self, output_dim, **kwargs):
        self.output_dim = output_dim
        super(GammaLayer, self).__init__(**kwargs)

    def build(self, input_shape):
        n_weight_rows = 4

        self.kernel_2 = self.add_weight(name='kernel_2',
                                        shape=(n_weight_rows, self.output_dim),
                                        initializer=glorot_normal(),
                                        trainable=True)
        self.kernel_3 = self.add_weight(name='kernel_3',
                                        shape=(n_weight_rows, self.output_dim),
                                        initializer=glorot_normal(),
                                        trainable=True)

        self.bias_2 = self.add_weight(name='bias_2',
                                      shape=(self.output_dim,),
                                      initializer=glorot_normal(),
                                      trainable=True)
        self.bias_3 = self.add_weight(name='bias_3',
                                      shape=(self.output_dim,),
                                      initializer=glorot_normal(),
                                      trainable=True)
        super(GammaLayer, self).build(input_shape)

    def call(self, x):
        # Here i use softplus to make the parameters strictly positive
        alpha = tf.math.softplus(K.dot(x, self.kernel_2) + self.bias_2)
        beta = tf.math.softplus(K.dot(x, self.kernel_3) + self.bias_3)
        return [alpha, beta]

    def compute_output_shape(self, input_shape):
        """
        The assumption is that the output is always one-dimensional
        """
        return [(input_shape[0], self.output_dim), (input_shape[0], self.output_dim)]


def gamma_loss(y_true, y_pred):
    alpha, beta = y_pred[0], y_pred[1]
    gamma_distr = tfp.distributions.Gamma(concentration=alpha, rate=beta)
    return -tf.reduce_mean(gamma_distr.log_prob(y_true))

X, y = make_regression(n_samples=1000, n_features=3, noise=0.1)

inputs = Input(shape=(3,))
x = Dense(6, activation='relu')(inputs)
x = Dense(4, activation='relu')(x)
x = GammaLayer(1, name='main_output')(x)
output_params = Concatenate(1, name="pvec")(x)

model = Model(inputs, output_params)
model.compile(loss=gamma_loss, optimizer='adam')
model.fit(X, y, epochs=30, batch_size=10) ```

Answer 1

您可以尝试在 softplus 之外添加额外的 1e-6 左右吗？对于非常负的值，softplus 变得非常接近于零。

如何使用 Tensorflow 概率中的 Gamma 函数作为回归模型中的对数似然损失

How to use the Gamma function from Tensorflow probability as log likelihood loss in a regression model

tensorflow

distribution

gamma

tensorflow-probability