来自 TensorFlow Probability 混合密度网络的混合参数

Question

如何从使用 TensorFlow Probability 创建的混合密度网络中获取混合参数？

我正在尝试学习一些有关混合密度网络的知识，并在 TensorFlow Probability 文档中遇到了一个示例 here。顺便说一句，我是这方面的初学者。

使用上面的示例作为起点，请参阅下面的完整代码。我不得不对原来的AdamOptimizer进行了更改，并在最后添加了一个model.predict()。调用 predict(X) 似乎是从条件分布 P(Y|X) 中抽取样本，但我想获取混合模型的参数以提供 X 的值，即权重、均值和标准偏差每个 num_components 混合成分。有什么想法吗？

我已经看到 MixtureNormal 层的 convert_to_tensor_fn 参数，并尝试添加：

convert_to_tensor_fn=tfp.distributions.Distribution.sample - 确认 predict() 抽样

和

convert_to_tensor_fn=tfp.distributions.Distribution.mean - 看起来像 predict() returns 条件期望

所以我当时希望有一些其他的选择来获取混合物成分，但到目前为止我还没有找到它。

import tensorflow as tf
import tensorflow_probability as tfp
import numpy as np

tfd = tfp.distributions
tfpl = tfp.layers
tfk = tf.keras
tfkl = tf.keras.layers

# Load data -- graph of a [cardioid](https://en.wikipedia.org/wiki/Cardioid).
n = 2000
t = tfd.Uniform(low=-np.pi, high=np.pi).sample([n, 1])
r = 2 * (1 - tf.cos(t))
x = r * tf.sin(t) + tfd.Normal(loc=0., scale=0.1).sample([n, 1])
y = r * tf.cos(t) + tfd.Normal(loc=0., scale=0.1).sample([n, 1])

# Model the distribution of y given x with a Mixture Density Network.
event_shape = [1]
num_components = 5
params_size = tfpl.MixtureNormal.params_size(num_components, event_shape)
model = tfk.Sequential([
  tfkl.Dense(12, activation='relu'),
  tfkl.Dense(params_size, activation=None),
  tfpl.MixtureNormal(num_components=num_components,           
    event_shape=event_shape
  )
])

# Fit.
batch_size = 100
epochs=20

#model.compile(optimizer=tf.train.AdamOptimizer(learning_rate=0.02),
#              loss=lambda y, model: -model.log_prob(y))
model.compile(optimizer=tf.optimizers.Adam(learning_rate=0.02), 
    loss=lambda y, model: -model.log_prob(y))

history = model.fit(x, y,
          batch_size=batch_size,
          epochs=epochs,
          steps_per_epoch=n // batch_size)

#
# use the model to make prediction (draws samples from the conditional distribution)
# but how do you get to the mixture parameters for each value of x_pred???
#
x_pred = tf.convert_to_tensor(np.linspace(-2.7,+2.7,1000))
y_pred = model.predict(x_pred)

既然有了答案，完整代码如下：

import tensorflow as tf
import tensorflow_probability as tfp
import numpy as np

tfd = tfp.distributions
tfpl = tfp.layers
tfk = tf.keras
tfkl = tf.keras.layers

# Load data -- graph of a [cardioid](https://en.wikipedia.org/wiki/Cardioid).
n = 2000
t = tfd.Uniform(low=-np.pi, high=np.pi).sample([n, 1])
r = 2 * (1 - tf.cos(t))
x = r * tf.sin(t) + tfd.Normal(loc=0., scale=0.1).sample([n, 1])
y = r * tf.cos(t) + tfd.Normal(loc=0., scale=0.1).sample([n, 1])

# Model the distribution of y given x with a Mixture Density Network.
event_shape = [1]
num_components = 5
params_size = tfpl.MixtureNormal.params_size(num_components, event_shape)
model = tfk.Sequential([
  tfkl.Dense(12, activation='relu'),
  tfkl.Dense(params_size, activation=None),
  tfpl.MixtureNormal(num_components=num_components,           
    event_shape=event_shape
  )
])

# Fit.
batch_size = 100
epochs=20

#model.compile(optimizer=tf.train.AdamOptimizer(learning_rate=0.02),
#              loss=lambda y, model: -model.log_prob(y))
model.compile(optimizer=tf.optimizers.Adam(learning_rate=0.02), 
    loss=lambda y, model: -model.log_prob(y))

history = model.fit(x, y,
          batch_size=batch_size,
          epochs=epochs,
          steps_per_epoch=n // batch_size)

#
# use the model to get parameters of the conditional distribution:
#
x = np.linspace(-2.7,+2.7,1000)
x_pred = tf.convert_to_tensor(x[:,np.newaxis])

#
# compute the mixture parameters at each x:
#
gm = model(x_pred)

#
# get the mixture parameters:
#
gm_weights = gm.mixture_distribution.probs_parameter().numpy()
gm_means = gm.components_distribution.mean().numpy()
gm_vars = gm.components_distribution.variance().numpy()

print(gm_weights)

Answer 1

我也为此苦苦挣扎。通过查看 Github (here) 上的源代码，我找到了一种获取给定输出分布参数的方法。

例如如果我有一个名为 'model' 的模型并在特定输入 'x_star' 调用它，则会返回一个分布对象 - 可以像这样访问您想要的属性：

x_star = 1
model_star = model(np.array([x_star]))
comp_weights = np.array(model_star.mixture_distribution.probs_parameter())
comp_means = np.array(model_star.components_distribution.mean())
comp_vars = np.array(model_star.components_distribution.variance())

我不确定他们为什么不宣传如何访问它。也许他们希望这些模型被用作黑匣子。

来自 TensorFlow Probability 混合密度网络的混合参数

Mixture parameters from a TensorFlow Probability mixture density network

python

neural-network

mixture-model

tensorflow