向 skflow 添加正则化器
Adding regularizer to skflow
我最近从 tensorflow 切换到 skflow。在 tensorflow 中,我们会将 lambda*tf.nn.l2_loss(weights) 添加到我们的损失中。现在我在 skflow 中有以下代码:
def deep_psi(X, y):
layers = skflow.ops.dnn(X, [5, 10, 20, 10, 5], keep_prob=0.5)
preds, loss = skflow.models.logistic_regression(layers, y)
return preds, loss
def exp_decay(global_step):
return tf.train.exponential_decay(learning_rate=0.01,
global_step=global_step,
decay_steps=1000,
decay_rate=0.005)
deep_cd = skflow.TensorFlowEstimator(model_fn=deep_psi,
n_classes=2,
steps=10000,
batch_size=10,
learning_rate=exp_decay,
verbose=True,)
如何以及在何处添加正则化项? Illia 暗示了一些事情 here 但我想不通。
你仍然可以添加额外的组件到损失中,你只需要从 dnn / logistic_regression 中检索权重并将它们添加到损失中:
def regularize_loss(loss, weights, lambda):
for weight in weights:
loss = loss + lambda * tf.nn.l2_loss(weight)
return loss
def deep_psi(X, y):
layers = skflow.ops.dnn(X, [5, 10, 20, 10, 5], keep_prob=0.5)
preds, loss = skflow.models.logistic_regression(layers, y)
weights = []
for layer in range(5): # n layers you passed to dnn
weights.append(tf.get_variable("dnn/layer%d/linear/Matrix" % layer))
# biases are also available at dnn/layer%d/linear/Bias
weights.append(tf.get_variable('logistic_regression/weights'))
return preds, regularize_loss(loss, weights, lambda)
```
注意,变量的路径可以是found here.
此外,我们想为所有具有变量的层(如 dnn
、conv2d
或 fully_connected
)添加正则化器支持,因此下周的 Tensorflow 构建应该会有一些东西像这样 dnn(.., regularize=tf.contrib.layers.l2_regularizer(lambda))
。发生这种情况时,我会更新此答案。
我最近从 tensorflow 切换到 skflow。在 tensorflow 中,我们会将 lambda*tf.nn.l2_loss(weights) 添加到我们的损失中。现在我在 skflow 中有以下代码:
def deep_psi(X, y):
layers = skflow.ops.dnn(X, [5, 10, 20, 10, 5], keep_prob=0.5)
preds, loss = skflow.models.logistic_regression(layers, y)
return preds, loss
def exp_decay(global_step):
return tf.train.exponential_decay(learning_rate=0.01,
global_step=global_step,
decay_steps=1000,
decay_rate=0.005)
deep_cd = skflow.TensorFlowEstimator(model_fn=deep_psi,
n_classes=2,
steps=10000,
batch_size=10,
learning_rate=exp_decay,
verbose=True,)
如何以及在何处添加正则化项? Illia 暗示了一些事情 here 但我想不通。
你仍然可以添加额外的组件到损失中,你只需要从 dnn / logistic_regression 中检索权重并将它们添加到损失中:
def regularize_loss(loss, weights, lambda):
for weight in weights:
loss = loss + lambda * tf.nn.l2_loss(weight)
return loss
def deep_psi(X, y):
layers = skflow.ops.dnn(X, [5, 10, 20, 10, 5], keep_prob=0.5)
preds, loss = skflow.models.logistic_regression(layers, y)
weights = []
for layer in range(5): # n layers you passed to dnn
weights.append(tf.get_variable("dnn/layer%d/linear/Matrix" % layer))
# biases are also available at dnn/layer%d/linear/Bias
weights.append(tf.get_variable('logistic_regression/weights'))
return preds, regularize_loss(loss, weights, lambda)
```
注意,变量的路径可以是found here.
此外,我们想为所有具有变量的层(如 dnn
、conv2d
或 fully_connected
)添加正则化器支持,因此下周的 Tensorflow 构建应该会有一些东西像这样 dnn(.., regularize=tf.contrib.layers.l2_regularizer(lambda))
。发生这种情况时,我会更新此答案。