Julia 的 Flux 中的 Keras W_constraint 和 W_regularizer 类似物

Question

我正在尝试解析 Keras json 文件以在 Julia 中创建一个 Flux 模型（Keras v1.1.0 和 Flux v0.10.4 ).

Dense层配置示例：

{
    "class_name": "Dense", 
    "config": {
        "W_constraint": null, 
        "b_constraint": null, 
        "name": "dense_1", 
        "output_dim": 512, 
        "activity_regularizer": null, 
        "trainable": true, 
        "init": "glorot_normal", 
        "bias": true, 
        "input_dtype": "float32", 
        "input_dim": 4096, 
        "b_regularizer": null, 
        "W_regularizer": {
            "l2": 0.0010000000474974513, 
            "name": "WeightRegularizer", 
            "l1": 0.0
        }, 
        "activation": "relu", 
        "batch_input_shape": [null, 4096]
    }
}

所以，在Flux中如何定义input/output维度、激活函数和参数初始化，我就很清楚了。但是 W_constraint 和 W_regularizer 呢？我在Flux的Dense层中没有发现任何类似的东西。它存在吗？我应该自己实施吗？ Dense 层的那些参数是否重要，或者在创建 Flux 模型时可以轻松跳过而不会严重改变性能？

Answer 1

regularization values are norms that are summed for all parameters of the network and added to the loss function; you have to do that "manually", but it's quite easy and described in the docs。

Keras 中的参数约束 appearently implemented by using projection methods, which are part of the optimizer. That is less trivial to implement, I suggest reading a bit about proximal gradient methods. You probably have to implement your own optimization type doing that in Flux (ideally, wrapping a one of the existing ones). Maybe ProximalOperators.jl 可以完成一些繁重的工作。另一方面，据我所知，具有参数约束的模型不太常见，您可能会暂时不使用它们。

Julia 的 Flux 中的 Keras W_constraint 和 W_regularizer 类似物

Keras W_constraint and W_regularizer analogues in Julia's Flux

julia

keras

flux.jl