为什么我不能在千层面回归模型的最后一层使用 dropout?
Why can't I use dropout in the last layer of a lasagne regression model?
玩具回归示例。使用 dropout=0.0
这很好用并且成本降低了。使用 dropout=0.5
我得到错误:
ValueError: Got num_leading_axes=1 for a 1-dimensional input, leaving no trailing axes for the dot product.
有什么想法吗?
import theano
import theano.tensor as T
import lasagne
import numpy as np
num_features=10
N=1000
# Set up the network
x=T.fmatrix('x')
y=T.fvector('y')
dropout=0.5
network = lasagne.layers.InputLayer(shape=(None, num_features), input_var=x)
if dropout > 0.0:
network = lasagne.layers.dropout(network, p=dropout),
network = lasagne.layers.DenseLayer( network, num_units=1, nonlinearity=None )
pred = lasagne.layers.get_output(network)
cost = lasagne.objectives.squared_error(pred, y).mean()
# Compile training function
params = lasagne.layers.get_all_params(network, trainable=True)
train_fn = theano.function([x, y], cost, updates=lasagne.updates.adamax(cost, params) )
# Generate some synthetic data
X=np.random.randn( N,num_features ).astype( theano.config.floatX )
b=np.random.randn( num_features ).astype( theano.config.floatX )
Y=np.dot( X, b ) + np.random.randn( N ).astype(theano.config.floatX ) * 0.1
# Train for 100 iterations
for i in range(100):
print train_fn(X,Y)
删除掉落层后的逗号。
该代码将在 InputLayer 或 DenseLayer 之后立即处理 dropout。逗号使用导致错误的网络变量 (network, ) 创建一个元组。
network = lasagne.layers.InputLayer(shape=(None, num_features), input_var=x)
if dropout > 0.0:
network = lasagne.layers.dropout(network, p=dropout),
network = lasagne.layers.DenseLayer( network, num_units=1, nonlinearity=None )
玩具回归示例。使用 dropout=0.0
这很好用并且成本降低了。使用 dropout=0.5
我得到错误:
ValueError: Got num_leading_axes=1 for a 1-dimensional input, leaving no trailing axes for the dot product.
有什么想法吗?
import theano
import theano.tensor as T
import lasagne
import numpy as np
num_features=10
N=1000
# Set up the network
x=T.fmatrix('x')
y=T.fvector('y')
dropout=0.5
network = lasagne.layers.InputLayer(shape=(None, num_features), input_var=x)
if dropout > 0.0:
network = lasagne.layers.dropout(network, p=dropout),
network = lasagne.layers.DenseLayer( network, num_units=1, nonlinearity=None )
pred = lasagne.layers.get_output(network)
cost = lasagne.objectives.squared_error(pred, y).mean()
# Compile training function
params = lasagne.layers.get_all_params(network, trainable=True)
train_fn = theano.function([x, y], cost, updates=lasagne.updates.adamax(cost, params) )
# Generate some synthetic data
X=np.random.randn( N,num_features ).astype( theano.config.floatX )
b=np.random.randn( num_features ).astype( theano.config.floatX )
Y=np.dot( X, b ) + np.random.randn( N ).astype(theano.config.floatX ) * 0.1
# Train for 100 iterations
for i in range(100):
print train_fn(X,Y)
删除掉落层后的逗号。 该代码将在 InputLayer 或 DenseLayer 之后立即处理 dropout。逗号使用导致错误的网络变量 (network, ) 创建一个元组。
network = lasagne.layers.InputLayer(shape=(None, num_features), input_var=x)
if dropout > 0.0:
network = lasagne.layers.dropout(network, p=dropout),
network = lasagne.layers.DenseLayer( network, num_units=1, nonlinearity=None )