神经网络使用交叉熵，不使用其他损失函数

Question

我正在使用 tensorflow 和 julia 创建一个神经网络。

我可以用 cross_entropy 损失函数创建一个网络并且它有效：

ENV["CUDA_VISIBLE_DEVICES"] = "0" # It is to use the gpu
using TensorFlow
using Distributions


function weight_variable(shape)
   initial = map(Float32, rand(Normal(0, .001), shape...))
   return Variable(initial)
end

function bias_variable(shape)
   initial = fill(Float32(.1), shape...)
   return Variable(initial)
end


sess = Session(Graph())

num_pixels = 12

num_classes = 10

x = placeholder((Float32), shape=[nothing, num_pixels])
y = placeholder(Float32, shape=[nothing, num_classes])

poids = weight_variable([num_pixels,num_classes]) # Weight

biases = bias_variable([num_classes])


cross_entropy = reduce_mean(-reduce_sum(y.*log(nn.softmax(x*poids + biases)))) # Cross entropy Loss function

optimizer = train.AdamOptimizer(0.0001)


train_op = train.minimize(optimizer,cross_entropy)

correct_prediction = equal(indmax(nn.softmax(x*poids + biases), 2), indmax(y,2))

accuracy = reduce_mean(cast(correct_prediction, Float32))


y1 = [0 0 1 0 0 0 0 0 0 0] # correct label

x1 = [0 0 0 5 6 3 2 0 0 0 0 0] # Input 

run(sess, global_variables_initializer())

for i in 1:10

x_ = run(sess,train_op,Dict(x => x1, y => y1))

acc = run(sess,accuracy,Dict(x => x1, y => y1))

info("train $i , accuracy = $acc")

end

close(sess)

现在，如果我只是用指数成本改变我的损失函数，就像这里：

ENV["CUDA_VISIBLE_DEVICES"] = "0" # It is to use the gpu
using TensorFlow
using Distributions


function weight_variable(shape)
   initial = map(Float32, rand(Normal(0, .001), shape...))
   return Variable(initial)
end

function bias_variable(shape)
   initial = fill(Float32(.1), shape...)
   return Variable(initial)
end


sess = Session(Graph())

num_pixels = 12

num_classes = 10

x = placeholder((Float32), shape=[nothing, num_pixels])
y = placeholder(Float32, shape=[nothing, num_classes])

poids = weight_variable([num_pixels,num_classes]) # Weight

biases = bias_variable([num_classes])

expo = reduce_mean((0.5*exp((1/0.5).*reduce_sum((nn.softmax(x*poids + biases)- y)^2)))) # Exponential loss function

optimizer = train.AdamOptimizer(0.0001)


train_op = train.minimize(optimizer,expo)

correct_prediction = equal(indmax(nn.softmax(x*poids + biases), 2), indmax(y,2))

accuracy = reduce_mean(cast(correct_prediction, Float32))


y1 = [0 0 1 0 0 0 0 0 0 0] # correct label

x1 = [0 0 0 5 6 3 2 0 0 0 0 0] # Input 

run(sess, global_variables_initializer())

for i in 1:10

x_ = run(sess,train_op,Dict(x => x1, y => y1))

acc = run(sess,accuracy,Dict(x => x1, y => y1))

info("train $i , accuracy = $acc")

end

close(sess)

它不起作用，我有以下错误：

ERROR: LoadError: Tensorflow error: Status: Node name 'gradients/Softmax_grad/Sum' already exists in the Graph

Stacktrace:
 [1] (::Atom.##110#114{String,String})() at /home/jabou/.julia/v0.6/Atom/src/eval.jl:104
 [2] withpath(::Atom.##110#114{String,String}, ::String) at /home/jabou/.julia/v0.6/CodeTools/src/utils.jl:30
 [3] withpath(::Function, ::String) at /home/jabou/.julia/v0.6/Atom/src/eval.jl:38
 [4] hideprompt(::Atom.##109#113{String,String}) at /home/jabou/.julia/v0.6/Atom/src/repl.jl:66
 [5] macro expansion at /home/jabou/.julia/v0.6/Atom/src/eval.jl:99 [inlined]
 [6] (::Atom.##108#112{Dict{String,Any}})() at ./task.jl:80
while loading /home/jabou/Bureau/Minimum nouveau.jl, in expression starting on line 37

我不明白为什么...你能帮我吗？

谢谢

Answer 1

在 Tensorflow.jl 中，我通常使用具有随机名称的变量作用域 ("mymodel" * randstring())，这样当您运行代码两次时（例如在交互式会话中），没有命名冲突。

variable_scope("mymodel" * randstring(), initializer=Normal(0, .1)) do
    global w1 = get_variable("weights1", [num_input, hidden_units1], Float32)
    global b1 = get_variable("b1",[hidden_units1],Float32)
    # other parameters ....
end

这有帮助吗？

Answer 2

问题的解决方案在这里：

必须使用1.4.0版本的TensorFlow。

神经网络使用交叉熵，不使用其他损失函数

Neural network works with the cross entropy and does not with an other loss function

neural-network

julia

tensorflow

loss-function