如何手动初始化权重值？

Question

我想尝试一下 Karpathy 在他的讲义中推荐的权重初始化，

the recommended heuristic is to initialize each neuron's weight vector as: w = np.random.randn(n) / sqrt(n), where n is the number of its inputs

来源：http://cs231n.github.io/neural-networks-2/#init

我是 python 的初学者，我不知道如何实现：/

weights = tf.Variable(??)

请帮忙？ ...

Answer 1

对于单个值，使用：

weights = tf.Variable(10)

对于具有随机值的向量：

shape = [784, 625]
weights = tf.Variable(tf.random_normal(shape, stddev=0.01)/tf.sqrt(n))

请注意，您需要 sess.run 来评估变量。

此外，请查看其他随机张量：https://www.tensorflow.org/versions/r0.8/api_docs/python/constant_op.html#random-tensors

Answer 2

n = 10
init_x = np.random.randn(n)
x = tf.Variable(init_x)
sess = tf.InteractiveSession()
sess.run(tf.initialize_all_variables())
print(sess.run(x))

Answer 3

我的做法是：

    self.w_full, self.b_full = [], []

    n_fc_layers = len(structure)
    structure.insert(0, self.n_inputs)

    with vs.variable_scope(self.scope):
        for lr_idx in range(n_fc_layers):
            n_in, n_out = structure[lr_idx], structure[lr_idx+1]
            self.w_full.append(
                vs.get_variable(
                   "FullWeights{}".format(lr_idx),
                    [n_in, n_out],
                    dtype=tf.float32,
                    initializer=tf.random_uniform_initializer(
                        minval=-tf.sqrt(tf.constant(6.0)/(n_in + n_out)),
                        maxval=tf.sqrt(tf.constant(6.0)/(n_in + n_out))
                    )
                )
            )

            self.b_full.append(
                vs.get_variable(
                    "FullBiases{}".format(lr_idx),
                    [n_out],
                    dtype=tf.float32,
                    initializer=tf.constant_initializer(0.0)
                )
            )

之后

structure.insert(0, self.n_inputs)

你将有 [n_inputs，第一个 FC 层大小，第二个 FC 层大小...输出层大小]

如何手动初始化权重值？

How to manually initialize the values for the weights?

neural-network

python-2.7

tensorflow