我的 TensorFlow 梯度下降发散

Question

import tensorflow as tf
import pandas as pd
import numpy as np

def normalize(data):
    return data - np.min(data) / np.max(data) - np.min(data)

df = pd.read_csv('sat.csv', skipinitialspace=True)
x_reading = df['reading_score']
x_math = df['math_score']
x_reading, x_math = np.array(x_reading[df.reading_score != 's']), np.array(x_math[df.math_score != 's'])

x_data = normalize(np.float32(np.array([x_reading, x_math])))

y_writing = df[['writing_score']]
y_data = normalize(np.float32(np.array(y_writing[df.writing_score != 's'])))

W = tf.Variable(tf.random_uniform([1, 2], -.5, .5)) #float32
b = tf.Variable(tf.ones([1]))
y = tf.matmul(W, x_data) + b

loss = tf.reduce_mean(tf.square(y - y_data.T))
optimizer = tf.train.GradientDescentOptimizer(0.005)
train = optimizer.minimize(loss)

init = tf.initialize_all_variables()

with tf.Session() as sess:
    sess.run(init)

    for step in range(1000):
        sess.run(train)
        print step, sess.run(W), sess.run(b), sess.run(loss)

这是我的代码。我的 sat.csv 包含 SAT 阅读、写作和数学成绩的数据。您可以猜到，这些功能之间的差异并不大。

这是 sat.csv 的一部分。

DBN,SCHOOL NAME,Num of Test Takers,reading_score,math_score,writing_score
01M292,HENRY STREET SCHOOL FOR INTERNATIONAL STUDIES,29,355,404,363
01M448,UNIVERSITY NEIGHBORHOOD HIGH SCHOOL,91,383,423,366
01M450,EAST SIDE COMMUNITY SCHOOL,70,377,402,370
01M458,FORSYTH SATELLITE ACADEMY,7,414,401,359
01M509,MARTA VALLE HIGH SCHOOL,44,390,433,384
01M515,LOWER EAST SIDE PREPARATORY HIGH SCHOOL,112,332,557,316
01M539,"NEW EXPLORATIONS INTO SCIENCE, TECHNOLOGY AND MATH HIGH SCHOOL",159,522,574,525
01M650,CASCADES HIGH SCHOOL,18,417,418,411
01M696,BARD HIGH SCHOOL EARLY COLLEGE,130,624,604,628
02M047,47 THE AMERICAN SIGN LANGUAGE AND ENGLISH SECONDARY SCHOOL,16,395,400,387

我只使用了数学、写作和阅读分数。如果我给出数学和阅读分数，我对上面代码的目标是预测写作分数。

我从未见过 Tensorflow 的梯度下降模型与如此简单的数据发生分歧。有什么问题吗？

Answer 1

以下是您可以尝试的几个选项：

标准化您的输入和输出数据
为您的权重设置较小的初始值
使用较低的学习率
将您的损失除以您拥有的样本数量（不将数据放在占位符中已经不常见）。

让我知道这些选项有什么帮助（如果有的话），祝你好运！

我的 TensorFlow 梯度下降发散

My TensorFlow Gradient Descent diverges

tensorflow

linear-regression

gradient-descent