将 TensorFlow 1 代码移植到 TensorFlow 2（没有 sess.run 的模型学习过程）

Question

我有这段 tf1 代码，摘自 S. Nikolenko 的好书《深度学习》。

这是一个简单的线性回归，将k和b分别学习为2和1。

%tensorflow_version 1.x

import numpy as np,tensorflow as tf
import pandas as pd

n_samples, batch_size, num_steps = 1000, 100, 20000 #set learning constants
X_data = np.random.uniform(1, 10, (n_samples, 1)) #generate array x from 1 to 10 of shape (1000,1)
print(X_data.shape)
y_data = 2 * X_data + 1 + np.random.normal(0, 2, (n_samples, 1)) #generate right answer and add noise to it (to make it scatter)

X = tf.placeholder(tf.float32, shape=(batch_size, 1)) #defining placeholders to put into session.run
y = tf.placeholder(tf.float32, shape=(batch_size, 1))

with tf.variable_scope('linear-regression'):
  k = tf.Variable(tf.random_normal((1, 1)), name='slope') #defining 2 variables with shape (1,1)
  b = tf.Variable(tf.zeros((1,)), name='bias') # and (1,)
  print(k.shape,b.shape)

y_pred = tf.matmul(X, k) + b # all predicted y in batch, represents linear formula k*x + b
loss = tf.reduce_sum((y - y_pred) ** 2)  # mean square
optimizer = tf.train.GradientDescentOptimizer(0.0001).minimize(loss)
display_step = 100

with tf.Session() as sess:
  sess.run(tf.initialize_variables([k,b]))
  for i in range(num_steps):
    indices = np.random.choice(n_samples, batch_size) # taking random indices
    X_batch, y_batch = X_data[indices], y_data[indices] # taking x and y from generated examples
    _, loss_val, k_val, b_val = sess.run([optimizer, loss, k, b ],
      feed_dict = { X : X_batch, y : y_batch })
    if (i+1) % display_step == 0:
      print('Epoch %d: %.8f, k=%.4f, b=%.4f' %
        (i+1, loss_val, k_val, b_val))

我正在努力将其移植到 TensorFlow 2

很长一段时间我都想不通我应该用什么来代替 sess.run() 和 feed_dict，它们在幕后发挥着神奇的作用，官方文档详细介绍了编写模型class 等等，但我想尽可能保持平坦。

还建议使用 tf.GradientTape 计算导数，但我正在努力将其正确应用于我的示例

%tensorflow_version 2.x

import numpy as np,tensorflow as tf
import pandas as pd

n_samples, batch_size, num_steps = 1000, 100, 200
X_data = np.random.uniform(1, 10, (n_samples, 1))
y_data = 2 * X_data + 1 + np.random.normal(0, 2, (n_samples, 1))

X = tf.Variable(tf.zeros((batch_size, 1)), dtype=tf.float32, shape=(batch_size, 1))
y = tf.Variable(tf.zeros((batch_size, 1)), dtype=tf.float32, shape=(batch_size, 1))

k = tf.Variable(tf.random.normal((1, 1)), name='slope')
b = tf.Variable(tf.zeros((1,)), name='bias')

loss = lambda: tf.reduce_sum((y - (tf.matmul(X, k) + b)) ** 2)
optimizer = tf.keras.optimizers.SGD(0.01).minimize(loss, [k, b, X, y])
display_step = 100


for i in range(num_steps):
  indices = np.random.choice(n_samples, batch_size)
  X_batch, y_batch = X_data[indices], y_data[indices]

我需要 SGD 优化器最小化给定的损失函数并学习 k 和 b 值，从这一点我该如何实现？

Answer 1

在大量手册之后，我知道了如何做到这一点隐藏在 tf1 中 sess.run 的幕后，但没有优化器：

计算损失
计算关于训练变量的梯度
将函数相对于每个训练变量的增长速度调整为学习率
为 k 和 b

X_batch, y_batch = X_data[indices], y_data[indices]
X.assign(tf.convert_to_tensor(X_batch))
y.assign(tf.convert_to_tensor(y_batch))
with tf.GradientTape(persistent=True) as tape:
  loss_val = loss()

dy_dk = tape.gradient(loss_val, k)
dy_db = tape.gradient(loss_val, b)

k.assign_sub(dy_dk * learn_rate)
b.assign_sub(dy_db * learn_rate)
if (i+1) % display_step == 0:
  print('Epoch %d: %.8f, k=%.4f, b=%.4f' %
        (i+1, loss_val, k.numpy(), b.numpy()))

将 TensorFlow 1 代码移植到 TensorFlow 2（没有 sess.run 的模型学习过程）

port TensorFlow 1 code to TensorFlow 2 (model learning process without sess.run)

porting

machine-learning

linear-regression

tensorflow