如何在 Python 中用梯度下降法找到 2 个参数？

Question

我有几行代码没有收敛。如果有人知道为什么，我将不胜感激。原始方程写在 def f(x,y,b,m) 中，我需要找到参数 b,m。

  np.random.seed(42)
  x = np.random.normal(0, 5, 100)
  y = 50 + 2 * x + np.random.normal(0, 2, len(x))

  def f(x, y, b, m):
      return (1/len(x))*np.sum((y - (b + m*x))**2) # it is supposed to be a sum operator

  def dfb(x, y, b, m): # partial derivative with respect to b
      return b - m*np.mean(x)+np.mean(y)

  def dfm(x, y, b, m): # partial derivative with respect to m
      return np.sum(x*y - b*x - m*x**2)

  b0 = np.mean(y)
  m0 = 0
  alpha = 0.0001
  beta = 0.0001
  epsilon = 0.01

  while True:

      b = b0 - alpha * dfb(x, y, b0, m0)
      m = m0 - alpha * dfm(x, y, b0, m0)

      if np.sum(np.abs(m-m0)) <= epsilon and np.sum(np.abs(b-b0)) <= epsilon:
          break
      else:
          m0 = m
          b0 = b
      print(m, f(x, y, b, m))

Answer 1

两种导数都有一些符号混淆：

def dfb(x, y, b, m): # partial derivative with respect to b
  # return b - m*np.mean(x)+np.mean(y)
  #          ^-------------^------ these are incorrect
  return b + m*np.mean(x) - np.mean(y)

def dfm(x, y, b, m): # partial derivative with respect to m
  #      v------ this should be negative
  return -np.sum(x*y - b*x - m*x**2)

其实这些导数还缺少一些常量：

dfb 应乘以 2
dfm 应乘以 2/len(x)

我想这还不算太糟糕，因为无论如何梯度都会按 alpha 缩放，但它可能会使收敛速度变差。

如果您使用正确的导数，您的代码将在一次次迭代后收敛：

def dfb(x, y, b, m): # partial derivative with respect to b
  return 2 * (b + m * np.mean(x) - np.mean(y))

def dfm(x, y, b, m): # partial derivative with respect to m
  # Used `mean` here since (2/len(x)) * np.sum(...)
  # is the same as 2 * np.mean(...)
  return -2 * np.mean(x * y - b * x - m * x**2)

如何在 Python 中用梯度下降法找到 2 个参数？

How to find 2 parameters with gradient descent method in Python?

python

numpy

gradient-descent