取 TensorFlow 数值积分结果的梯度时避免矩阵求逆错误
avoiding matrix inversion error when taking gradient of TensorFlow numerical integration results
我正在使用 tfp.math.ode.BDF
to numerically integrate a system of ordinary differential equations (ODEs). See my Colaboratory notebook here。
与 API 文档中的示例代码一样,函数 ode_fn(t, y, theta)
定义了要求解的 ODE 系统。我能够采用 ode_fn
wrt theta
的梯度并将 ODE 与 tfp.math.ode.BDF
.
集成
但是,当我尝试对 theta
求 ODE 解结果的梯度时,出现以下错误。当我用一组更简单的 ODE 替换 ode_fn
时,代码运行没有任何问题。是否应调整求解器设置以避免此错误?
InvalidArgumentError Traceback (most recent call last)
<ipython-input-9-77ebcb7dd888> in <module>()
----> 1 print(g.gradient(foo, theta0))
5 frames
/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/backprop.py in gradient(self, target, sources, output_gradients, unconnected_gradients)
1088 output_gradients=output_gradients,
1089 sources_raw=flat_sources_raw,
-> 1090 unconnected_gradients=unconnected_gradients)
1091
1092 if not self._persistent:
/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/imperative_grad.py in imperative_grad(tape, target, sources, output_gradients, sources_raw, unconnected_gradients)
75 output_gradients,
76 sources_raw,
---> 77 compat.as_str(unconnected_gradients.value))
/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/function.py in _backward_function_wrapper(*args)
1301 break
1302 return backward._call_flat( # pylint: disable=protected-access
-> 1303 processed_args, remapped_captures)
1304
1305 return _backward_function_wrapper, recorded_outputs
/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/function.py in _call_flat(self, args, captured_inputs, cancellation_manager)
1962 # No tape is watching; skip to running the function.
1963 return self._build_call_outputs(self._inference_function.call(
-> 1964 ctx, args, cancellation_manager=cancellation_manager))
1965 forward_backward = self._select_forward_and_backward_functions(
1966 args,
/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/function.py in call(self, ctx, args, cancellation_manager)
594 inputs=args,
595 attrs=attrs,
--> 596 ctx=ctx)
597 else:
598 outputs = execute.execute_with_cancellation(
/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/execute.py in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
58 ctx.ensure_initialized()
59 tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
---> 60 inputs, attrs, num_outputs)
61 except core._NotOkStatusException as e:
62 if name is not None:
InvalidArgumentError: Input matrix is not invertible.
[[{{node gradients/IdentityN_grad/bdfGradients/while/body/_718/gradients/IdentityN_grad/bdfGradients/while/bdf/while/body/_2247/gradients/IdentityN_grad/bdfGradients/while/bdf/while/while/body/_3245/gradients/IdentityN_grad/bdfGradients/while/bdf/while/while/while/body/_5670/gradients/IdentityN_grad/bdfGradients/while/bdf/while/while/while/while/body/_7588/gradients/IdentityN_grad/bdfGradients/while/bdf/while/while/while/while/triangular_solve/MatrixTriangularSolve}}]] [Op:__inference___backward_debug_ode_solver_9192_32890]
Function call stack:
__backward_debug_ode_solver_9192
我通过执行以下操作设法避免了这些数字错误:
- 重新缩放我的因变量和参数,使所有内容都在彼此的几个数量级内。
- 增加 ODE 求解器 return 的因变量值所在的自变量值的数量。我只需要 20 个均匀间隔的时间点来计算我的损失函数,但是如果我想根据我的参数计算该损失函数的梯度,ODE 求解器必须 return 至少 60 个均匀间隔的时间点。我只是把加分扔掉了。
我正在使用 tfp.math.ode.BDF
to numerically integrate a system of ordinary differential equations (ODEs). See my Colaboratory notebook here。
与 API 文档中的示例代码一样,函数 ode_fn(t, y, theta)
定义了要求解的 ODE 系统。我能够采用 ode_fn
wrt theta
的梯度并将 ODE 与 tfp.math.ode.BDF
.
但是,当我尝试对 theta
求 ODE 解结果的梯度时,出现以下错误。当我用一组更简单的 ODE 替换 ode_fn
时,代码运行没有任何问题。是否应调整求解器设置以避免此错误?
InvalidArgumentError Traceback (most recent call last)
<ipython-input-9-77ebcb7dd888> in <module>()
----> 1 print(g.gradient(foo, theta0))
5 frames
/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/backprop.py in gradient(self, target, sources, output_gradients, unconnected_gradients)
1088 output_gradients=output_gradients,
1089 sources_raw=flat_sources_raw,
-> 1090 unconnected_gradients=unconnected_gradients)
1091
1092 if not self._persistent:
/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/imperative_grad.py in imperative_grad(tape, target, sources, output_gradients, sources_raw, unconnected_gradients)
75 output_gradients,
76 sources_raw,
---> 77 compat.as_str(unconnected_gradients.value))
/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/function.py in _backward_function_wrapper(*args)
1301 break
1302 return backward._call_flat( # pylint: disable=protected-access
-> 1303 processed_args, remapped_captures)
1304
1305 return _backward_function_wrapper, recorded_outputs
/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/function.py in _call_flat(self, args, captured_inputs, cancellation_manager)
1962 # No tape is watching; skip to running the function.
1963 return self._build_call_outputs(self._inference_function.call(
-> 1964 ctx, args, cancellation_manager=cancellation_manager))
1965 forward_backward = self._select_forward_and_backward_functions(
1966 args,
/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/function.py in call(self, ctx, args, cancellation_manager)
594 inputs=args,
595 attrs=attrs,
--> 596 ctx=ctx)
597 else:
598 outputs = execute.execute_with_cancellation(
/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/execute.py in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
58 ctx.ensure_initialized()
59 tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
---> 60 inputs, attrs, num_outputs)
61 except core._NotOkStatusException as e:
62 if name is not None:
InvalidArgumentError: Input matrix is not invertible.
[[{{node gradients/IdentityN_grad/bdfGradients/while/body/_718/gradients/IdentityN_grad/bdfGradients/while/bdf/while/body/_2247/gradients/IdentityN_grad/bdfGradients/while/bdf/while/while/body/_3245/gradients/IdentityN_grad/bdfGradients/while/bdf/while/while/while/body/_5670/gradients/IdentityN_grad/bdfGradients/while/bdf/while/while/while/while/body/_7588/gradients/IdentityN_grad/bdfGradients/while/bdf/while/while/while/while/triangular_solve/MatrixTriangularSolve}}]] [Op:__inference___backward_debug_ode_solver_9192_32890]
Function call stack:
__backward_debug_ode_solver_9192
我通过执行以下操作设法避免了这些数字错误:
- 重新缩放我的因变量和参数,使所有内容都在彼此的几个数量级内。
- 增加 ODE 求解器 return 的因变量值所在的自变量值的数量。我只需要 20 个均匀间隔的时间点来计算我的损失函数,但是如果我想根据我的参数计算该损失函数的梯度,ODE 求解器必须 return 至少 60 个均匀间隔的时间点。我只是把加分扔掉了。