class nbla::Nesterov
-
template<typename T>
class Nesterov : public nbla::Solver Nesterov accelerated gradient.
This is defined as
\[\begin{split} v_t \leftarrow \gamma v_{t-1} - \eta \Delta w_t \\ w_{t+1} \leftarrow w_t - \gamma v_{t-1} + \left(1 + \gamma \right) v_t \end{split}\]See also
See the paper linked below for more details. Yurii Nesterov A method for unconstrained convex minimization problem with the rate of convergence o(1/k2)
- Param lr:
\(\eta\) Learning rate.
- Param momentum:
\(\gamma\) Momentum
Public Functions
-
inline virtual float learning_rate()
Set learning rate.