class nbla::Nesterov

template<typename T>
class Nesterov : public nbla::Solver

Nesterov accelerated gradient.

This is defined as

\[\begin{split} v_t \leftarrow \gamma v_{t-1} - \eta \Delta w_t \\ w_{t+1} \leftarrow w_t - \gamma v_{t-1} + \left(1 + \gamma \right) v_t \end{split}\]

See also

See the paper linked below for more details. Yurii Nesterov A method for unconstrained convex minimization problem with the rate of convergence o(1/k2)

Param lr:

\(\eta\) Learning rate.

Param momentum:

\(\gamma\) Momentum

Public Functions

inline virtual float learning_rate()

Set learning rate.