class nbla::Momentum
-
template<typename T>
class Momentum : public nbla::Solver -
This is defined as
\[\begin{split} v_t \leftarrow \gamma v_{t-1} + \eta \Delta w_t\\ w_{t+1} \leftarrow w_t - v_t \end{split}\]See also
See the paper linked below for more details. Ning Qian : On the Momentum Term in Gradient Descent Learning Algorithms http://www.columbia.edu/~nq6/publications/momentum.pdf
- Param lr:
\(\eta\) Learning rate.
- Param momentum:
\(\gamma\) Momentum
Public Functions
-
inline virtual float learning_rate()
Set learning rate.