class nbla::RMSpropGraves

template<typename T>
class RMSpropGraves : public nbla::Solver

RMSpropGraves.

This is defined as

\[\begin{split} n_t \leftarrow \rho n_{t-1} + \left(1 - \rho \right) e^2\\ g_t \leftarrow \rho g_{t-1} + \left(1 - \rho \right) e\\ d_t \leftarrow \beta d_{t-1} - \eta \frac{e}{\sqrt{n_t - {g_t}^2 + \epsilon}}\\ w_{t+1} \leftarrow w_t + d_t \end{split}\]

See also

See the paper linked below for more details. A. Graves Generating Sequences With Recurrent Neural Networks http://arxiv.org/pdf/1308.0850v5.pdf

Param lr:

\(\eta\) Learning rate.

Param decay:

\(\rho\) Decay rate.

Param momentum:

\(\beta\) Momentum.

Param eps:

\(\epsilon\) Tiny factor for avoiding 0-division.

Public Functions

inline virtual float learning_rate()

Set learning rate.