class nbla::Adadelta
-
template<typename T>
class Adadelta : public nbla::Solver -
This is defined as
\[\begin{split} g_t \leftarrow \Delta w_t\\ v_t \leftarrow - \frac{RMS \left[ v_t \right]_{t-1}}{RMS \left[ g \right]_t}g_t\\ w_{t+1} \leftarrow w_t + \eta v_t \end{split}\]See also
See the paper linked below for more details. Matthew D. Zeiler ADADELTA: An Adaptive Learning Rate Method https://arxiv.org/abs/1212.5701
- Param lr:
\(\eta\) Learning rate.
- Param decay:
\(\gamma\) Decay rate.
- Param eps:
\(\epsilon\) Tiny factor for avoiding 0-division.
Public Functions
-
inline virtual float learning_rate()
Set learning rate.