class nbla::Lars

template<typename T>
class Lars : public nbla::Solver

LARS solver defined as.

\[\begin{split} \lambda \leftarrow \eta \frac{\| w_t \|}{\| \Delta w_t + \beta w_t \|} \\ v_{t+1} \leftarrow m v_t + \gamma \lambda (\Delta w_t + \beta w_t) \\ w_{t+1} \leftarrow w_t - v_{t+1} \end{split}\]

See also

See the paper linked below for more details. Yang Youity, Igor Gitmann, and Boris Ginsburg: Large Batch Training of Convolutional Networks https://arxiv.org/pdf/1708.03888

Param lr:

\(\gamma\) Learning rate.

Param momentum:

\(m\) Momentum.

Param coefficient:

\(\eta\) Coefficient of the local learning rate.

Param eps:

\(\epsilon\) Tiny factor for avoiding 0-division.

Public Functions

inline virtual float learning_rate()

Set learning rate.