Packages

•  package root

  Definition Classes

  root

•  package breeze

  Definition Classes

  root

•  package optimize
  Definition Classes

  breeze

•  object AdaptiveGradientDescent

  Implements the L2^2 and L1 updates from Duchi et al 2010 Adaptive Subgradient Methods for Online Learning and Stochastic Optimization.

  Implements the L2^2 and L1 updates from Duchi et al 2010 Adaptive Subgradient Methods for Online Learning and Stochastic Optimization.

  Basically, we use "forward regularization" and an adaptive step size based on the previous gradients.

  Definition Classes

  optimize

•  class L2Regularization[T] extends StochasticGradientDescent[T]

  Implements the L2 regularization update.

  Implements the L2 regularization update.

  Each step is:

  x_{t+1}i = (s_{ti} * x_{ti} - \eta * g_ti) / (eta * regularization + delta + s_ti)

  where g_ti is the gradient and s_ti = \sqrt(\sum_t'^{{t} g_ti}2)

  Definition Classes

  AdaptiveGradientDescent

• History

breeze.optimize.AdaptiveGradientDescent.L2Regularization

History