Type Members

1.  trait ConvergenceCheck[T] extends AnyRef
2.  trait ConvergenceReason extends AnyRef
3.  case class FunctionValuesConverged[T](tolerance: Double, relative: Boolean, historyLength: Int) extends ConvergenceCheck[T] with Product with Serializable
4.  case class MonitorFunctionValuesCheck[T](f: (T) => Double, numFailures: Int, improvementRequirement: Double, evalFrequency: Int) extends ConvergenceCheck[T] with SerializableLogging with Product with Serializable
5.  case class OptParams(batchSize: Int = 512, regularization: Double = 0.0, alpha: Double = 0.5, maxIterations: Int = 1000, useL1: Boolean = false, tolerance: Double = 1E-5, useStochastic: Boolean = false, randomSeed: Int = 0) extends Product with Serializable

   OptParams is a Configuration-compatible case class that can be used to select optimization routines at runtime.

   OptParams is a Configuration-compatible case class that can be used to select optimization routines at runtime.

   Configurations: 1) useStochastic=false,useL1=false: LBFGS with L2 regularization 2) useStochastic=false,useL1=true: OWLQN with L1 regularization 3) useStochastic=true,useL1=false: AdaptiveGradientDescent with L2 regularization 3) useStochastic=true,useL1=true: AdaptiveGradientDescent with L1 regularization

   batchSize

   size of batches to use if useStochastic and you give a BatchDiffFunction

   regularization

   regularization constant to use.

   alpha

   rate of change to use, only applies to SGD.

   useL1

   if true, use L1 regularization. Otherwise, use L2.

   tolerance

   convergence tolerance, looking at both average improvement and the norm of the gradient.

   useStochastic

   if false, use LBFGS or OWLQN. If true, use some variant of Stochastic Gradient Descent.

6.  case class SequenceConvergenceCheck[T](checks: IndexedSeq[ConvergenceCheck[T]]) extends ConvergenceCheck[T] with Product with Serializable
7.  case class State[+T, +ConvergenceInfo, +History](x: T, value: Double, grad: T, adjustedValue: Double, adjustedGradient: T, iter: Int, initialAdjVal: Double, history: History, convergenceInfo: ConvergenceInfo, searchFailed: Boolean = false, convergenceReason: Option[ConvergenceReason] = None) extends Product with Serializable

   Tracks the information about the optimizer, including the current point, its value, gradient, and then any history.

   Tracks the information about the optimizer, including the current point, its value, gradient, and then any history. Also includes information for checking convergence.

   x

   the current point being considered

   value

   f(x)

   grad

   f.gradientAt(x)

   adjustedValue

   f(x) + r(x), where r is any regularization added to the objective. For LBFGS, this is f(x).

   adjustedGradient

   f'(x) + r'(x), where r is any regularization added to the objective. For LBFGS, this is f'(x).

   iter

   what iteration number we are on.

   initialAdjVal

   f(x_0) + r(x_0), used for checking convergence

   history

   any information needed by the optimizer to do updates.

   searchFailed

   did the line search fail?

   convergenceReason

   the convergence reason