proximal

Type Members

1. class NonlinearMinimizer extends SerializableLogging

   

   NonlinearMinimizer solves the problem that has the following structure minimize f(x) + g(x)

   NonlinearMinimizer solves the problem that has the following structure minimize f(x) + g(x)

   g(x) represents the following constraints

   1. x >= 0 2. lb <= x <= ub 3. L1(x) 4. Aeq*x = beq 5. aeq'x = beq 6. 1'x = s, x >= 0 ProbabilitySimplex from the reference Proximal Algorithms by Boyd et al, Duchi et al

   f(x) can be a smooth convex function defined by DiffFunction or a proximal operator. For now the exposed API takes DiffFunction

   g(x) is defined by a proximal operator

   For proximal algorithms like L1 through soft-thresholding and Huber Loss (look into library of proximal algorithms for further details) we provide ADMM based Proximal algorithm based on the following reference: https://web.stanford.edu/~boyd/papers/admm/logreg-l1/distr_l1_logreg.html

   A subset of proximal operators are projection operators. For projection, NonlinearMinimizer companion object provides project API which generates a Spectral Projected Gradient (SPG) or Projected Quasi Newton (PQN) solver. For projection operators like positivity, bounds, probability simplex etc, these algorithms converges faster as compared to ADMM based proximal algorithm.

   TO DO

   1. Implement FISTA / Nesterov's accelerated method and compare with ADMM 2. For non-convex function experiment with TRON-like Primal solver

2. case class ProjectBox(l: DenseVector[Double], u: DenseVector[Double]) extends Proximal with Product with Serializable

   

3. case class ProjectEquality(Aeq: DenseMatrix[Double], beq: DenseVector[Double]) extends Proximal with Product with Serializable

   

4. case class ProjectHyperPlane(a: DenseVector[Double], b: Double) extends Proximal with Product with Serializable

   

5. case class ProjectIdentity() extends Proximal with Product with Serializable

   

6. case class ProjectL1(s: Double) extends Proximal with Product with Serializable

   

   Projection formula from Duchi et al's paper Efficient Projections onto the l1-Ball for Learning in High Dimensions

7. case class ProjectPos() extends Proximal with Product with Serializable

   

8. case class ProjectProbabilitySimplex(s: Double) extends Proximal with Product with Serializable

   

9. case class ProjectSoc() extends Proximal with Product with Serializable

   

10. trait Proximal extends AnyRef

    

11. case class ProximalHuber() extends Proximal with Product with Serializable

    

12. case class ProximalL1(lambda: Double = 1.0) extends Proximal with Product with Serializable

    

13. case class ProximalL2() extends Proximal with Product with Serializable

    

14. case class ProximalLinear(c: DenseVector[Double]) extends Proximal with Product with Serializable

    

15. case class ProximalLogBarrier() extends Proximal with Product with Serializable

    

16. case class ProximalLp(c: DenseVector[Double]) extends Proximal with Product with Serializable

    

17. case class ProximalSumSquare() extends Proximal with Product with Serializable

    

18. class QuadraticMinimizer extends SerializableLogging

    

    Proximal operators and ADMM based Primal-Dual QP Solver

    Proximal operators and ADMM based Primal-Dual QP Solver

    Reference: http://www.stanford.edu/~boyd/papers/admm/quadprog/quadprog.html

    It solves problem that has the following structure

    1/2 x'Hx + f'x + g(x) s.t Aeqx = b

    g(x) represents the following constraints which covers ALS based matrix factorization use-cases

    1. x >= 0 2. lb <= x <= ub 3. L1(x) 4. L2(x) 5. Generic regularization on x