org.deeplearning4j.optimize.solvers

Class BaseOptimizer

java.lang.Object

org.deeplearning4j.optimize.solvers.BaseOptimizer

All Implemented Interfaces:

Serializable, ConvexOptimizer

Direct Known Subclasses:

ConjugateGradient, LBFGS, LineGradientDescent, StochasticGradientDescent

public abstract class BaseOptimizer
extends Object
implements ConvexOptimizer

Base optimizer

Author:

Adam Gibson

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
protected ComputationGraphUpdater	computationGraphUpdater
protected NeuralNetConfiguration	conf
static String	GRADIENT_KEY
protected int	iteration
protected Collection<IterationListener>	iterationListeners
protected BackTrackLineSearch	lineMaximizer
protected static org.slf4j.Logger	log
protected Model	model
protected double	oldScore
static String	PARAMS_KEY
protected double	score
static String	SCORE_KEY
static String	SEARCH_DIR
protected Map<String,Object>	searchState
protected double	step
protected StepFunction	stepFunction
protected double	stepMax
protected Collection<TerminationCondition>	terminationConditions
protected Updater	updater

Constructor Summary

Constructors

Constructor and Description
BaseOptimizer(NeuralNetConfiguration conf, StepFunction stepFunction, Collection<IterationListener> iterationListeners, Collection<TerminationCondition> terminationConditions, Model model)
BaseOptimizer(NeuralNetConfiguration conf, StepFunction stepFunction, Collection<IterationListener> iterationListeners, Model model)

Method Summary

Modifier and Type	Method and Description
int	batchSize() The batch size for the optimizer
boolean	checkTerminalConditions(org.nd4j.linalg.api.ndarray.INDArray gradient, double oldScore, double score, int i) Check termination conditions setup a search state
ComputationGraphUpdater	getComputationGraphUpdater()
NeuralNetConfiguration	getConf()
static StepFunction	getDefaultStepFunctionForOptimizer(Class<? extends ConvexOptimizer> optimizerClass)
Updater	getUpdater()
Pair<Gradient,Double>	gradientAndScore() The gradient and score for this optimizer
boolean	optimize() Optimize call.
protected void	postFirstStep(org.nd4j.linalg.api.ndarray.INDArray gradient)
void	postStep(org.nd4j.linalg.api.ndarray.INDArray gradient) Post step to update searchDirection with new gradient and parameter information
void	preProcessLine() Pre preProcess to setup initial searchDirection approximation
double	score() The score for the optimizer so far
void	setBatchSize(int batchSize) Set the batch size for the optimizer
void	setListeners(Collection<IterationListener> listeners)
void	setUpdater(Updater updater)
void	setUpdaterComputationGraph(ComputationGraphUpdater updater)
void	setupSearchState(Pair<Gradient,Double> pair) Setup the initial search state
void	updateGradientAccordingToParams(Gradient gradient, Model model, int batchSize) Update the gradient according to the configuration such as adagrad, momentum, and sparsity

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

conf

protected NeuralNetConfiguration conf

iteration

protected int iteration

log

protected static final org.slf4j.Logger log

stepFunction

protected StepFunction stepFunction

iterationListeners

protected Collection<IterationListener> iterationListeners

terminationConditions

protected Collection<TerminationCondition> terminationConditions

model

protected Model model

lineMaximizer

protected BackTrackLineSearch lineMaximizer

updater

protected Updater updater

computationGraphUpdater

protected ComputationGraphUpdater computationGraphUpdater

step

protected double step

score

protected double score

oldScore

protected double oldScore

stepMax

protected double stepMax

GRADIENT_KEY

public static final String GRADIENT_KEY

See Also:

Constant Field Values

SCORE_KEY

public static final String SCORE_KEY

See Also:

Constant Field Values

PARAMS_KEY

public static final String PARAMS_KEY

See Also:

Constant Field Values

SEARCH_DIR

public static final String SEARCH_DIR

See Also:

Constant Field Values

searchState

protected Map<String,Object> searchState

Constructor Detail

BaseOptimizer

public BaseOptimizer(NeuralNetConfiguration conf,
                     StepFunction stepFunction,
                     Collection<IterationListener> iterationListeners,
                     Model model)

Parameters:

conf -

stepFunction -

iterationListeners -

model -

BaseOptimizer

public BaseOptimizer(NeuralNetConfiguration conf,
                     StepFunction stepFunction,
                     Collection<IterationListener> iterationListeners,
                     Collection<TerminationCondition> terminationConditions,
                     Model model)

Parameters:

conf -

stepFunction -

iterationListeners -

terminationConditions -

model -

Method Detail

score

public double score()

Description copied from interface: ConvexOptimizer

The score for the optimizer so far

Specified by:

score in interface ConvexOptimizer

Returns:

the score for this optimizer so far

getUpdater

public Updater getUpdater()

Specified by:

getUpdater in interface ConvexOptimizer

setUpdater

public void setUpdater(Updater updater)

Specified by:

setUpdater in interface ConvexOptimizer

getComputationGraphUpdater

public ComputationGraphUpdater getComputationGraphUpdater()

Specified by:

getComputationGraphUpdater in interface ConvexOptimizer

setUpdaterComputationGraph

public void setUpdaterComputationGraph(ComputationGraphUpdater updater)

Specified by:

setUpdaterComputationGraph in interface ConvexOptimizer

setListeners

public void setListeners(Collection<IterationListener> listeners)

Specified by:

setListeners in interface ConvexOptimizer

getConf

public NeuralNetConfiguration getConf()

Specified by:

getConf in interface ConvexOptimizer

gradientAndScore

public Pair<Gradient,Double> gradientAndScore()

Description copied from interface: ConvexOptimizer

The gradient and score for this optimizer

Specified by:

gradientAndScore in interface ConvexOptimizer

Returns:

the gradient and score for this optimizer

optimize

public boolean optimize()

Optimize call. This runs the optimizer.

Specified by:

optimize in interface ConvexOptimizer

Returns:

whether it converged or not

postFirstStep

protected void postFirstStep(org.nd4j.linalg.api.ndarray.INDArray gradient)

checkTerminalConditions

public boolean checkTerminalConditions(org.nd4j.linalg.api.ndarray.INDArray gradient,
                                       double oldScore,
                                       double score,
                                       int i)

Description copied from interface: ConvexOptimizer

Check termination conditions setup a search state

Specified by:

checkTerminalConditions in interface ConvexOptimizer

Parameters:

gradient - layer gradients

i - what iteration the optimizer is on

batchSize

public int batchSize()

Description copied from interface: ConvexOptimizer

The batch size for the optimizer

Specified by:

batchSize in interface ConvexOptimizer

Returns:

setBatchSize

public void setBatchSize(int batchSize)

Description copied from interface: ConvexOptimizer

Set the batch size for the optimizer

Specified by:

setBatchSize in interface ConvexOptimizer

preProcessLine

public void preProcessLine()

Pre preProcess to setup initial searchDirection approximation

Specified by:

preProcessLine in interface ConvexOptimizer

postStep

public void postStep(org.nd4j.linalg.api.ndarray.INDArray gradient)

Post step to update searchDirection with new gradient and parameter information

Specified by:

postStep in interface ConvexOptimizer

updateGradientAccordingToParams

public void updateGradientAccordingToParams(Gradient gradient,
                                            Model model,
                                            int batchSize)

Description copied from interface: ConvexOptimizer

Update the gradient according to the configuration such as adagrad, momentum, and sparsity

Specified by:

updateGradientAccordingToParams in interface ConvexOptimizer

Parameters:

gradient - the gradient to modify

model - the model with the parameters to update

batchSize - batchSize for update

setupSearchState

public void setupSearchState(Pair<Gradient,Double> pair)

Setup the initial search state

Specified by:

setupSearchState in interface ConvexOptimizer

Parameters:

pair -

getDefaultStepFunctionForOptimizer

public static StepFunction getDefaultStepFunctionForOptimizer(Class<? extends ConvexOptimizer> optimizerClass)