org.deeplearning4j.optimize.solvers

Class BaseOptimizer

java.lang.Object

org.deeplearning4j.optimize.solvers.BaseOptimizer

All Implemented Interfaces:

Serializable, ConvexOptimizer

Direct Known Subclasses:

ConjugateGradient, LBFGS, LineGradientDescent, StochasticGradientDescent

public abstract class BaseOptimizer
extends Object
implements ConvexOptimizer

Base optimizer

Author:

Adam Gibson

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
protected GradientsAccumulator	accumulator
protected ComputationGraphUpdater	computationGraphUpdater
protected NeuralNetConfiguration	conf
static String	GRADIENT_KEY
protected BackTrackLineSearch	lineMaximizer
protected static org.slf4j.Logger	log
protected Model	model
protected double	oldScore
static String	PARAMS_KEY
protected double	score
static String	SCORE_KEY
static String	SEARCH_DIR
protected Map<String,Object>	searchState
protected double	step
protected StepFunction	stepFunction
protected double	stepMax
protected Collection<TrainingListener>	trainingListeners
protected Updater	updater

Constructor Summary

Constructors

Constructor and Description
BaseOptimizer(NeuralNetConfiguration conf, StepFunction stepFunction, Collection<TrainingListener> trainingListeners, Model model)

Method Summary

Modifier and Type	Method and Description
static void	applyConstraints(Model model)
int	batchSize() The batch size for the optimizer
ComputationGraphUpdater	getComputationGraphUpdater()
ComputationGraphUpdater	getComputationGraphUpdater(boolean initializIfReq)
NeuralNetConfiguration	getConf()
static StepFunction	getDefaultStepFunctionForOptimizer(Class<? extends ConvexOptimizer> optimizerClass)
static int	getEpochCount(Model model)
GradientsAccumulator	getGradientsAccumulator() This method returns GradientsAccumulator instance used in this optimizer.
static int	getIterationCount(Model model)
Updater	getUpdater()
Updater	getUpdater(boolean initializeIfReq)
Pair<Gradient,Double>	gradientAndScore(LayerWorkspaceMgr workspaceMgr) The gradient and score for this optimizer
static void	incrementIterationCount(Model model, int incrementBy)
boolean	optimize(LayerWorkspaceMgr workspaceMgr) Optimize call.
protected void	postFirstStep(INDArray gradient)
void	postStep(INDArray gradient) Post step to update searchDirection with new gradient and parameter information
void	preProcessLine() Pre preProcess to setup initial searchDirection approximation
double	score() The score for the optimizer so far
void	setBatchSize(int batchSize) Set the batch size for the optimizer
void	setGradientsAccumulator(GradientsAccumulator accumulator) This method specifies GradientsAccumulator instance to be used for updates sharing across multiple models
void	setListeners(Collection<TrainingListener> listeners)
void	setUpdater(Updater updater)
void	setUpdaterComputationGraph(ComputationGraphUpdater updater)
void	setupSearchState(Pair<Gradient,Double> pair) Setup the initial search state
void	updateGradientAccordingToParams(Gradient gradient, Model model, int batchSize, LayerWorkspaceMgr workspaceMgr) Update the gradient according to the configuration such as adagrad, momentum, and sparsity

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.deeplearning4j.optimize.api.ConvexOptimizer

getStepFunction

Field Detail

conf

protected NeuralNetConfiguration conf

log

protected static final org.slf4j.Logger log

stepFunction

protected StepFunction stepFunction

trainingListeners

protected Collection<TrainingListener> trainingListeners

model

protected Model model

lineMaximizer

protected BackTrackLineSearch lineMaximizer

updater

protected Updater updater

computationGraphUpdater

protected ComputationGraphUpdater computationGraphUpdater

step

protected double step

score

protected double score

oldScore

protected double oldScore

stepMax

protected double stepMax

GRADIENT_KEY

public static final String GRADIENT_KEY

See Also:

Constant Field Values

SCORE_KEY

public static final String SCORE_KEY

See Also:

Constant Field Values

PARAMS_KEY

public static final String PARAMS_KEY

See Also:

Constant Field Values

SEARCH_DIR

public static final String SEARCH_DIR

See Also:

Constant Field Values

searchState

protected Map<String,Object> searchState

accumulator

protected GradientsAccumulator accumulator

Constructor Detail

BaseOptimizer

public BaseOptimizer(NeuralNetConfiguration conf,
                     StepFunction stepFunction,
                     Collection<TrainingListener> trainingListeners,
                     Model model)

Parameters:

conf -

stepFunction -

trainingListeners -

model -

Method Detail

setGradientsAccumulator

public void setGradientsAccumulator(GradientsAccumulator accumulator)

Description copied from interface: ConvexOptimizer

This method specifies GradientsAccumulator instance to be used for updates sharing across multiple models

Specified by:

setGradientsAccumulator in interface ConvexOptimizer

getGradientsAccumulator

public GradientsAccumulator getGradientsAccumulator()

Description copied from interface: ConvexOptimizer

This method returns GradientsAccumulator instance used in this optimizer. This method can return null.

Specified by:

getGradientsAccumulator in interface ConvexOptimizer

Returns:

score

public double score()

Description copied from interface: ConvexOptimizer

The score for the optimizer so far

Specified by:

score in interface ConvexOptimizer

Returns:

the score for this optimizer so far

getUpdater

public Updater getUpdater()

Specified by:

getUpdater in interface ConvexOptimizer

getUpdater

public Updater getUpdater(boolean initializeIfReq)

Specified by:

getUpdater in interface ConvexOptimizer

setUpdater

public void setUpdater(Updater updater)

Specified by:

setUpdater in interface ConvexOptimizer

getComputationGraphUpdater

public ComputationGraphUpdater getComputationGraphUpdater()

Specified by:

getComputationGraphUpdater in interface ConvexOptimizer

getComputationGraphUpdater

public ComputationGraphUpdater getComputationGraphUpdater(boolean initializIfReq)

Specified by:

getComputationGraphUpdater in interface ConvexOptimizer

setUpdaterComputationGraph

public void setUpdaterComputationGraph(ComputationGraphUpdater updater)

Specified by:

setUpdaterComputationGraph in interface ConvexOptimizer

setListeners

public void setListeners(Collection<TrainingListener> listeners)

Specified by:

setListeners in interface ConvexOptimizer

getConf

public NeuralNetConfiguration getConf()

Specified by:

getConf in interface ConvexOptimizer

gradientAndScore

public Pair<Gradient,Double> gradientAndScore(LayerWorkspaceMgr workspaceMgr)

Description copied from interface: ConvexOptimizer

The gradient and score for this optimizer

Specified by:

gradientAndScore in interface ConvexOptimizer

Returns:

the gradient and score for this optimizer

optimize

public boolean optimize(LayerWorkspaceMgr workspaceMgr)

Optimize call. This runs the optimizer.

Specified by:

optimize in interface ConvexOptimizer

Returns:

whether it converged or not

postFirstStep

protected void postFirstStep(INDArray gradient)

batchSize

public int batchSize()

Description copied from interface: ConvexOptimizer

The batch size for the optimizer

Specified by:

batchSize in interface ConvexOptimizer

Returns:

setBatchSize

public void setBatchSize(int batchSize)

Description copied from interface: ConvexOptimizer

Set the batch size for the optimizer

Specified by:

setBatchSize in interface ConvexOptimizer

preProcessLine

public void preProcessLine()

Pre preProcess to setup initial searchDirection approximation

Specified by:

preProcessLine in interface ConvexOptimizer

postStep

public void postStep(INDArray gradient)

Post step to update searchDirection with new gradient and parameter information

Specified by:

postStep in interface ConvexOptimizer

updateGradientAccordingToParams

public void updateGradientAccordingToParams(Gradient gradient,
                                            Model model,
                                            int batchSize,
                                            LayerWorkspaceMgr workspaceMgr)

Description copied from interface: ConvexOptimizer

Update the gradient according to the configuration such as adagrad, momentum, and sparsity

Specified by:

updateGradientAccordingToParams in interface ConvexOptimizer

Parameters:

gradient - the gradient to modify

model - the model with the parameters to update

batchSize - batchSize for update

setupSearchState

public void setupSearchState(Pair<Gradient,Double> pair)

Setup the initial search state

Specified by:

setupSearchState in interface ConvexOptimizer

Parameters:

pair -

getDefaultStepFunctionForOptimizer

public static StepFunction getDefaultStepFunctionForOptimizer(Class<? extends ConvexOptimizer> optimizerClass)

getIterationCount

public static int getIterationCount(Model model)

incrementIterationCount

public static void incrementIterationCount(Model model,
                                           int incrementBy)

getEpochCount

public static int getEpochCount(Model model)

applyConstraints

public static void applyConstraints(Model model)