org.deeplearning4j.nn.updater

Class UpdaterBlock

java.lang.Object

org.deeplearning4j.nn.updater.UpdaterBlock

public class UpdaterBlock
extends Object

UpdaterBlock: used in BaseMultiLayerUpdater, this class implements updating (i.e., Adam, RMSProp, Momentum, etc) across multiple contiguous layers/parameters, as described in the BaseMultiLayerUpdater javadoc.

Author:

Alex Black

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	UpdaterBlock.ParamState

Constructor Summary

Constructors

Constructor and Description
UpdaterBlock(int paramOffsetStart, int paramOffsetEnd, int updaterViewOffsetStart, int updaterViewOffsetEnd, List<UpdaterBlock.ParamState> layersAndVariablesInBlock)

Method Summary

Modifier and Type	Method and Description
GradientUpdater	getGradientUpdater()
void	init()
boolean	isPretrainUpdaterBlock()
void	postApply(Trainable layer, String paramName, INDArray gradientView, INDArray paramsView) Apply L1 and L2 regularization, if necessary.
boolean	skipDueToPretrainConfig()
void	update(int iteration, int epoch) Update the gradient for this block
void	updateExternalGradient(int iteration, int epoch, INDArray fullNetworkGradientView, INDArray fullNetworkParamsArray)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

UpdaterBlock

public UpdaterBlock(int paramOffsetStart,
                    int paramOffsetEnd,
                    int updaterViewOffsetStart,
                    int updaterViewOffsetEnd,
                    List<UpdaterBlock.ParamState> layersAndVariablesInBlock)

Parameters:

paramOffsetStart - Start offset of the parameters in this block (relative to overall net params view array)

paramOffsetEnd - End offset of the parameters in this block (relative to overall net params view array)

updaterViewOffsetStart - Start offset of the updater state array in this block (relative to overall net updater state view array)

updaterViewOffsetEnd - End offset of the updater state array in this block (relative to overall net updater state view array)

layersAndVariablesInBlock - List of layers and variables in this updater block. By definition, all layers and variables in this list must have an identical updater configuration.

Method Detail

init

public void init()

isPretrainUpdaterBlock

public boolean isPretrainUpdaterBlock()

skipDueToPretrainConfig

public boolean skipDueToPretrainConfig()

getGradientUpdater

public GradientUpdater getGradientUpdater()

update

public void update(int iteration,
                   int epoch)

Update the gradient for this block

Parameters:

iteration - The current iteration (i.e., total number of parameter updates so far)

updateExternalGradient

public void updateExternalGradient(int iteration,
                                   int epoch,
                                   INDArray fullNetworkGradientView,
                                   INDArray fullNetworkParamsArray)

postApply

public void postApply(Trainable layer,
                      String paramName,
                      INDArray gradientView,
                      INDArray paramsView)

Apply L1 and L2 regularization, if necessary. Note that L1/L2 may differ for different layers in the same block

Parameters:

layer - The layer to apply L1/L2 to

paramName - Parameter name in the given layer

gradientView - Gradient view array for the layer + param

paramsView - Parameter view array for the layer + param