org.deeplearning4j.nn.params

Class DefaultParamInitializer

java.lang.Object

org.deeplearning4j.nn.params.DefaultParamInitializer

All Implemented Interfaces:

ParamInitializer

Direct Known Subclasses:

CenterLossParamInitializer, ElementWiseParamInitializer, EmbeddingLayerParamInitializer, OCNNParamInitializer, PretrainParamInitializer, VariationalAutoencoderParamInitializer

public class DefaultParamInitializer
extends Object
implements ParamInitializer

Static weight initializer with just a weight matrix and a bias

Author:

Adam Gibson

Field Summary

Fields

Modifier and Type	Field and Description
static String	BIAS_KEY
static String	GAIN_KEY
static String	WEIGHT_KEY

Constructor Summary

Constructors

Constructor and Description
DefaultParamInitializer()

Method Summary

Modifier and Type	Method and Description
List<String>	biasKeys(Layer layer) Bias parameter keys given the layer configuration
protected INDArray	createBias(long nOut, double biasInit, INDArray biasParamView, boolean initializeParameters)
protected INDArray	createBias(NeuralNetConfiguration conf, INDArray biasParamView, boolean initializeParameters)
protected INDArray	createGain(long nOut, double gainInit, INDArray gainParamView, boolean initializeParameters)
protected INDArray	createGain(NeuralNetConfiguration conf, INDArray gainParamView, boolean initializeParameters)
protected INDArray	createWeightMatrix(long nIn, long nOut, IWeightInit weightInit, INDArray weightParamView, boolean initializeParameters)
protected INDArray	createWeightMatrix(NeuralNetConfiguration conf, INDArray weightParamView, boolean initializeParameters)
Map<String,INDArray>	getGradientsFromFlattened(NeuralNetConfiguration conf, INDArray gradientView) Return a map of gradients (in their standard non-flattened representation), taken from the flattened (row vector) gradientView array.
static DefaultParamInitializer	getInstance()
protected boolean	hasBias(Layer layer)
protected boolean	hasLayerNorm(Layer layer)
Map<String,INDArray>	init(NeuralNetConfiguration conf, INDArray paramsView, boolean initializeParams) Initialize the parameters
boolean	isBiasParam(Layer layer, String key) Is the specified parameter a bias?
boolean	isWeightParam(Layer layer, String key) Is the specified parameter a weight?
long	numParams(Layer l)
long	numParams(NeuralNetConfiguration conf)
List<String>	paramKeys(Layer layer) Get a list of all parameter keys given the layer configuration
List<String>	weightKeys(Layer layer) Weight parameter keys given the layer configuration

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

WEIGHT_KEY

public static final String WEIGHT_KEY

See Also:

Constant Field Values

BIAS_KEY

public static final String BIAS_KEY

See Also:

Constant Field Values

GAIN_KEY

public static final String GAIN_KEY

See Also:

Constant Field Values

Constructor Detail

DefaultParamInitializer

public DefaultParamInitializer()

Method Detail

getInstance

public static DefaultParamInitializer getInstance()

numParams

public long numParams(NeuralNetConfiguration conf)

Specified by:

numParams in interface ParamInitializer

numParams

public long numParams(Layer l)

Specified by:

numParams in interface ParamInitializer

paramKeys

public List<String> paramKeys(Layer layer)

Description copied from interface: ParamInitializer

Get a list of all parameter keys given the layer configuration

Specified by:

paramKeys in interface ParamInitializer

Parameters:

layer - Layer

Returns:

All parameter keys

weightKeys

public List<String> weightKeys(Layer layer)

Description copied from interface: ParamInitializer

Weight parameter keys given the layer configuration

Specified by:

weightKeys in interface ParamInitializer

Parameters:

layer - Layer

Returns:

Weight parameter keys

biasKeys

public List<String> biasKeys(Layer layer)

Description copied from interface: ParamInitializer

Bias parameter keys given the layer configuration

Specified by:

biasKeys in interface ParamInitializer

Parameters:

layer - Layer

Returns:

Bias parameter keys

isWeightParam

public boolean isWeightParam(Layer layer,
                             String key)

Description copied from interface: ParamInitializer

Is the specified parameter a weight?

Specified by:

isWeightParam in interface ParamInitializer

Parameters:

layer - Layer

key - Key to check

Returns:

True if parameter is a weight

isBiasParam

public boolean isBiasParam(Layer layer,
                           String key)

Description copied from interface: ParamInitializer

Is the specified parameter a bias?

Specified by:

isBiasParam in interface ParamInitializer

Parameters:

layer - Layer

key - Key to check

Returns:

True if parameter is a bias

init

public Map<String,INDArray> init(NeuralNetConfiguration conf,
                                 INDArray paramsView,
                                 boolean initializeParams)

Description copied from interface: ParamInitializer

Initialize the parameters

Specified by:

init in interface ParamInitializer

Parameters:

conf - the configuration

paramsView - a view of the full network (backprop) parameters

initializeParams - if true: initialize the parameters according to the configuration. If false: don't modify the values in the paramsView array (but do select out the appropriate subset, reshape etc as required)

Returns:

Map of parameters keyed by type (view of the 'paramsView' array)

getGradientsFromFlattened

public Map<String,INDArray> getGradientsFromFlattened(NeuralNetConfiguration conf,
                                                      INDArray gradientView)

Description copied from interface: ParamInitializer

Return a map of gradients (in their standard non-flattened representation), taken from the flattened (row vector) gradientView array. The idea is that operates in exactly the same way as the paramsView does in #init(Map, NeuralNetConfiguration, INDArray); thus the position in the view (and, the array orders) must match those of the parameters

Specified by:

getGradientsFromFlattened in interface ParamInitializer

Parameters:

conf - Configuration

gradientView - The flattened gradients array, as a view of the larger array

Returns:

A map containing an array by parameter type, that is a view of the full network gradients array

createBias

protected INDArray createBias(NeuralNetConfiguration conf,
                              INDArray biasParamView,
                              boolean initializeParameters)

createBias

protected INDArray createBias(long nOut,
                              double biasInit,
                              INDArray biasParamView,
                              boolean initializeParameters)

createGain

protected INDArray createGain(NeuralNetConfiguration conf,
                              INDArray gainParamView,
                              boolean initializeParameters)

createGain

protected INDArray createGain(long nOut,
                              double gainInit,
                              INDArray gainParamView,
                              boolean initializeParameters)

createWeightMatrix

protected INDArray createWeightMatrix(NeuralNetConfiguration conf,
                                      INDArray weightParamView,
                                      boolean initializeParameters)

createWeightMatrix

protected INDArray createWeightMatrix(long nIn,
                                      long nOut,
                                      IWeightInit weightInit,
                                      INDArray weightParamView,
                                      boolean initializeParameters)

hasBias

protected boolean hasBias(Layer layer)

hasLayerNorm

protected boolean hasLayerNorm(Layer layer)