Package org.deeplearning4j.nn.layers

Class BaseLayer<LayerConfT extends BaseLayer>

    • Method Detail

      • backpropGradient

        public Pair<Gradient,​INDArray> backpropGradient​(INDArray epsilon,
                                                      LayerWorkspaceMgr workspaceMgr)
        Description copied from interface: Layer
        Calculate the gradient relative to the error in the next layer
        Parameters:
        epsilon - w^(L+1)*delta^(L+1). Or, equiv: dC/da, i.e., (dC/dz)*(dz/da) = dC/da, where C is cost function a=sigma(z) is activation.
        workspaceMgr - Workspace manager
        Returns:
        Pair where Gradient is gradient for this layer, INDArray is epsilon (activation gradient) needed by next layer, but before element-wise multiply by sigmaPrime(z). So for standard feed-forward layer, if this layer is L, then return.getSecond() == dL/dIn = (w^(L)*(delta^(L))^T)^T. Note that the returned array should be placed in the ArrayType.ACTIVATION_GRAD workspace via the workspace manager

      • setScoreWithZ

        protected void setScoreWithZ​(INDArray z)

      • setParams

        public void setParams​(INDArray params)
        Description copied from interface: Model
        Set the parameters for this model. This expects a linear ndarray which then be unpacked internally relative to the expected ordering of the model
        Specified by:
        setParams in interface Model
        Overrides:
        setParams in class AbstractLayer<LayerConfT extends BaseLayer>
        Parameters:
        params - the parameters for the model

      • setParamsViewArray

        public void setParamsViewArray​(INDArray params)
        Description copied from interface: Model
        Set the initial parameters array as a view of the full (backprop) network parameters NOTE: this is intended to be used internally in MultiLayerNetwork and ComputationGraph, not by users.
        Specified by:
        setParamsViewArray in interface Model
        Overrides:
        setParamsViewArray in class AbstractLayer<LayerConfT extends BaseLayer>
        Parameters:
        params - a 1 x nParams row vector that is a view of the larger (MLN/CG) parameters array

      • setBackpropGradientsViewArray

        public void setBackpropGradientsViewArray​(INDArray gradients)
        Description copied from interface: Model
        Set the gradients array as a view of the full (backprop) network parameters NOTE: this is intended to be used internally in MultiLayerNetwork and ComputationGraph, not by users.
        Specified by:
        setBackpropGradientsViewArray in interface Model
        Overrides:
        setBackpropGradientsViewArray in class AbstractLayer<LayerConfT extends BaseLayer>
        Parameters:
        gradients - a 1 x nParams row vector that is a view of the larger (MLN/CG) gradients array

      • paramTable

        public Map<String,​INDArray> paramTable​(boolean backpropParamsOnly)
        Description copied from interface: Model
        Table of parameters by key, for backprop For many models (dense layers, etc) - all parameters are backprop parameters
        Specified by:
        paramTable in interface Model
        Specified by:
        paramTable in interface Trainable
        Overrides:
        paramTable in class AbstractLayer<LayerConfT extends BaseLayer>
        Parameters:
        backpropParamsOnly - If true, return backprop params only. If false: return all params (equivalent to paramsTable())
        Returns:
        Parameter table

      • getParamWithNoise

        protected INDArray getParamWithNoise​(String param,
                                     boolean training,
                                     LayerWorkspaceMgr workspaceMgr)
        Get the parameter, after applying any weight noise (such as DropConnect) if necessary. Note that during training, this will store the post-noise parameters, as these should be used for both forward pass and backprop, for a single iteration. Consequently, the parameters (post noise) should be cleared after each training iteration
        Parameters:
        param - Parameter key
        training - If true: during training
        Returns:
        The parameter, after applying any noise

      • activate

        public INDArray activate​(boolean training,
                         LayerWorkspaceMgr workspaceMgr)
        Description copied from interface: Layer
        Perform forward pass and return the activations array with the last set input
        Parameters:
        training - training or test mode
        workspaceMgr - Workspace manager
        Returns:
        the activation (layer output) of the last specified input. Note that the returned array should be placed in the ArrayType.ACTIVATIONS workspace via the workspace manager

      • calcRegularizationScore

        public double calcRegularizationScore​(boolean backpropParamsOnly)
        Description copied from interface: Layer
        Calculate the regularization component of the score, for the parameters in this layer
        For example, the L1, L2 and/or weight decay components of the loss function
        Specified by:
        calcRegularizationScore in interface Layer
        Overrides:
        calcRegularizationScore in class AbstractLayer<LayerConfT extends BaseLayer>
        Parameters:
        backpropParamsOnly - If true: calculate regularization score based on backprop params only. If false: calculate based on all params (including pretrain params, if any)
        Returns:
        the regularization score of

      • clearNoiseWeightParams

        public void clearNoiseWeightParams()

      • hasBias

        public boolean hasBias()
        Does this layer have no bias term? Many layers (dense, convolutional, output, embedding) have biases by default, but no-bias versions are possible via configuration
        Returns:
        True if a bias term is present, false otherwise

      • hasLayerNorm

        public boolean hasLayerNorm()
        Does this layer support and is it enabled layer normalization? Only Dense and SimpleRNN Layers support layer normalization.
        Returns:
        True if layer normalization is enabled on this layer, false otherwise