Package org.deeplearning4j.nn.conf

Class NeuralNetConfiguration.Builder

  • All Implemented Interfaces:
    Cloneable
    Enclosing class:
    NeuralNetConfiguration
    public static class NeuralNetConfiguration.Builder
extends Object
implements Cloneable
    NeuralNetConfiguration builder, used as a starting point for creating a MultiLayerConfiguration or ComputationGraphConfiguration.
    Note that values set here on the layer will be applied to all relevant layers - unless the value is overridden on a layer's configuration

    • Method Detail

      • trainingWorkspaceMode

        public NeuralNetConfiguration.Builder trainingWorkspaceMode​(@NonNull
                                                            @NonNull WorkspaceMode workspaceMode)
        This method defines Workspace mode being used during training:
        NONE: workspace won't be used
        ENABLED: workspaces will be used for training (reduced memory and better performance)
        Parameters:
        workspaceMode - Workspace mode for training
        Returns:
        Builder

      • inferenceWorkspaceMode

        public NeuralNetConfiguration.Builder inferenceWorkspaceMode​(@NonNull
                                                             @NonNull WorkspaceMode workspaceMode)
        This method defines Workspace mode being used during inference:
        NONE: workspace won't be used
        ENABLED: workspaces will be used for inference (reduced memory and better performance)
        Parameters:
        workspaceMode - Workspace mode for inference
        Returns:
        Builder

      • cacheMode

        public NeuralNetConfiguration.Builder cacheMode​(@NonNull
                                                @NonNull CacheMode cacheMode)
        This method defines how/if preOutput cache is handled: NONE: cache disabled (default value) HOST: Host memory will be used DEVICE: GPU memory will be used (on CPU backends effect will be the same as for HOST)
        Parameters:
        cacheMode - Cache mode to use
        Returns:
        Builder

      • minimize

        public NeuralNetConfiguration.Builder minimize​(boolean minimize)
        Objective function to minimize or maximize cost function Default set to minimize true.

      • maxNumLineSearchIterations

        public NeuralNetConfiguration.Builder maxNumLineSearchIterations​(int maxNumLineSearchIterations)
        Maximum number of line search iterations. Only applies for line search optimizers: Line Search SGD, Conjugate Gradient, LBFGS is NOT applicable for standard SGD
        Parameters:
        maxNumLineSearchIterations - > 0
        Returns:

      • stepFunction

        @Deprecated
public NeuralNetConfiguration.Builder stepFunction​(StepFunction stepFunction)
        Deprecated.
        Step function to apply for back track line search. Only applies for line search optimizers: Line Search SGD, Conjugate Gradient, LBFGS Options: DefaultStepFunction (default), NegativeDefaultStepFunction GradientStepFunction (for SGD), NegativeGradientStepFunction

      • list

        public NeuralNetConfiguration.ListBuilder list()
        Create a ListBuilder (for creating a MultiLayerConfiguration)
        Usage:
         .list()
 .layer(new DenseLayer.Builder()...build())
 ...
 .layer(new OutputLayer.Builder()...build())

      • list

        public NeuralNetConfiguration.ListBuilder list​(Layer... layers)
        Create a ListBuilder (for creating a MultiLayerConfiguration) with the specified layers
        Usage:
         .list(
      new DenseLayer.Builder()...build(),
      ...,
      new OutputLayer.Builder()...build())
        Parameters:
        layers - The layer configurations for the network

      • optimizationAlgo

        public NeuralNetConfiguration.Builder optimizationAlgo​(OptimizationAlgorithm optimizationAlgo)
        Optimization algorithm to use. Most common: OptimizationAlgorithm.STOCHASTIC_GRADIENT_DESCENT
        Parameters:
        optimizationAlgo - Optimization algorithm to use when training

      • activation

        public NeuralNetConfiguration.Builder activation​(IActivation activationFunction)
        Activation function / neuron non-linearity
        Note: values set by this method will be applied to all applicable layers in the network, unless a different value is explicitly set on a given layer. In other words: values set via this method are used as the default value, and can be overridden on a per-layer basis.
        See Also:
        activation(Activation)

      • activation

        public NeuralNetConfiguration.Builder activation​(Activation activation)
        Activation function / neuron non-linearity
        Note: values set by this method will be applied to all applicable layers in the network, unless a different value is explicitly set on a given layer. In other words: values set via this method are used as the default value, and can be overridden on a per-layer basis.

      • weightInit

        public NeuralNetConfiguration.Builder weightInit​(IWeightInit weightInit)
        Weight initialization scheme to use, for initial weight values Note: values set by this method will be applied to all applicable layers in the network, unless a different value is explicitly set on a given layer. In other words: values set via this method are used as the default value, and can be overridden on a per-layer basis.
        See Also:
        IWeightInit

      • weightInit

        public NeuralNetConfiguration.Builder weightInit​(WeightInit weightInit)
        Weight initialization scheme to use, for initial weight values Note: values set by this method will be applied to all applicable layers in the network, unless a different value is explicitly set on a given layer. In other words: values set via this method are used as the default value, and can be overridden on a per-layer basis.
        See Also:
        WeightInit

      • weightInit

        public NeuralNetConfiguration.Builder weightInit​(Distribution distribution)
        Set weight initialization scheme to random sampling via the specified distribution. Equivalent to: .weightInit(new WeightInitDistribution(distribution)) Note: values set by this method will be applied to all applicable layers in the network, unless a different value is explicitly set on a given layer. In other words: values set via this method are used as the default value, and can be overridden on a per-layer basis.
        Parameters:
        distribution - Distribution to use for weight initialization

      • biasInit

        public NeuralNetConfiguration.Builder biasInit​(double biasInit)
        Constant for bias initialization. Default: 0.0
        Note: values set by this method will be applied to all applicable layers in the network, unless a different value is explicitly set on a given layer. In other words: values set via this method are used as the default value, and can be overridden on a per-layer basis.
        Parameters:
        biasInit - Constant for bias initialization

      • dist

        @Deprecated
public NeuralNetConfiguration.Builder dist​(Distribution dist)
        Deprecated.
        Use weightInit(Distribution)
        Distribution to sample initial weights from. Equivalent to: .weightInit(new WeightInitDistribution(distribution)).
        Note: values set by this method will be applied to all applicable layers in the network, unless a different value is explicitly set on a given layer. In other words: values set via this method are used as the default value, and can be overridden on a per-layer basis.
        See Also:
        weightInit(Distribution)

      • l1

        public NeuralNetConfiguration.Builder l1​(double l1)
        L1 regularization coefficient for the weights (excluding biases).
        Note: values set by this method will be applied to all applicable layers in the network, unless a different value is explicitly set on a given layer. In other words: values set via this method are used as the default value, and can be overridden on a per-layer basis.

      • l2

        public NeuralNetConfiguration.Builder l2​(double l2)
        L2 regularization coefficient for the weights (excluding biases).
        Note: Generally, WeightDecay (set via weightDecay(double) should be preferred to L2 regularization. See WeightDecay javadoc for further details.
        Note: values set by this method will be applied to all applicable layers in the network, unless a different value is explicitly set on a given layer. In other words: values set via this method are used as the default value, and can be overridden on a per-layer basis.
        Note: L2 regularization and weight decay usually should not be used together; if any weight decay (or L2) has been added for the biases, these will be removed first.
        See Also:
        weightDecay(double, boolean)

      • l1Bias

        public NeuralNetConfiguration.Builder l1Bias​(double l1Bias)
        L1 regularization coefficient for the bias.
        Note: values set by this method will be applied to all applicable layers in the network, unless a different value is explicitly set on a given layer. In other words: values set via this method are used as the default value, and can be overridden on a per-layer basis.

      • l2Bias

        public NeuralNetConfiguration.Builder l2Bias​(double l2Bias)
        L2 regularization coefficient for the bias.
        Note: Generally, WeightDecay (set via weightDecayBias(double,boolean) should be preferred to L2 regularization. See WeightDecay javadoc for further details.
        Note: values set by this method will be applied to all applicable layers in the network, unless a different value is explicitly set on a given layer. In other words: values set via this method are used as the default value, and can be overridden on a per-layer basis.
        Note: L2 regularization and weight decay usually should not be used together; if any weight decay (or L2) has been added for the biases, these will be removed first.
        See Also:
        weightDecayBias(double, boolean)

      • weightDecay

        public NeuralNetConfiguration.Builder weightDecay​(double coefficient)
        Add weight decay regularization for the network parameters (excluding biases).
        This applies weight decay with multiplying the learning rate - see WeightDecay for more details.
        Note: values set by this method will be applied to all applicable layers in the network, unless a different value is explicitly set on a given layer. In other words: values set via this method are used as the default value, and can be overridden on a per-layer basis.
        Parameters:
        coefficient - Weight decay regularization coefficient
        See Also:
        weightDecay(double, boolean)

      • weightDecay

        public NeuralNetConfiguration.Builder weightDecay​(double coefficient,
                                                  boolean applyLR)
        Add weight decay regularization for the network parameters (excluding biases). See WeightDecay for more details.
        Note: values set by this method will be applied to all applicable layers in the network, unless a different value is explicitly set on a given layer. In other words: values set via this method are used as the default value, and can be overridden on a per-layer basis.
        Parameters:
        coefficient - Weight decay regularization coefficient
        applyLR - Whether the learning rate should be multiplied in when performing weight decay updates. See WeightDecay for more details.
        See Also:
        weightDecay(double, boolean)

      • weightDecayBias

        public NeuralNetConfiguration.Builder weightDecayBias​(double coefficient)
        Weight decay for the biases only - see weightDecay(double) for more details. This applies weight decay with multiplying the learning rate.
        Note: values set by this method will be applied to all applicable layers in the network, unless a different value is explicitly set on a given layer. In other words: values set via this method are used as the default value, and can be overridden on a per-layer basis.
        Parameters:
        coefficient - Weight decay regularization coefficient
        See Also:
        weightDecayBias(double, boolean)

      • weightDecayBias

        public NeuralNetConfiguration.Builder weightDecayBias​(double coefficient,
                                                      boolean applyLR)
        Weight decay for the biases only - see weightDecay(double) for more details
        Note: values set by this method will be applied to all applicable layers in the network, unless a different value is explicitly set on a given layer. In other words: values set via this method are used as the default value, and can be overridden on a per-layer basis.
        Parameters:
        coefficient - Weight decay regularization coefficient

      • regularization

        public NeuralNetConfiguration.Builder regularization​(List<Regularization> regularization)
        Set the regularization for the parameters (excluding biases) - for example WeightDecay
        Note: values set by this method will be applied to all applicable layers in the network, unless a different value is explicitly set on a given layer. In other words: values set via this method are used as the default value, and can be overridden on a per-layer basis.
        Parameters:
        regularization - Regularization to apply for the network parameters/weights (excluding biases)

      • regularizationBias

        public NeuralNetConfiguration.Builder regularizationBias​(List<Regularization> regularizationBias)
        Set the regularization for the biases only - for example WeightDecay
        Note: values set by this method will be applied to all applicable layers in the network, unless a different value is explicitly set on a given layer. In other words: values set via this method are used as the default value, and can be overridden on a per-layer basis.
        Parameters:
        regularizationBias - Regularization to apply for the network biases only

      • dropOut

        public NeuralNetConfiguration.Builder dropOut​(double inputRetainProbability)
        Dropout probability. This is the probability of retaining each input activation value for a layer. dropOut(x) will keep an input activation with probability x, and set to 0 with probability 1-x.
        dropOut(0.0) is a special value / special case - when set to 0.0., dropout is disabled (not applied). Note that a dropout value of 1.0 is functionally equivalent to no dropout: i.e., 100% probability of retaining each input activation.

        Note 1: Dropout is applied at training time only - and is automatically not applied at test time (for evaluation, etc)
        Note 2: This sets the probability per-layer. Care should be taken when setting lower values for complex networks (too much information may be lost with aggressive (very low) dropout values).
        Note 3: Frequently, dropout is not applied to (or, has higher retain probability for) input (first layer) layers. Dropout is also often not applied to output layers. This needs to be handled MANUALLY by the user - set .dropout(0) on those layers when using global dropout setting.
        Note 4: Implementation detail (most users can ignore): DL4J uses inverted dropout, as described here: http://cs231n.github.io/neural-networks-2/


        Note: values set by this method will be applied to all applicable layers in the network, unless a different value is explicitly set on a given layer. In other words: values set via this method are used as the default value, and can be overridden on a per-layer basis.
        Parameters:
        inputRetainProbability - Dropout probability (probability of retaining each input activation value for a layer)
        See Also:
        dropOut(IDropout)

      • dropOut

        public NeuralNetConfiguration.Builder dropOut​(IDropout dropout)
        Set the dropout for all layers in this network
        Note: values set by this method will be applied to all applicable layers in the network, unless a different value is explicitly set on a given layer. In other words: values set via this method are used as the default value, and can be overridden on a per-layer basis.
        Parameters:
        dropout - Dropout, such as Dropout, GaussianDropout, GaussianNoise etc
        Returns:

      • weightNoise

        public NeuralNetConfiguration.Builder weightNoise​(IWeightNoise weightNoise)
        Set the weight noise (such as DropConnect and WeightNoise) for the layers in this network.
        Note: values set by this method will be applied to all applicable layers in the network, unless a different value is explicitly set on a given layer. In other words: values set via this method are used as the default value, and can be overridden on a per-layer basis.
        Parameters:
        weightNoise - Weight noise instance to use

      • updater

        public NeuralNetConfiguration.Builder updater​(IUpdater updater)
        Gradient updater configuration. For example, Adam or Nesterovs
        Note: values set by this method will be applied to all applicable layers in the network, unless a different value is explicitly set on a given layer. In other words: values set via this method are used as the default value, and can be overridden on a per-layer basis.
        Parameters:
        updater - Updater to use

      • biasUpdater

        public NeuralNetConfiguration.Builder biasUpdater​(IUpdater updater)
        Gradient updater configuration, for the biases only. If not set, biases will use the updater as set by updater(IUpdater)
        Note: values set by this method will be applied to all applicable layers in the network, unless a different value is explicitly set on a given layer. In other words: values set via this method are used as the default value, and can be overridden on a per-layer basis.
        Parameters:
        updater - Updater to use for bias parameters

      • gradientNormalization

        public NeuralNetConfiguration.Builder gradientNormalization​(GradientNormalization gradientNormalization)
        Gradient normalization strategy. Used to specify gradient renormalization, gradient clipping etc. See GradientNormalization for details
        Note: values set by this method will be applied to all applicable layers in the network, unless a different value is explicitly set on a given layer. In other words: values set via this method are used as the default value, and can be overridden on a per-layer basis.
        Parameters:
        gradientNormalization - Type of normalization to use. Defaults to None.
        See Also:
        GradientNormalization

      • gradientNormalizationThreshold

        public NeuralNetConfiguration.Builder gradientNormalizationThreshold​(double threshold)
        Threshold for gradient normalization, only used for GradientNormalization.ClipL2PerLayer, GradientNormalization.ClipL2PerParamType, and GradientNormalization.ClipElementWiseAbsoluteValue
        Not used otherwise.
        L2 threshold for first two types of clipping, or absolute value threshold for last type of clipping.
        Note: values set by this method will be applied to all applicable layers in the network, unless a different value is explicitly set on a given layer. In other words: values set via this method are used as the default value, and can be overridden on a per-layer basis.

      • convolutionMode

        public NeuralNetConfiguration.Builder convolutionMode​(ConvolutionMode convolutionMode)
        Sets the convolution mode for convolutional layers, which impacts padding and output sizes. See ConvolutionMode for details. Defaults to ConvolutionMode.TRUNCATE
        Note: values set by this method will be applied to all applicable layers in the network, unless a different value is explicitly set on a given layer. In other words: values set via this method are used as the default value, and can be overridden on a per-layer basis.
        Parameters:
        convolutionMode - Convolution mode to use

      • cudnnAlgoMode

        public NeuralNetConfiguration.Builder cudnnAlgoMode​(ConvolutionLayer.AlgoMode cudnnAlgoMode)
        Sets the cuDNN algo mode for convolutional layers, which impacts performance and memory usage of cuDNN. See ConvolutionLayer.AlgoMode for details. Defaults to "PREFER_FASTEST", but "NO_WORKSPACE" uses less memory.
        Note: values set by this method will be applied to all applicable layers in the network, unless a different value is explicitly set on a given layer. In other words: values set via this method are used as the default value, and can be overridden on a per-layer basis.
        Parameters:
        cudnnAlgoMode - cuDNN algo mode to use

      • constrainAllParameters

        public NeuralNetConfiguration.Builder constrainAllParameters​(LayerConstraint... constraints)
        Set constraints to be applied to all layers. Default: no constraints.
        Constraints can be used to enforce certain conditions (non-negativity of parameters, max-norm regularization, etc). These constraints are applied at each iteration, after the parameters have been updated.
        Note: values set by this method will be applied to all applicable layers in the network, unless a different value is explicitly set on a given layer. In other words: values set via this method are used as the default value, and can be overridden on a per-layer basis.
        Parameters:
        constraints - Constraints to apply to all parameters of all layers

      • constrainBias

        public NeuralNetConfiguration.Builder constrainBias​(LayerConstraint... constraints)
        Set constraints to be applied to all layers. Default: no constraints.
        Constraints can be used to enforce certain conditions (non-negativity of parameters, max-norm regularization, etc). These constraints are applied at each iteration, after the parameters have been updated.
        Note: values set by this method will be applied to all applicable layers in the network, unless a different value is explicitly set on a given layer. In other words: values set via this method are used as the default value, and can be overridden on a per-layer basis.
        Parameters:
        constraints - Constraints to apply to all bias parameters of all layers

      • constrainWeights

        public NeuralNetConfiguration.Builder constrainWeights​(LayerConstraint... constraints)
        Set constraints to be applied to all layers. Default: no constraints.
        Constraints can be used to enforce certain conditions (non-negativity of parameters, max-norm regularization, etc). These constraints are applied at each iteration, after the parameters have been updated.
        Note: values set by this method will be applied to all applicable layers in the network, unless a different value is explicitly set on a given layer. In other words: values set via this method are used as the default value, and can be overridden on a per-layer basis.
        Parameters:
        constraints - Constraints to apply to all weight parameters of all layers