Package org.deeplearning4j.nn.transferlearning

Class FineTuneConfiguration.Builder

    • Field Detail

      • removeL2

        protected boolean removeL2

      • removeL2Bias

        protected boolean removeL2Bias

      • removeL1

        protected boolean removeL1

      • removeL1Bias

        protected boolean removeL1Bias

      • removeWD

        protected boolean removeWD

      • removeWDBias

        protected boolean removeWDBias

    • Constructor Detail

      • Builder

        public Builder()

    • Method Detail

      • weightInit

        public FineTuneConfiguration.Builder weightInit​(Distribution distribution)
        Set weight initialization scheme to random sampling via the specified distribution. Equivalent to: .weightInit(new WeightInitDistribution(distribution))
        Parameters:
        distribution - Distribution to use for weight initialization

      • biasInit

        public FineTuneConfiguration.Builder biasInit​(double biasInit)
        Constant for bias initialization. Default: 0.0
        Parameters:
        biasInit - Constant for bias initialization

      • weightDecay

        public FineTuneConfiguration.Builder weightDecay​(double coefficient)
        Add weight decay regularization for the network parameters (excluding biases).
        This applies weight decay with multiplying the learning rate - see WeightDecay for more details.
        Parameters:
        coefficient - Weight decay regularization coefficient
        See Also:
        weightDecay(double, boolean)

      • weightDecay

        public FineTuneConfiguration.Builder weightDecay​(double coefficient,
                                                 boolean applyLR)
        Add weight decay regularization for the network parameters (excluding biases). See WeightDecay for more details.
        Parameters:
        coefficient - Weight decay regularization coefficient
        applyLR - Whether the learning rate should be multiplied in when performing weight decay updates. See WeightDecay for more details.
        See Also:
        weightDecay(double, boolean)

      • weightDecayBias

        public FineTuneConfiguration.Builder weightDecayBias​(double coefficient,
                                                     boolean applyLR)
        Weight decay for the biases only - see weightDecay(double) for more details
        Parameters:
        coefficient - Weight decay regularization coefficient

      • dropOut

        public FineTuneConfiguration.Builder dropOut​(double inputRetainProbability)
        Dropout probability. This is the probability of retaining each input activation value for a layer. dropOut(x) will keep an input activation with probability x, and set to 0 with probability 1-x.
        dropOut(0.0) is a special value / special case - when set to 0.0., dropout is disabled (not applied). Note that a dropout value of 1.0 is functionally equivalent to no dropout: i.e., 100% probability of retaining each input activation.

        Note 1: Dropout is applied at training time only - and is automatically not applied at test time (for evaluation, etc)
        Note 2: This sets the probability per-layer. Care should be taken when setting lower values for complex networks (too much information may be lost with aggressive (very low) dropout values).
        Note 3: Frequently, dropout is not applied to (or, has higher retain probability for) input (first layer) layers. Dropout is also often not applied to output layers. This needs to be handled MANUALLY by the user - set .dropout(0) on those layers when using global dropout setting.
        Note 4: Implementation detail (most users can ignore): DL4J uses inverted dropout, as described here: http://cs231n.github.io/neural-networks-2/

        Parameters:
        inputRetainProbability - Dropout probability (probability of retaining each input activation value for a layer)
        See Also:
        dropout(IDropout)

      • biasUpdater

        public FineTuneConfiguration.Builder biasUpdater​(IUpdater biasUpdater)
        Gradient updater configuration, for the biases only. If not set, biases will use the updater as set by updater(IUpdater)
        Parameters:
        biasUpdater - Updater to use for bias parameters

      • miniBatch

        public FineTuneConfiguration.Builder miniBatch​(boolean miniBatch)
        Whether scores and gradients should be divided by the minibatch size.
        Most users should leave this ast he default value of true.

      • gradientNormalizationThreshold

        public FineTuneConfiguration.Builder gradientNormalizationThreshold​(double gradientNormalizationThreshold)
        Threshold for gradient normalization, only used for GradientNormalization.ClipL2PerLayer, GradientNormalization.ClipL2PerParamType, and GradientNormalization.ClipElementWiseAbsoluteValue
        Not used otherwise.
        L2 threshold for first two types of clipping, or absolute value threshold for last type of clipping

      • convolutionMode

        public FineTuneConfiguration.Builder convolutionMode​(ConvolutionMode convolutionMode)
        Sets the convolution mode for convolutional layers, which impacts padding and output sizes. See ConvolutionMode for details. Defaults to ConvolutionMode.TRUNCATE
        Parameters:
        convolutionMode - Convolution mode to use

      • constraints

        public FineTuneConfiguration.Builder constraints​(List<LayerConstraint> constraints)
        Set constraints to be applied to all layers. Default: no constraints.
        Constraints can be used to enforce certain conditions (non-negativity of parameters, max-norm regularization, etc). These constraints are applied at each iteration, after the parameters have been updated.
        Parameters:
        constraints - Constraints to apply to all parameters of all layers

      • backpropType

        public FineTuneConfiguration.Builder backpropType​(BackpropType backpropType)
        The type of backprop. Default setting is used for most networks (MLP, CNN etc), but optionally truncated BPTT can be used for training recurrent neural networks. If using TruncatedBPTT make sure you set both tBPTTForwardLength() and tBPTTBackwardLength()
        Parameters:
        backpropType - Type of backprop. Default: BackpropType.Standard

      • tbpttFwdLength

        public FineTuneConfiguration.Builder tbpttFwdLength​(int tbpttFwdLength)
        When doing truncated BPTT: how many steps of forward pass should we do before doing (truncated) backprop?
        Only applicable when doing backpropType(BackpropType.TruncatedBPTT)
        Typically tBPTTForwardLength parameter is same as the tBPTTBackwardLength parameter, but may be larger than it in some circumstances (but never smaller)
        Ideally your training data time series length should be divisible by this This is the k1 parameter on pg23 of http://www.cs.utoronto.ca/~ilya/pubs/ilya_sutskever_phd_thesis.pdf
        Parameters:
        tbpttFwdLength - Forward length > 0, >= backwardLength

      • trainingWorkspaceMode

        public FineTuneConfiguration.Builder trainingWorkspaceMode​(WorkspaceMode trainingWorkspaceMode)
        This method defines Workspace mode being used during training: NONE: workspace won't be used ENABLED: workspaces will be used for training (reduced memory and better performance)
        Parameters:
        trainingWorkspaceMode - Workspace mode for training
        Returns:
        Builder

      • inferenceWorkspaceMode

        public FineTuneConfiguration.Builder inferenceWorkspaceMode​(WorkspaceMode inferenceWorkspaceMode)
        This method defines Workspace mode being used during inference:
        NONE: workspace won't be used
        ENABLED: workspaces will be used for inference (reduced memory and better performance)
        Parameters:
        inferenceWorkspaceMode - Workspace mode for inference
        Returns:
        Builder