Class TargetSparsityThresholdAlgorithm
- java.lang.Object
-
- org.deeplearning4j.optimize.solvers.accumulation.encoding.threshold.TargetSparsityThresholdAlgorithm
-
- All Implemented Interfaces:
Serializable
,ThresholdAlgorithm
public class TargetSparsityThresholdAlgorithm extends Object implements ThresholdAlgorithm
- See Also:
- Serialized Form
-
-
Field Summary
Fields Modifier and Type Field Description static double
DEFAULT_DECAY_RATE
static double
DEFAULT_INITIAL_THRESHOLD
static double
DEFAULT_SPARSITY_TARGET
-
Constructor Summary
Constructors Constructor Description TargetSparsityThresholdAlgorithm()
Create the adaptive threshold algorithm with the default initial thresholdDEFAULT_INITIAL_THRESHOLD
, default sparsity targetDEFAULT_SPARSITY_TARGET
and default decay rateDEFAULT_DECAY_RATE
TargetSparsityThresholdAlgorithm(double initialThreshold, double sparsityTarget, double decayRate)
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description double
calculateThreshold(int iteration, int epoch, Double lastThreshold, Boolean lastWasDense, Double lastSparsityRatio, INDArray updatesPlusResidual)
TargetSparsityThresholdAlgorithm
clone()
ThresholdAlgorithmReducer
newReducer()
Create a new ThresholdAlgorithmReducer.String
toString()
-
-
-
Field Detail
-
DEFAULT_INITIAL_THRESHOLD
public static final double DEFAULT_INITIAL_THRESHOLD
- See Also:
- Constant Field Values
-
DEFAULT_SPARSITY_TARGET
public static final double DEFAULT_SPARSITY_TARGET
- See Also:
- Constant Field Values
-
DEFAULT_DECAY_RATE
public static final double DEFAULT_DECAY_RATE
-
-
Constructor Detail
-
TargetSparsityThresholdAlgorithm
public TargetSparsityThresholdAlgorithm()
Create the adaptive threshold algorithm with the default initial thresholdDEFAULT_INITIAL_THRESHOLD
, default sparsity targetDEFAULT_SPARSITY_TARGET
and default decay rateDEFAULT_DECAY_RATE
-
TargetSparsityThresholdAlgorithm
public TargetSparsityThresholdAlgorithm(double initialThreshold, double sparsityTarget, double decayRate)
- Parameters:
initialThreshold
- The initial threshold to usesparsityTarget
- The sparsity targetdecayRate
- The decay rate. For example 0.95
-
-
Method Detail
-
calculateThreshold
public double calculateThreshold(int iteration, int epoch, Double lastThreshold, Boolean lastWasDense, Double lastSparsityRatio, INDArray updatesPlusResidual)
- Specified by:
calculateThreshold
in interfaceThresholdAlgorithm
- Parameters:
iteration
- Current neural network training iterationepoch
- Current neural network training epochlastThreshold
- The encoding threshold used in the last iteration - if available. May be null for first iteration in an epoch (where no 'last iteration' value is available)lastWasDense
- Whether the last encoding was dense (true) or sparse (false). May be null for the first iteration in an epoch (where no 'last iteration' value is available)lastSparsityRatio
- The sparsity ratio of the last iteration. Sparsity ratio is defined as numElements(encoded)/length(updates). A sparsity ratio of 1.0 would mean all entries present in encoded representation; a sparsity ratio of 0.0 would mean the encoded vector did not contain any values. Note: when the last encoding was dense, lastSparsityRatio is always null - this means that the sparsity ratio is larger than 1/16 = 0.0625updatesPlusResidual
- The actual array (updates plus residual) that will be encoded using the threshold calculated/returned by this method- Returns:
-
newReducer
public ThresholdAlgorithmReducer newReducer()
Description copied from interface:ThresholdAlgorithm
Create a new ThresholdAlgorithmReducer. Note that implementations should NOT add the curret ThresholdAlgorithm to it.- Specified by:
newReducer
in interfaceThresholdAlgorithm
- Returns:
- ThresholdAlgorithmReducer
-
clone
public TargetSparsityThresholdAlgorithm clone()
- Specified by:
clone
in interfaceThresholdAlgorithm
- Overrides:
clone
in classObject
- Returns:
- A clone of the current threshold algorithm
-
-