org.nd4j.linalg.dataset.api.preprocessor.classimbalance

Class UnderSamplingByMaskingPreProcessor

java.lang.Object

org.nd4j.linalg.dataset.api.preprocessor.classimbalance.BaseUnderSamplingPreProcessor

org.nd4j.linalg.dataset.api.preprocessor.classimbalance.UnderSamplingByMaskingPreProcessor

All Implemented Interfaces:

Serializable, DataSetPreProcessor

public class UnderSamplingByMaskingPreProcessor
extends BaseUnderSamplingPreProcessor
implements DataSetPreProcessor

For use in time series with unbalanced binary classes trained with truncated back prop through time Undersamples the majority class by randomly masking time steps belonging to it Given a target distribution for the minority class and the window size (usually the value used with tbptt) the preprocessor will approximate the given target distribution for every window of given size for every sample of the minibatch By default '0' is considered the majority class and '1' the minorityLabel class Default can be overriden with .overrideMinorityDefault()

ONLY masks belonging to the majority class are modified If a tbptt segment contains only majority class labels all time steps in that segment are masked. Can be overriden with donotMaskMinorityWindows() in which case 1 - target distribution % of time steps are masked

Author:

susaneraly

See Also:

Serialized Form

Field Summary

Fields inherited from class org.nd4j.linalg.dataset.api.preprocessor.classimbalance.BaseUnderSamplingPreProcessor

tbpttWindowSize

Constructor Summary

Constructors

Constructor and Description
UnderSamplingByMaskingPreProcessor(double targetDist, int windowSize) The target distribution to approximate.

Method Summary

Modifier and Type	Method and Description
void	overrideMinorityDefault() Will change the default minority label from "1" to "0" and correspondingly the majority class from "0" to "1"
void	preProcess(DataSet toPreProcess) Pre process a dataset

Methods inherited from class org.nd4j.linalg.dataset.api.preprocessor.classimbalance.BaseUnderSamplingPreProcessor

adjustMasks, donotMaskAllMajorityWindows, donotMaskMinorityWindows

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

UnderSamplingByMaskingPreProcessor

public UnderSamplingByMaskingPreProcessor(double targetDist,
                                          int windowSize)

The target distribution to approximate. Valid values are between (0,0.5]. Eg. For a targetDist = 0.25 and tbpttWindowSize = 100: Every 100 time steps (starting from the last time step) will randomly mask majority time steps to approximate a 25:75 ratio of minorityLabel to majority classes

Parameters:

targetDist -

windowSize - Usually set to the size of the tbptt

Method Detail

overrideMinorityDefault

public void overrideMinorityDefault()

Will change the default minority label from "1" to "0" and correspondingly the majority class from "0" to "1"

preProcess

public void preProcess(DataSet toPreProcess)

Description copied from interface: DataSetPreProcessor

Pre process a dataset

Specified by:

preProcess in interface DataSetPreProcessor

Parameters:

toPreProcess - the data set to pre process