org.datavec.api.io.filters

Class BalancedPathFilter

java.lang.Object

org.datavec.api.io.filters.RandomPathFilter

org.datavec.api.io.filters.BalancedPathFilter

All Implemented Interfaces:

PathFilter

public class BalancedPathFilter
extends RandomPathFilter

Randomizes the order of paths in an array and removes paths randomly to have the same number of paths for each label. Further interlaces the paths on output based on their labels, to obtain easily optimal batches for training.

Author:

saudet

Field Summary

Fields

Modifier and Type	Field and Description
protected PathLabelGenerator	labelGenerator
protected String[]	labels
protected long	maxLabels
protected long	maxPathsPerLabel
protected long	minPathsPerLabel

Fields inherited from class org.datavec.api.io.filters.RandomPathFilter

extensions, maxPaths, random

Constructor Summary

Constructors

Constructor and Description
BalancedPathFilter(Random random, PathLabelGenerator labelGenerator, long maxPathsPerLabel) Calls this(random, null, labelGenerator, 0, 0, 0, maxPathsPerLabel).
BalancedPathFilter(Random random, PathLabelGenerator labelGenerator, long maxPaths, long maxLabels, long maxPathsPerLabel) Calls this(random, extensions, labelGenerator, 0, maxLabels, 0, maxPathsPerLabel).
BalancedPathFilter(Random random, String[] extensions, PathLabelGenerator labelGenerator) Calls this(random, extensions, labelGenerator, 0, 0, 0, 0).
BalancedPathFilter(Random random, String[] extensions, PathLabelGenerator labelGenerator, long maxPathsPerLabel) Calls this(random, extensions, labelGenerator, 0, 0, 0, maxPathsPerLabel).
BalancedPathFilter(Random random, String[] extensions, PathLabelGenerator labelGenerator, long maxLabels, long maxPathsPerLabel) Calls this(random, extensions, labelGenerator, 0, maxLabels, 0, maxPathsPerLabel).
BalancedPathFilter(Random random, String[] extensions, PathLabelGenerator labelGenerator, long maxPaths, long maxLabels, long minPathsPerLabel, long maxPathsPerLabel, String... labels) Constructs an instance of the PathFilter.

Method Summary

Modifier and Type	Method and Description
protected boolean	acceptLabel(String name)
URI[]	filter(URI[] paths)

Methods inherited from class org.datavec.api.io.filters.RandomPathFilter

accept

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

labelGenerator

protected PathLabelGenerator labelGenerator

maxLabels

protected long maxLabels

minPathsPerLabel

protected long minPathsPerLabel

maxPathsPerLabel

protected long maxPathsPerLabel

labels

protected String[] labels

Constructor Detail

BalancedPathFilter

public BalancedPathFilter(Random random,
                          String[] extensions,
                          PathLabelGenerator labelGenerator)

Calls this(random, extensions, labelGenerator, 0, 0, 0, 0).

BalancedPathFilter

public BalancedPathFilter(Random random,
                          PathLabelGenerator labelGenerator,
                          long maxPathsPerLabel)

Calls this(random, null, labelGenerator, 0, 0, 0, maxPathsPerLabel).

BalancedPathFilter

public BalancedPathFilter(Random random,
                          String[] extensions,
                          PathLabelGenerator labelGenerator,
                          long maxPathsPerLabel)

Calls this(random, extensions, labelGenerator, 0, 0, 0, maxPathsPerLabel).

BalancedPathFilter

public BalancedPathFilter(Random random,
                          PathLabelGenerator labelGenerator,
                          long maxPaths,
                          long maxLabels,
                          long maxPathsPerLabel)

Calls this(random, extensions, labelGenerator, 0, maxLabels, 0, maxPathsPerLabel).

BalancedPathFilter

public BalancedPathFilter(Random random,
                          String[] extensions,
                          PathLabelGenerator labelGenerator,
                          long maxLabels,
                          long maxPathsPerLabel)

Calls this(random, extensions, labelGenerator, 0, maxLabels, 0, maxPathsPerLabel).

BalancedPathFilter

public BalancedPathFilter(Random random,
                          String[] extensions,
                          PathLabelGenerator labelGenerator,
                          long maxPaths,
                          long maxLabels,
                          long minPathsPerLabel,
                          long maxPathsPerLabel,
                          String... labels)

Constructs an instance of the PathFilter. If minPathsPerLabel > 0, it might return an unbalanced set if the value is larger than the number of examples available for the label with the minimum amount.

Parameters:

random - object to use

extensions - of files to keep

labelGenerator - to obtain labels from paths

maxPaths - max number of paths to return (0 == unlimited)

maxLabels - max number of labels to return (0 == unlimited)

minPathsPerLabel - min number of paths per labels to return

maxPathsPerLabel - max number of paths per labels to return (0 == unlimited)

labels - of the paths to keep (empty set == keep all paths)

Method Detail

acceptLabel

protected boolean acceptLabel(String name)

filter

public URI[] filter(URI[] paths)

Specified by:

filter in interface PathFilter

Overrides:

filter in class RandomPathFilter