Package moa.tasks

Class EvaluatePrequentialDelayedCV

java.lang.Object

moa.AbstractMOAObject

moa.options.AbstractOptionHandler

moa.tasks.AbstractTask

moa.tasks.MainTask

moa.tasks.ClassificationMainTask

moa.tasks.EvaluatePrequentialDelayedCV

All Implemented Interfaces:

Configurable, java.io.Serializable, CapabilitiesHandler, MOAObject, OptionHandler, Task

public class EvaluatePrequentialDelayedCV
extends ClassificationMainTask

Task for delayed cross-validation evaluation of a classifier on a stream by testing and only training with the example after the arrival of other k examples (delayed labeling).

See details in:
Heitor Murilo Gomes, Albert Bifet, Jesse Read, Jean Paul Barddal, Fabricio Enembreck, Bernhard Pfharinger, Geoff Holmes, Talel Abdessalem. Adaptive random forests for evolving data stream classification. In Machine Learning, DOI: 10.1007/s10994-017-5642-8, Springer, 2017.

Cross-validation for data streams was originally proposed in:
Albert Bifet, Gianmarco De Francisci Morales, Jesse Read, Geoff Holmes, Bernhard Pfahringer: Efficient Online Evaluation of Big Data Stream Classifiers. KDD 2015: 59-68

Version:

$Revision: 7 $

Author:

Richard Kirkby ([email protected]), Albert Bifet (abifet at cs dot waikato dot ac dot nz), Heitor Murilo Gomes (heitor dot gomes at telecom-paristech dot fr)

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field	Description
IntOption	delayLengthOption
FileOption	dumpFileOption
ClassOption	evaluatorOption
IntOption	instanceLimitOption
ClassOption	learnerOption
IntOption	memCheckFrequencyOption
IntOption	numFoldsOption
IntOption	randomSeedOption
IntOption	sampleFrequencyOption
ClassOption	streamOption
IntOption	timeLimitOption
protected java.util.LinkedList<java.util.LinkedList<Example>>	trainInstances
MultiChoiceOption	validationMethodologyOption

Fields inherited from class moa.options.AbstractOptionHandler

config

Fields inherited from class moa.tasks.ClassificationMainTask

events

Fields inherited from class moa.tasks.MainTask

INSTANCES_BETWEEN_MONITOR_UPDATES, outputFileOption

Constructor Summary

Constructors

Constructor	Description
EvaluatePrequentialDelayedCV()

Method Summary

Modifier and Type	Method	Description
protected java.lang.Object	doMainTask(TaskMonitor monitor, ObjectRepository repository)	This method performs this task.
Measurement[]	getEvaluationMeasurements(Measurement[] modelMeasurements, LearningPerformanceEvaluator[] subEvaluators)
java.lang.String	getPurposeString()	Dictionary with option texts and objects
java.lang.Class<?>	getTaskResultType()	Gets the result type of this task.

Methods inherited from class moa.AbstractMOAObject

copy, measureByteSize, measureByteSize, toString

Methods inherited from class moa.options.AbstractOptionHandler

copy, getCLICreationString, getOptions, getPreparedClassOption, prepareClassOptions, prepareForUse, prepareForUse

Methods inherited from class moa.tasks.AbstractTask

doTask, doTask, getDescription, getTaskName, prepareForUseImpl

Methods inherited from interface moa.capabilities.CapabilitiesHandler

getCapabilities

Methods inherited from class moa.tasks.ClassificationMainTask

defineImmutableCapabilities, getEventsList, setEventsList

Methods inherited from class moa.tasks.MainTask

doTaskImpl

Methods inherited from interface moa.MOAObject

copy, measureByteSize

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

learnerOption

public ClassOption learnerOption

streamOption

public ClassOption streamOption

evaluatorOption

public ClassOption evaluatorOption

delayLengthOption

public IntOption delayLengthOption

instanceLimitOption

public IntOption instanceLimitOption

timeLimitOption

public IntOption timeLimitOption

sampleFrequencyOption

public IntOption sampleFrequencyOption

memCheckFrequencyOption

public IntOption memCheckFrequencyOption

dumpFileOption

public FileOption dumpFileOption

numFoldsOption

public IntOption numFoldsOption

validationMethodologyOption

public MultiChoiceOption validationMethodologyOption

randomSeedOption

public IntOption randomSeedOption

trainInstances

protected java.util.LinkedList<java.util.LinkedList<Example>> trainInstances

Constructor Detail

EvaluatePrequentialDelayedCV

public EvaluatePrequentialDelayedCV()

Method Detail

getPurposeString

public java.lang.String getPurposeString()

Description copied from class: AbstractOptionHandler

Dictionary with option texts and objects

Specified by:

getPurposeString in interface OptionHandler

Overrides:

getPurposeString in class AbstractOptionHandler

Returns:

the string with the purpose of this object

getTaskResultType

public java.lang.Class<?> getTaskResultType()

Description copied from interface: Task

Gets the result type of this task. Tasks can return LearningCurve, LearningEvaluation, Classifier, String, Instances..

Returns:

a class object of the result of this task

doMainTask

protected java.lang.Object doMainTask(TaskMonitor monitor,
                                      ObjectRepository repository)

Description copied from class: MainTask

This method performs this task. AbstractTask implements doTask, that uses doTaskImpl. MainTask implements doTaskImpl using doMainTask so its extensions only need to implement doMainTask.

Specified by:

doMainTask in class MainTask

Parameters:

monitor - the TaskMonitor to use

repository - the ObjectRepository to use

Returns:

an object with the result of this task

getEvaluationMeasurements

public Measurement[] getEvaluationMeasurements(Measurement[] modelMeasurements,
                                               LearningPerformanceEvaluator[] subEvaluators)