SMO (weka-stable 3.8.4 API)

java.lang.Object
- weka.classifiers.AbstractClassifier
- - weka.classifiers.functions.SMO

All Implemented Interfaces:: Serializable, Cloneable, Classifier, BatchPredictor, CapabilitiesHandler, CapabilitiesIgnorer, CommandlineRunnable, OptionHandler, RevisionHandler, TechnicalInformationHandler, WeightedInstancesHandler

public class SMO
extends AbstractClassifier
implements WeightedInstancesHandler, TechnicalInformationHandler

Implements John Platt's sequential minimal optimization algorithm for training a support vector classifier.

This implementation globally replaces all missing values and transforms nominal attributes into binary ones. It also normalizes all attributes by default. (In that case the coefficients in the output are based on the normalized data, not the original data --- this is important for interpreting the classifier.)

Multi-class problems are solved using pairwise classification (aka 1-vs-1).

To obtain proper probability estimates, use the option that fits calibration models to the outputs of the support vector machine. In the multi-class case, the predicted probabilities are coupled using Hastie and Tibshirani's pairwise coupling method.

Note: for improved speed normalization should be turned off when operating on SparseInstances.

For more information on the SMO algorithm, see

J. Platt: Fast Training of Support Vector Machines using Sequential Minimal Optimization. In B. Schoelkopf and C. Burges and A. Smola, editors, Advances in Kernel Methods - Support Vector Learning, 1998.

S.S. Keerthi, S.K. Shevade, C. Bhattacharyya, K.R.K. Murthy (2001). Improvements to Platt's SMO Algorithm for SVM Classifier Design. Neural Computation. 13(3):637-649.

Trevor Hastie, Robert Tibshirani: Classification by Pairwise Coupling. In: Advances in Neural Information Processing Systems, 1998.

BibTeX:

 @incollection{Platt1998,
    author = {J. Platt},
    booktitle = {Advances in Kernel Methods - Support Vector Learning},
    editor = {B. Schoelkopf and C. Burges and A. Smola},
    publisher = {MIT Press},
    title = {Fast Training of Support Vector Machines using Sequential Minimal Optimization},
    year = {1998},
    URL = {http://research.microsoft.com/\~jplatt/smo.html},
    PS = {http://research.microsoft.com/\~jplatt/smo-book.ps.gz},
    PDF = {http://research.microsoft.com/\~jplatt/smo-book.pdf}
 }
 
 @article{Keerthi2001,
    author = {S.S. Keerthi and S.K. Shevade and C. Bhattacharyya and K.R.K. Murthy},
    journal = {Neural Computation},
    number = {3},
    pages = {637-649},
    title = {Improvements to Platt's SMO Algorithm for SVM Classifier Design},
    volume = {13},
    year = {2001},
    PS = {http://guppy.mpe.nus.edu.sg/\~mpessk/svm/smo_mod_nc.ps.gz}
 }
 
 @inproceedings{Hastie1998,
    author = {Trevor Hastie and Robert Tibshirani},
    booktitle = {Advances in Neural Information Processing Systems},
    editor = {Michael I. Jordan and Michael J. Kearns and Sara A. Solla},
    publisher = {MIT Press},
    title = {Classification by Pairwise Coupling},
    volume = {10},
    year = {1998},
    PS = {http://www-stat.stanford.edu/\~hastie/Papers/2class.ps}
 }

Valid options are:

 -no-checks
  Turns off all checks - use with caution!
  Turning them off assumes that data is purely numeric, doesn't
  contain any missing values, and has a nominal class. Turning them
  off also means that no header information will be stored if the
  machine is linear. Finally, it also assumes that no instance has
  a weight equal to 0.
  (default: checks on)

 -C <double>
  The complexity constant C. (default 1)

 -N
  Whether to 0=normalize/1=standardize/2=neither. (default 0=normalize)

 -L <double>
  The tolerance parameter. (default 1.0e-3)

 -P <double>
  The epsilon for round-off error. (default 1.0e-12)

 -M
  Fit calibration models to SVM outputs.

 -V <double>
  The number of folds for the internal
  cross-validation. (default -1, use training data)

 -W <double>
  The random number seed. (default 1)

 -K <classname and parameters>
  The Kernel to use.
  (default: weka.classifiers.functions.supportVector.PolyKernel)

 -calibrator <scheme specification>
  Full name of calibration model, followed by options.
  (default: "weka.classifiers.functions.Logistic")

 -output-debug-info
  If set, classifier is run in debug mode and
  may output additional info to the console

 -do-not-check-capabilities
  If set, classifier capabilities are not checked before classifier is built
  (use with caution).

 -num-decimal-places
  The number of decimal places for the output of numbers in the model (default 2).

 
 Options specific to kernel weka.classifiers.functions.supportVector.PolyKernel:

 -E <num>
  The Exponent to use.
  (default: 1.0)

 -L
  Use lower-order terms.
  (default: no)

 -C <num>
  The size of the cache (a prime number), 0 for full cache and 
  -1 to turn it off.
  (default: 250007)

 -output-debug-info
  Enables debugging output (if available) to be printed.
  (default: off)

 -no-checks
  Turns off all checks - use with caution!
  (default: checks on)

 
 Options specific to calibrator weka.classifiers.functions.Logistic:

 -C
  Use conjugate gradient descent rather than BFGS updates.

 -R <ridge>
  Set the ridge in the log-likelihood.

 -M <number>
  Set the maximum number of iterations (default -1, until convergence).

 -output-debug-info
  If set, classifier is run in debug mode and
  may output additional info to the console

 -do-not-check-capabilities
  If set, classifier capabilities are not checked before classifier is built
  (use with caution).

 -num-decimal-places
  The number of decimal places for the output of numbers in the model (default 2).

Version:: $Revision: 14825 $
Author:: Eibe Frank ([email protected]), Shane Legg ([email protected]) (sparse vector code), Stuart Inglis ([email protected]) (sparse vector code)
See Also:: Serialized Form

Nested Class Summary

Nested Classes
Modifier and Type Class and Description

class SMO.BinarySMO
Class for building a binary support vector machine.

Nested Classes
Modifier and Type	Class and Description
`class`	`SMO.BinarySMO` Class for building a binary support vector machine.

Field Summary

Fields
Modifier and Type	Field and Description
`static int`	`FILTER_NONE` filter: No normalization/standardization
`static int`	`FILTER_NORMALIZE` filter: Normalize training data
`static int`	`FILTER_STANDARDIZE` filter: Standardize training data
`static Tag[]`	`TAGS_FILTER` The filter to apply to the training data

Fields inherited from class weka.classifiers.AbstractClassifier
BATCH_SIZE_DEFAULT, NUM_DECIMAL_PLACES_DEFAULT

Constructor Summary

Constructors
Constructor and Description

SMO()

Constructors
Constructor and Description
`SMO()`

Method Summary

All Methods Static Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`String[][][]`	`attributeNames()` Returns the attribute names.
`double[][]`	`bias()` Returns the bias of each binary SMO.
`String`	`buildCalibrationModelsTipText()` Returns the tip text for this property
`void`	`buildClassifier(Instances insts)` Method for building the classifier.
`String`	`calibratorTipText()` Returns the tip text for this property
`String`	`checksTurnedOffTipText()` Returns the tip text for this property
`String[]`	`classAttributeNames()`
`String`	`cTipText()` Returns the tip text for this property
`double[]`	`distributionForInstance(Instance inst)` Estimates class probabilities for given instance.
`String`	`epsilonTipText()` Returns the tip text for this property
`String`	`filterTypeTipText()` Returns the tip text for this property
`boolean`	`getBuildCalibrationModels()` Get the value of buildCalibrationModels.
`double`	`getC()` Get the value of C.
`Classifier`	`getCalibrator()` Returns the calibrator to use
`Capabilities`	`getCapabilities()` Returns default capabilities of the classifier.
`boolean`	`getChecksTurnedOff()` Returns whether the checks are turned off or not.
`double`	`getEpsilon()` Get the value of epsilon.
`SelectedTag`	`getFilterType()` Gets how the training data will be transformed.
`Kernel`	`getKernel()` Returns the kernel to use
`int`	`getNumFolds()` Get the value of numFolds.
`String[]`	`getOptions()` Gets the current settings of the classifier.
`int`	`getRandomSeed()` Get the value of randomSeed.
`String`	`getRevision()` Returns the revision string.
`TechnicalInformation`	`getTechnicalInformation()` Returns an instance of a TechnicalInformation object, containing detailed information about the technical background of this class, e.g., paper reference or book this class is based on.
`double`	`getToleranceParameter()` Get the value of tolerance parameter.
`String`	`globalInfo()` Returns a string describing classifier
`String`	`kernelTipText()` Returns the tip text for this property
`Enumeration<Option>`	`listOptions()` Returns an enumeration describing the available options.
`static void`	`main(String[] argv)` Main method for testing this class.
`int`	`numClassAttributeValues()`
`String`	`numFoldsTipText()` Returns the tip text for this property
`int[]`	`obtainVotes(Instance inst)` Returns an array of votes for the given instance.
`String`	`randomSeedTipText()` Returns the tip text for this property
`void`	`setBuildCalibrationModels(boolean newbuildCalibrationModels)` Set the value of buildCalibrationModels.
`void`	`setC(double v)` Set the value of C.
`void`	`setCalibrator(Classifier value)` sets the calibrator to use
`void`	`setChecksTurnedOff(boolean value)` Disables or enables the checks (which could be time-consuming).
`void`	`setEpsilon(double v)` Set the value of epsilon.
`void`	`setFilterType(SelectedTag newType)` Sets how the training data will be transformed.
`void`	`setKernel(Kernel value)` sets the kernel to use
`void`	`setNumFolds(int newnumFolds)` Set the value of numFolds.
`void`	`setOptions(String[] options)` Parses a given list of options.
`void`	`setRandomSeed(int newrandomSeed)` Set the value of randomSeed.
`void`	`setToleranceParameter(double v)` Set the value of tolerance parameter.
`int[][][]`	`sparseIndices()` Returns the indices in sparse format.
`double[][][]`	`sparseWeights()` Returns the weights in sparse format.
`String`	`toleranceParameterTipText()` Returns the tip text for this property
`String`	`toString()` Prints out the classifier.
`void`	`turnChecksOff()` Turns off checks for missing values, etc.
`void`	`turnChecksOn()` Turns on checks for missing values, etc.

Methods inherited from class java.lang.Object
equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface weka.core.OptionHandler
makeCopy

- Field Detail
  - FILTER_NORMALIZE
```
public static final int FILTER_NORMALIZE
```
    filter: Normalize training data
    
    See Also:
    
    Constant Field Values
  - FILTER_STANDARDIZE
```
public static final int FILTER_STANDARDIZE
```
    filter: Standardize training data
    
    See Also:
    
    Constant Field Values
  - FILTER_NONE
```
public static final int FILTER_NONE
```
    filter: No normalization/standardization
    
    See Also:
    
    Constant Field Values
  - TAGS_FILTER
```
public static final Tag[] TAGS_FILTER
```
    The filter to apply to the training data
- Constructor Detail
  - SMO
```
public SMO()
```
- Method Detail
  - globalInfo
```
public String globalInfo()
```
    Returns a string describing classifier
    
    Returns:
    
    a description suitable for displaying in the explorer/experimenter gui
  - getTechnicalInformation
```
public TechnicalInformation getTechnicalInformation()
```
    Returns an instance of a TechnicalInformation object, containing detailed information about the technical background of this class, e.g., paper reference or book this class is based on.
    
    Specified by:
    
    getTechnicalInformation in interface TechnicalInformationHandler
    
    Returns:
    
    the technical information about this class
  - turnChecksOff
```
public void turnChecksOff()
```
    Turns off checks for missing values, etc. Use with caution.
  - turnChecksOn
```
public void turnChecksOn()
```
    Turns on checks for missing values, etc.
  - getCapabilities
```
public Capabilities getCapabilities()
```
    Returns default capabilities of the classifier.
    
    Specified by:
    
    getCapabilities in interface Classifier
    
    Specified by:
    
    getCapabilities in interface CapabilitiesHandler
    
    Overrides:
    
    getCapabilities in class AbstractClassifier
    
    Returns:
    
    the capabilities of this classifier
    
    See Also:
    
    Capabilities
  - buildClassifier
```
public void buildClassifier(Instances insts)
                     throws Exception
```
    Method for building the classifier. Implements a one-against-one wrapper for multi-class problems.
    
    Specified by:
    
    buildClassifier in interface Classifier
    
    Parameters:
    
    insts - the set of training instances
    
    Throws:
    
    Exception - if the classifier can't be built successfully
  - distributionForInstance
```
public double[] distributionForInstance(Instance inst)
                                 throws Exception
```
    Estimates class probabilities for given instance.
    
    Specified by:
    
    distributionForInstance in interface Classifier
    
    Overrides:
    
    distributionForInstance in class AbstractClassifier
    
    Parameters:
    
    inst - the instance to compute the probabilities for
    
    Returns:
    
    an array containing the estimated membership probabilities of the test instance in each class or the numeric prediction
    
    Throws:
    
    Exception - in case of an error
  - obtainVotes
```
public int[] obtainVotes(Instance inst)
                  throws Exception
```
    Returns an array of votes for the given instance.
    
    Parameters:
    
    inst - the instance
    
    Returns:
    
    array of votex
    
    Throws:
    
    Exception - if something goes wrong
  - sparseWeights
```
public double[][][] sparseWeights()
```
    Returns the weights in sparse format.
  - sparseIndices
```
public int[][][] sparseIndices()
```
    Returns the indices in sparse format.
  - bias
```
public double[][] bias()
```
    Returns the bias of each binary SMO.
  - numClassAttributeValues
```
public int numClassAttributeValues()
```
  - classAttributeNames
```
public String[] classAttributeNames()
```
  - attributeNames
```
public String[][][] attributeNames()
```
    Returns the attribute names.
  - listOptions
```
public Enumeration<Option> listOptions()
```
    Returns an enumeration describing the available options.
    
    Specified by:
    
    listOptions in interface OptionHandler
    
    Overrides:
    
    listOptions in class AbstractClassifier
    
    Returns:
    
    an enumeration of all the available options.
  - setOptions
```
public void setOptions(String[] options)
                throws Exception
```
    Parses a given list of options.
    Valid options are:
```
 -no-checks
    Turns off all checks - use with caution!
    Turning them off assumes that data is purely numeric, doesn't
    contain any missing values, and has a nominal class. Turning them
    off also means that no header information will be stored if the
    machine is linear. Finally, it also assumes that no instance has
    a weight equal to 0.
    (default: checks on)
```
```
 -C <double>
    The complexity constant C. (default 1)
```
```
 -N
    Whether to 0=normalize/1=standardize/2=neither. (default 0=normalize)
```
```
 -L <double>
    The tolerance parameter. (default 1.0e-3)
```
```
 -P <double>
    The epsilon for round-off error. (default 1.0e-12)
```
```
 -M
    Fit calibration models to SVM outputs. 
```
```
 -V <double>
    The number of folds for the internal
    cross-validation. (default -1, use training data)
```
```
 -W <double>
    The random number seed. (default 1)
```
```
 -K <classname and parameters>
    The Kernel to use.
    (default: weka.classifiers.functions.supportVector.PolyKernel)
```
```
 -calibrator <scheme specification>
    Full name of calibration model, followed by options.
    (default: "weka.classifiers.functions.Logistic")
```
```
 -output-debug-info
    If set, classifier is run in debug mode and
    may output additional info to the console
```
```
 -do-not-check-capabilities
    If set, classifier capabilities are not checked before classifier is built
    (use with caution).
```
```
 -num-decimal-places
    The number of decimal places for the output of numbers in the model (default 2).
```
```
 
   Options specific to kernel weka.classifiers.functions.supportVector.PolyKernel:
   
```
```
 -E <num>
    The Exponent to use.
    (default: 1.0)
```
```
 -L
    Use lower-order terms.
    (default: no)
```
```
 -C <num>
    The size of the cache (a prime number), 0 for full cache and 
    -1 to turn it off.
    (default: 250007)
```
```
 -output-debug-info
    Enables debugging output (if available) to be printed.
    (default: off)
```
```
 -no-checks
    Turns off all checks - use with caution!
    (default: checks on)
```
```
 
   Options specific to calibrator weka.classifiers.functions.Logistic:
   
```
```
 -C
    Use conjugate gradient descent rather than BFGS updates.
```
```
 -R <ridge>
    Set the ridge in the log-likelihood.
```
```
 -M <number>
    Set the maximum number of iterations (default -1, until convergence).
```
```
 -output-debug-info
    If set, classifier is run in debug mode and
    may output additional info to the console
```
```
 -do-not-check-capabilities
    If set, classifier capabilities are not checked before classifier is built
    (use with caution).
```
```
 -num-decimal-places
    The number of decimal places for the output of numbers in the model (default 2).
```
    Specified by:
    
    setOptions in interface OptionHandler
    
    Overrides:
    
    setOptions in class AbstractClassifier
    
    Parameters:
    
    options - the list of options as an array of strings
    
    Throws:
    
    Exception - if an option is not supported
  - getOptions
```
public String[] getOptions()
```
    Gets the current settings of the classifier.
    
    Specified by:
    
    getOptions in interface OptionHandler
    
    Overrides:
    
    getOptions in class AbstractClassifier
    
    Returns:
    
    an array of strings suitable for passing to setOptions
  - setChecksTurnedOff
```
public void setChecksTurnedOff(boolean value)
```
    Disables or enables the checks (which could be time-consuming). Use with caution!
    
    Parameters:
    
    value - if true turns off all checks
  - getChecksTurnedOff
```
public boolean getChecksTurnedOff()
```
    Returns whether the checks are turned off or not.
    
    Returns:
    
    true if the checks are turned off
  - checksTurnedOffTipText
```
public String checksTurnedOffTipText()
```
    Returns the tip text for this property
    
    Returns:
    
    tip text for this property suitable for displaying in the explorer/experimenter gui
  - kernelTipText
```
public String kernelTipText()
```
    Returns the tip text for this property
    
    Returns:
    
    tip text for this property suitable for displaying in the explorer/experimenter gui
  - setKernel
```
public void setKernel(Kernel value)
```
    sets the kernel to use
    
    Parameters:
    
    value - the kernel to use
  - getKernel
```
public Kernel getKernel()
```
    Returns the kernel to use
    
    Returns:
    
    the current kernel
  - calibratorTipText
```
public String calibratorTipText()
```
    Returns the tip text for this property
    
    Returns:
    
    tip text for this property suitable for displaying in the explorer/experimenter gui
  - setCalibrator
```
public void setCalibrator(Classifier value)
```
    sets the calibrator to use
    
    Parameters:
    
    value - the calibrator to use
  - getCalibrator
```
public Classifier getCalibrator()
```
    Returns the calibrator to use
    
    Returns:
    
    the current calibrator
  - cTipText
```
public String cTipText()
```
    Returns the tip text for this property
    
    Returns:
    
    tip text for this property suitable for displaying in the explorer/experimenter gui
  - getC
```
public double getC()
```
    Get the value of C.
    
    Returns:
    
    Value of C.
  - setC
```
public void setC(double v)
```
    Set the value of C.
    
    Parameters:
    
    v - Value to assign to C.
  - toleranceParameterTipText
```
public String toleranceParameterTipText()
```
    Returns the tip text for this property
    
    Returns:
    
    tip text for this property suitable for displaying in the explorer/experimenter gui
  - getToleranceParameter
```
public double getToleranceParameter()
```
    Get the value of tolerance parameter.
    
    Returns:
    
    Value of tolerance parameter.
  - setToleranceParameter
```
public void setToleranceParameter(double v)
```
    Set the value of tolerance parameter.
    
    Parameters:
    
    v - Value to assign to tolerance parameter.
  - epsilonTipText
```
public String epsilonTipText()
```
    Returns the tip text for this property
    
    Returns:
    
    tip text for this property suitable for displaying in the explorer/experimenter gui
  - getEpsilon
```
public double getEpsilon()
```
    Get the value of epsilon.
    
    Returns:
    
    Value of epsilon.
  - setEpsilon
```
public void setEpsilon(double v)
```
    Set the value of epsilon.
    
    Parameters:
    
    v - Value to assign to epsilon.
  - filterTypeTipText
```
public String filterTypeTipText()
```
    Returns the tip text for this property
    
    Returns:
    
    tip text for this property suitable for displaying in the explorer/experimenter gui
  - getFilterType
```
public SelectedTag getFilterType()
```
    Gets how the training data will be transformed. Will be one of FILTER_NORMALIZE, FILTER_STANDARDIZE, FILTER_NONE.
    
    Returns:
    
    the filtering mode
  - setFilterType
```
public void setFilterType(SelectedTag newType)
```
    Sets how the training data will be transformed. Should be one of FILTER_NORMALIZE, FILTER_STANDARDIZE, FILTER_NONE.
    
    Parameters:
    
    newType - the new filtering mode
  - buildCalibrationModelsTipText
```
public String buildCalibrationModelsTipText()
```
    Returns the tip text for this property
    
    Returns:
    
    tip text for this property suitable for displaying in the explorer/experimenter gui
  - getBuildCalibrationModels
```
public boolean getBuildCalibrationModels()
```
    Get the value of buildCalibrationModels.
    
    Returns:
    
    Value of buildCalibrationModels.
  - setBuildCalibrationModels
```
public void setBuildCalibrationModels(boolean newbuildCalibrationModels)
```
    Set the value of buildCalibrationModels.
    
    Parameters:
    
    newbuildCalibrationModels - Value to assign to buildCalibrationModels.
  - numFoldsTipText
```
public String numFoldsTipText()
```
    Returns the tip text for this property
    
    Returns:
    
    tip text for this property suitable for displaying in the explorer/experimenter gui
  - getNumFolds
```
public int getNumFolds()
```
    Get the value of numFolds.
    
    Returns:
    
    Value of numFolds.
  - setNumFolds
```
public void setNumFolds(int newnumFolds)
```
    Set the value of numFolds.
    
    Parameters:
    
    newnumFolds - Value to assign to numFolds.
  - randomSeedTipText
```
public String randomSeedTipText()
```
    Returns the tip text for this property
    
    Returns:
    
    tip text for this property suitable for displaying in the explorer/experimenter gui
  - getRandomSeed
```
public int getRandomSeed()
```
    Get the value of randomSeed.
    
    Returns:
    
    Value of randomSeed.
  - setRandomSeed
```
public void setRandomSeed(int newrandomSeed)
```
    Set the value of randomSeed.
    
    Parameters:
    
    newrandomSeed - Value to assign to randomSeed.
  - toString
```
public String toString()
```
    Prints out the classifier.
    
    Overrides:
    
    toString in class Object
    
    Returns:
    
    a description of the classifier as a string
  - getRevision
```
public String getRevision()
```
    Returns the revision string.
    
    Specified by:
    
    getRevision in interface RevisionHandler
    
    Overrides:
    
    getRevision in class AbstractClassifier
    
    Returns:
    
    the revision
  - main
```
public static void main(String[] argv)
```
    Main method for testing this class.

Class SMO

Nested Class Summary

Field Summary

Fields inherited from class weka.classifiers.AbstractClassifier

Constructor Summary

Method Summary

Methods inherited from class weka.classifiers.AbstractClassifier

Methods inherited from class java.lang.Object

Methods inherited from interface weka.core.OptionHandler

Field Detail

FILTER_NORMALIZE

FILTER_STANDARDIZE

FILTER_NONE

TAGS_FILTER

Constructor Detail

SMO

Method Detail

globalInfo

getTechnicalInformation

turnChecksOff

turnChecksOn

getCapabilities

buildClassifier

distributionForInstance

obtainVotes

sparseWeights

sparseIndices

bias

numClassAttributeValues

classAttributeNames

attributeNames

listOptions

setOptions

getOptions

setChecksTurnedOff

getChecksTurnedOff

checksTurnedOffTipText

kernelTipText

setKernel

getKernel

calibratorTipText

setCalibrator

getCalibrator

cTipText

getC

setC

toleranceParameterTipText

getToleranceParameter

setToleranceParameter

epsilonTipText

getEpsilon

setEpsilon

filterTypeTipText

getFilterType

setFilterType

buildCalibrationModelsTipText

getBuildCalibrationModels

setBuildCalibrationModels

numFoldsTipText

getNumFolds

setNumFolds

randomSeedTipText

getRandomSeed

setRandomSeed

toString

getRevision

main