smile.classification

Class LogisticRegression

java.lang.Object

smile.classification.LogisticRegression

All Implemented Interfaces:

java.io.Serializable, java.util.function.ToDoubleFunction<double[]>, java.util.function.ToIntFunction<double[]>, Classifier<double[]>, OnlineClassifier<double[]>, SoftClassifier<double[]>

Direct Known Subclasses:

LogisticRegression.Binomial, LogisticRegression.Multinomial

public abstract class LogisticRegression
extends java.lang.Object
implements SoftClassifier<double[]>, OnlineClassifier<double[]>

Logistic regression. Logistic regression (logit model) is a generalized linear model used for binomial regression. Logistic regression applies maximum likelihood estimation after transforming the dependent into a logit variable. A logit is the natural log of the odds of the dependent equaling a certain value or not (usually 1 in binary logistic models, the highest value in multinomial models). In this way, logistic regression estimates the odds of a certain event (value) occurring.

Goodness-of-fit tests such as the likelihood ratio test are available as indicators of model appropriateness, as is the Wald statistic to test the significance of individual independent variables.

Logistic regression has many analogies to ordinary least squares (OLS) regression. Unlike OLS regression, however, logistic regression does not assume linearity of relationship between the raw values of the independent variables and the dependent, does not require normally distributed variables, does not assume homoscedasticity, and in general has less stringent requirements.

Compared with linear discriminant analysis, logistic regression has several advantages:

• It is more robust: the independent variables don't have to be normally distributed, or have equal variance in each group
• It does not assume a linear relationship between the independent variables and dependent variable.
• It may handle nonlinear effects since one can add explicit interaction and power terms.

However, it requires much more data to achieve stable, meaningful results.

Logistic regression also has strong connections with neural network and maximum entropy modeling. For example, binary logistic regression is equivalent to a one-layer, single-output neural network with a logistic activation function trained under log loss. Similarly, multinomial logistic regression is equivalent to a one-layer, softmax-output neural network.

Logistic regression estimation also obeys the maximum entropy principle, and thus logistic regression is sometimes called "maximum entropy modeling", and the resulting classifier the "maximum entropy classifier".

See Also:

GLM, MLP, Maxent, LDA, Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	LogisticRegression.Binomial Binomial logistic regression.
static class	LogisticRegression.Multinomial Multinomial logistic regression.

Constructor Summary

Constructors

Constructor and Description
LogisticRegression(int p, double L, double lambda, smile.util.IntSet labels) Constructor.

Method Summary

Modifier and Type	Method and Description
double	AIC() Returns the AIC score.
static LogisticRegression.Binomial	binomial(double[][] x, int[] y) Fits binomial logistic regression.
static LogisticRegression.Binomial	binomial(double[][] x, int[] y, double lambda, double tol, int maxIter) Fits binomial logistic regression.
static LogisticRegression.Binomial	binomial(double[][] x, int[] y, java.util.Properties prop) Fits binomial logistic regression.
static LogisticRegression.Binomial	binomial(smile.data.formula.Formula formula, smile.data.DataFrame data) Fits binomial logistic regression.
static LogisticRegression.Binomial	binomial(smile.data.formula.Formula formula, smile.data.DataFrame data, java.util.Properties prop) Fits binomial logistic regression.
static LogisticRegression	fit(double[][] x, int[] y) Fits logistic regression.
static LogisticRegression	fit(double[][] x, int[] y, double lambda, double tol, int maxIter) Fits logistic regression.
static LogisticRegression	fit(double[][] x, int[] y, java.util.Properties prop) Fits logistic regression.
static LogisticRegression	fit(smile.data.formula.Formula formula, smile.data.DataFrame data) Fits logistic regression.
static LogisticRegression	fit(smile.data.formula.Formula formula, smile.data.DataFrame data, java.util.Properties prop) Fits logistic regression.
double	getLearningRate() Returns the learning rate of stochastic gradient descent.
double	loglikelihood() Returns the log-likelihood of model.
static LogisticRegression.Multinomial	multinomial(double[][] x, int[] y) Fits multinomial logistic regression.
static LogisticRegression.Multinomial	multinomial(double[][] x, int[] y, double lambda, double tol, int maxIter) Fits multinomial logistic regression.
static LogisticRegression.Multinomial	multinomial(double[][] x, int[] y, java.util.Properties prop) Fits multinomial logistic regression.
static LogisticRegression.Multinomial	multinomial(smile.data.formula.Formula formula, smile.data.DataFrame data) Fits multinomial logistic regression.
static LogisticRegression.Multinomial	multinomial(smile.data.formula.Formula formula, smile.data.DataFrame data, java.util.Properties prop) Fits multinomial logistic regression.
void	setLearningRate(double rate) Sets the learning rate of stochastic gradient descent.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface smile.classification.SoftClassifier

predict, predict

Methods inherited from interface smile.classification.OnlineClassifier

update, update

Methods inherited from interface smile.classification.Classifier

applyAsDouble, applyAsInt, predict, predict, score

Constructor Detail

LogisticRegression

public LogisticRegression(int p,
                          double L,
                          double lambda,
                          smile.util.IntSet labels)

Constructor.

Parameters:

p - the dimension of input data.

L - the log-likelihood of learned model.

lambda - λ > 0 gives a "regularized" estimate of linear weights which often has superior generalization performance, especially when the dimensionality is high.

labels - class labels

Method Detail

binomial

public static LogisticRegression.Binomial binomial(smile.data.formula.Formula formula,
                                                   smile.data.DataFrame data)

Fits binomial logistic regression.

Parameters:

formula - a symbolic description of the model to be fitted.

data - the data frame of the explanatory and response variables.

binomial

public static LogisticRegression.Binomial binomial(smile.data.formula.Formula formula,
                                                   smile.data.DataFrame data,
                                                   java.util.Properties prop)

Fits binomial logistic regression.

Parameters:

formula - a symbolic description of the model to be fitted.

data - the data frame of the explanatory and response variables.

binomial

public static LogisticRegression.Binomial binomial(double[][] x,
                                                   int[] y)

Fits binomial logistic regression.

Parameters:

x - training samples.

y - training labels.

binomial

public static LogisticRegression.Binomial binomial(double[][] x,
                                                   int[] y,
                                                   java.util.Properties prop)

Fits binomial logistic regression.

Parameters:

x - training samples.

y - training labels.

binomial

public static LogisticRegression.Binomial binomial(double[][] x,
                                                   int[] y,
                                                   double lambda,
                                                   double tol,
                                                   int maxIter)

Fits binomial logistic regression.

Parameters:

x - training samples.

y - training labels.

lambda - λ > 0 gives a "regularized" estimate of linear weights which often has superior generalization performance, especially when the dimensionality is high.

tol - the tolerance for stopping iterations.

maxIter - the maximum number of iterations.

multinomial

public static LogisticRegression.Multinomial multinomial(smile.data.formula.Formula formula,
                                                         smile.data.DataFrame data)

Fits multinomial logistic regression.

Parameters:

formula - a symbolic description of the model to be fitted.

data - the data frame of the explanatory and response variables.

multinomial

public static LogisticRegression.Multinomial multinomial(smile.data.formula.Formula formula,
                                                         smile.data.DataFrame data,
                                                         java.util.Properties prop)

Fits multinomial logistic regression.

Parameters:

formula - a symbolic description of the model to be fitted.

data - the data frame of the explanatory and response variables.

multinomial

public static LogisticRegression.Multinomial multinomial(double[][] x,
                                                         int[] y)

Fits multinomial logistic regression.

Parameters:

x - training samples.

y - training labels.

multinomial

public static LogisticRegression.Multinomial multinomial(double[][] x,
                                                         int[] y,
                                                         java.util.Properties prop)

Fits multinomial logistic regression.

Parameters:

x - training samples.

y - training labels.

multinomial

public static LogisticRegression.Multinomial multinomial(double[][] x,
                                                         int[] y,
                                                         double lambda,
                                                         double tol,
                                                         int maxIter)

Fits multinomial logistic regression.

Parameters:

x - training samples.

y - training labels.

lambda - λ > 0 gives a "regularized" estimate of linear weights which often has superior generalization performance, especially when the dimensionality is high.

tol - the tolerance for stopping iterations.

maxIter - the maximum number of iterations.

fit

public static LogisticRegression fit(smile.data.formula.Formula formula,
                                     smile.data.DataFrame data)

Fits logistic regression.

Parameters:

formula - a symbolic description of the model to be fitted.

data - the data frame of the explanatory and response variables.

fit

public static LogisticRegression fit(smile.data.formula.Formula formula,
                                     smile.data.DataFrame data,
                                     java.util.Properties prop)

Fits logistic regression.

Parameters:

formula - a symbolic description of the model to be fitted.

data - the data frame of the explanatory and response variables.

fit

public static LogisticRegression fit(double[][] x,
                                     int[] y)

Fits logistic regression.

Parameters:

x - training samples.

y - training labels.

fit

public static LogisticRegression fit(double[][] x,
                                     int[] y,
                                     java.util.Properties prop)

Fits logistic regression.

Parameters:

x - training samples.

y - training labels.

fit

public static LogisticRegression fit(double[][] x,
                                     int[] y,
                                     double lambda,
                                     double tol,
                                     int maxIter)

Fits logistic regression.

Parameters:

x - training samples.

y - training labels.

lambda - λ > 0 gives a "regularized" estimate of linear weights which often has superior generalization performance, especially when the dimensionality is high.

tol - the tolerance for stopping iterations.

maxIter - the maximum number of iterations.

setLearningRate

public void setLearningRate(double rate)

Sets the learning rate of stochastic gradient descent. It is a good practice to adapt the learning rate for different data sizes. For example, it is typical to set the learning rate to eta/n, where eta is in [0.1, 0.3] and n is the size of the training data.

Parameters:

rate - the learning rate.

getLearningRate

public double getLearningRate()

Returns the learning rate of stochastic gradient descent.

loglikelihood

public double loglikelihood()

Returns the log-likelihood of model.

AIC

public double AIC()

Returns the AIC score.