StochasticMultinomialLogisticRegression (Java Statistical Analysis Tool 0.0.8 API)

java.lang.Object
- jsat.classifiers.linear.StochasticMultinomialLogisticRegression

All Implemented Interfaces:

Serializable, Cloneable, Classifier, Parameterized, SimpleWeightVectorModel
```
public class StochasticMultinomialLogisticRegression
extends Object
implements Classifier, Parameterized, SimpleWeightVectorModel
```
This is a Stochastic implementation of Multinomial Logistic Regression. It supports regularization from several different priors, and performs prior updates in a lazy fashion to avoid destroying the sparsity of training inputs.
Algorithm is based on the technical report:
Carpenter, B. (2008). Lazy Sparse Stochastic Gradient Descent for Regularized Mutlinomial Logistic Regression. Retrieved from http://lingpipe-blog.com/lingpipe-white-papers/

Author:

Edward Raff

See Also:

Serialized Form

Nested Class Summary

Nested Classes
Modifier and Type	Class and Description
`static class`	`StochasticMultinomialLogisticRegression.Prior` Represents a prior of the coefficients that can be applied to perform regularization.

Constructor Summary

Constructors
Modifier	Constructor and Description
	`StochasticMultinomialLogisticRegression()` Creates a new Stochastic Multinomial Logistic Regression that uses a `StochasticMultinomialLogisticRegression.Prior.GAUSSIAN` prior with a regularization scale of 1e-6.
	`StochasticMultinomialLogisticRegression(double initialLearningRate, int epochs)` Creates a new Stochastic Multinomial Logistic Regression that uses a `StochasticMultinomialLogisticRegression.Prior.GAUSSIAN` prior with a regularization scale of 1e-6.
	`StochasticMultinomialLogisticRegression(double initialLearningRate, int epochs, double regularization, StochasticMultinomialLogisticRegression.Prior prior)` Creates a new Stochastic Multinomial Logistic Regression object
`protected`	`StochasticMultinomialLogisticRegression(StochasticMultinomialLogisticRegression toClone)` Copy constructor

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`CategoricalResults`	`classify(DataPoint data)` Performs classification on the given data point.
`Classifier`	`clone()`
`double`	`getAlpha()` Returns the extra parameter value
`double`	`getBias(int index)` Returns the bias term used with the weight vector for the given class index.
`Vec`	`getCoefficientVector(int id)` Returns the raw coefficient vector used without the bias term.
`int`	`getEpochs()` Returns the maximum number of epochs
`double`	`getInitialLearningRate()` Returns the current initial learning rate
`DecayRate`	`getLearningRateDecay()` Returns the decay rate in use
`int`	`getMiniBatchSize()` Returns the number of data points used to perform each gradient update
`Parameter`	`getParameter(String paramName)` Returns the parameter with the given name.
`List<Parameter>`	`getParameters()` Returns the list of parameters that can be altered for this learner.
`StochasticMultinomialLogisticRegression.Prior`	`getPrior()` Returns the prior used for regularization
`Vec`	`getRawWeight(int index)` Returns the raw weight vector associated with the given class index.
`double`	`getRegularization()` Returns the regularization coefficient in use
`double`	`getTolerance()` Returns the minimum tolerance for early stopping.
`boolean`	`isClipping()` Returns whether or not coefficient clipping is on.
`boolean`	`isStandardized()` Returns whether or not the input is standardized for the priors
`boolean`	`isUseBias()` Returns `true` if the bias term is in use
`int`	`numWeightsVecs()` Returns the number of weight vectors that can be returned.
`void`	`setAlpha(double alpha)` Sets the extra parameter alpha.
`void`	`setClipping(boolean clipping)` Sets whether or not the clip changes in coefficient values caused by regularization so that they can not make the coefficients go from positive to negative or negative to positive.
`void`	`setEpochs(int epochs)` Sets the maximum number of epochs that occur in each iteration.
`void`	`setInitialLearningRate(double initialLearningRate)` Sets the initial learning rate to use for the first epoch.
`void`	`setLearningRateDecay(DecayRate learningRateDecay)` Sets the decay rate used to reduce the learning rate after each epoch.
`void`	`setMiniBatchSize(int miniBatchSize)` Sets the amount of data points used to form each gradient update.
`void`	`setPrior(StochasticMultinomialLogisticRegression.Prior prior)` Sets the prior used to perform regularization
`void`	`setRegularization(double regularization)` Sets the coefficient applied to the regularization penalty at each update.
`void`	`setStandardized(boolean standardized)` Sets whether or not to perform implicit standardization of the feature values when performing regularization by the prior.
`void`	`setTolerance(double tolerance)` Sets the tolerance that determines when the training stops early because the change has become too insignificant.
`void`	`setUseBias(boolean useBias)` Sets whether or not to learn the bias term for a model.
`boolean`	`supportsWeightedData()` Indicates whether the model knows how to train using weighted data points.
`void`	`trainC(ClassificationDataSet dataSet)` Trains the classifier and constructs a model for classification using the given data set.
`void`	`trainC(ClassificationDataSet dataSet, ExecutorService threadPool)` Trains the classifier and constructs a model for classification using the given data set.

Methods inherited from class java.lang.Object
equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Constructor Detail
  - StochasticMultinomialLogisticRegression
```
public StochasticMultinomialLogisticRegression(double initialLearningRate,
                                               int epochs,
                                               double regularization,
                                               StochasticMultinomialLogisticRegression.Prior prior)
```
    Creates a new Stochastic Multinomial Logistic Regression object
    
    Parameters:
    
    initialLearningRate - the initial learning rate to use
    
    epochs - the maximum number of training epochs to go through
    
    regularization - the scale factor applied to the regularization term
    
    prior - the prior to use for regularization
  - StochasticMultinomialLogisticRegression
```
public StochasticMultinomialLogisticRegression(double initialLearningRate,
                                               int epochs)
```
    Creates a new Stochastic Multinomial Logistic Regression that uses a StochasticMultinomialLogisticRegression.Prior.GAUSSIAN prior with a regularization scale of 1e-6.
    
    Parameters:
    
    initialLearningRate - the initial learning rate to use
    
    epochs - the maximum number of training epochs to go through
  - StochasticMultinomialLogisticRegression
```
public StochasticMultinomialLogisticRegression()
```
    Creates a new Stochastic Multinomial Logistic Regression that uses a StochasticMultinomialLogisticRegression.Prior.GAUSSIAN prior with a regularization scale of 1e-6. It will do at most 50 epochs with a learning rate of 0.1
  - StochasticMultinomialLogisticRegression
```
protected StochasticMultinomialLogisticRegression(StochasticMultinomialLogisticRegression toClone)
```
    Copy constructor
    
    Parameters:
    
    toClone - the classifier to create a copy of
- Method Detail
  - setUseBias
```
public void setUseBias(boolean useBias)
```
    Sets whether or not to learn the bias term for a model. If no bias term is in use, the model learned must pass through the origin of the world. The use of the bias term is very important for low dimensional problems, but less so for many higher dimensional problems.
    
    Parameters:
    
    useBias - true if the bias term should be used, false otherwise
  - isUseBias
```
public boolean isUseBias()
```
    Returns true if the bias term is in use
    
    Returns:
    
    true if the bias term is in use
  - setEpochs
```
public void setEpochs(int epochs)
```
    Sets the maximum number of epochs that occur in each iteration. Each epoch goes through the whole data set once.
    
    Parameters:
    
    epochs - the maximum number of epochs to train
  - getEpochs
```
public int getEpochs()
```
    Returns the maximum number of epochs
    
    Returns:
    
    the maximum number of epochs
  - setAlpha
```
public void setAlpha(double alpha)
```
    Sets the extra parameter alpha. This is used for some priors that take an extra parameter. This is StochasticMultinomialLogisticRegression.Prior.CAUCHY and StochasticMultinomialLogisticRegression.Prior.ELASTIC. If these two priors are not in use, the value is ignored.
    
    Parameters:
    
    alpha - the extra parameter value to use. Must be positive
  - getAlpha
```
public double getAlpha()
```
    Returns the extra parameter value
    
    Returns:
    
    the extra parameter value
  - setClipping
```
public void setClipping(boolean clipping)
```
    Sets whether or not the clip changes in coefficient values caused by regularization so that they can not make the coefficients go from positive to negative or negative to positive. If clipping is on, the value will go to zero instead. If off, the value will be allowed to change signs.
    If there is no regularization, this has no impact.
    
    Parameters:
    
    clipping - true if clipping should be used, false otherwise
  - isClipping
```
public boolean isClipping()
```
    Returns whether or not coefficient clipping is on.
    
    Returns:
    
    true if clipping is on.
  - setInitialLearningRate
```
public void setInitialLearningRate(double initialLearningRate)
```
    Sets the initial learning rate to use for the first epoch. The learning rate will decay according to the decay rate in use.
    
    Parameters:
    
    initialLearningRate - the initial learning rate to use
  - getInitialLearningRate
```
public double getInitialLearningRate()
```
    Returns the current initial learning rate
    
    Returns:
    
    the learning rate in use
  - setLearningRateDecay
```
public void setLearningRateDecay(DecayRate learningRateDecay)
```
    Sets the decay rate used to reduce the learning rate after each epoch.
    
    Parameters:
    
    learningRateDecay - the decay rate to use
  - getLearningRateDecay
```
public DecayRate getLearningRateDecay()
```
    Returns the decay rate in use
    
    Returns:
    
    the decay rate in use
  - setRegularization
```
public void setRegularization(double regularization)
```
    Sets the coefficient applied to the regularization penalty at each update. This is usual set to a small value less than 1. If set to zero, it effectively turns off the use of regularization.
    
    Parameters:
    
    regularization - the non negative regularization coefficient to apply
  - getRegularization
```
public double getRegularization()
```
    Returns the regularization coefficient in use
    
    Returns:
    
    the regularization coefficient in use
  - setPrior
```
public void setPrior(StochasticMultinomialLogisticRegression.Prior prior)
```
    Sets the prior used to perform regularization
    
    Parameters:
    
    prior - the prior to use
  - getPrior
```
public StochasticMultinomialLogisticRegression.Prior getPrior()
```
    Returns the prior used for regularization
    
    Returns:
    
    the prior used
  - setTolerance
```
public void setTolerance(double tolerance)
```
    Sets the tolerance that determines when the training stops early because the change has become too insignificant.
    
    Parameters:
    
    tolerance - the minimum change in log likelihood to stop training
  - getTolerance
```
public double getTolerance()
```
    Returns the minimum tolerance for early stopping.
    
    Returns:
    
    the minimum change in log likelihood to stop training
  - setStandardized
```
public void setStandardized(boolean standardized)
```
    Sets whether or not to perform implicit standardization of the feature values when performing regularization by the prior. If set on, the input data will be adjusted to have zero mean and unit variance. This is done without destroying sparsity. If there is not regularization, this parameter has no impact.
    
    Parameters:
    
    standardized - true if the input will be standardized, false if ti will be left as is.
  - isStandardized
```
public boolean isStandardized()
```
    Returns whether or not the input is standardized for the priors
    
    Returns:
    
    true if the input is standardized for the priors
  - setMiniBatchSize
```
public void setMiniBatchSize(int miniBatchSize)
```
    Sets the amount of data points used to form each gradient update. Increasing the batch size can help convergence. By default, a mini batch size of 1 is used.
    
    Parameters:
    
    miniBatchSize - the number of data points used to perform each update
  - getMiniBatchSize
```
public int getMiniBatchSize()
```
    Returns the number of data points used to perform each gradient update
    
    Returns:
    
    the number of data points used to perform each gradient update
  - getRawWeight
```
public Vec getRawWeight(int index)
```
    Description copied from interface: SimpleWeightVectorModel
    
    Returns the raw weight vector associated with the given class index. If the given class is an implicit zero vector, a ConstantVector object may be returned.
    Do not alter the returned weight vector, as it will change the model's values.
    
    If a regression problem, only index = 0 should be used
    
    Specified by:
    
    getRawWeight in interface SimpleWeightVectorModel
    
    Parameters:
    
    index - the class index to get the weight vector for
    
    Returns:
    
    the weight vector used for the specified class
  - getBias
```
public double getBias(int index)
```
    Description copied from interface: SimpleWeightVectorModel
    
    Returns the bias term used with the weight vector for the given class index. If the model does not support or was not trained with bias weights, 0 will be returned.
    
    If a regression problem, only index = 0 should be used
    
    Specified by:
    
    getBias in interface SimpleWeightVectorModel
    
    Parameters:
    
    index - the class index to get the weight vector for
    
    Returns:
    
    the bias term for the specified class
  - numWeightsVecs
```
public int numWeightsVecs()
```
    Description copied from interface: SimpleWeightVectorModel
    
    Returns the number of weight vectors that can be returned. For binary classification problems the value may be 1 if only a single weight vector's sign is used to determine the class. For multi-class problems, the weight vector count includes the implicit zero vector (if one is being used).
    
    Specified by:
    
    numWeightsVecs in interface SimpleWeightVectorModel
    
    Returns:
    
    the number of weight vectors for which SimpleWeightVectorModel.getRawWeight(int) can be called.
  - classify
```
public CategoricalResults classify(DataPoint data)
```
    Description copied from interface: Classifier
    
    Performs classification on the given data point.
    
    Specified by:
    
    classify in interface Classifier
    
    Parameters:
    
    data - the data point to classify
    
    Returns:
    
    the results of the classification.
  - trainC
```
public void trainC(ClassificationDataSet dataSet,
                   ExecutorService threadPool)
```
    Description copied from interface: Classifier
    
    Trains the classifier and constructs a model for classification using the given data set. If the training method knows how, it will used the threadPool to conduct training in parallel. This method will block until the training has completed.
    
    Specified by:
    
    trainC in interface Classifier
    
    Parameters:
    
    dataSet - the data set to train on
    
    threadPool - the source of threads to use.
  - trainC
```
public void trainC(ClassificationDataSet dataSet)
```
    Description copied from interface: Classifier
    
    Trains the classifier and constructs a model for classification using the given data set.
    
    Specified by:
    
    trainC in interface Classifier
    
    Parameters:
    
    dataSet - the data set to train on
  - supportsWeightedData
```
public boolean supportsWeightedData()
```
    Description copied from interface: Classifier
    
    Indicates whether the model knows how to train using weighted data points. If it does, the model will train assuming the weights. The values returned by this method may change depending on the parameters set for the model.
    
    Specified by:
    
    supportsWeightedData in interface Classifier
    
    Returns:
    
    true if the model supports weighted data, false otherwise
  - getCoefficientVector
```
public Vec getCoefficientVector(int id)
```
    Returns the raw coefficient vector used without the bias term. For a multinomial Logistic model, there are C-1 coefficient vectors. C is the number of output classes. Altering the returned vector will alter the model. The i'th index of the vector corresponds to the weight therm for the i'th index in an input.
    
    Parameters:
    
    id - which coefficient vector to obtain
    
    Returns:
    
    the vector of variable coefficients.
  - clone
```
public Classifier clone()
```
    Specified by:
    
    clone in interface Classifier
    
    Overrides:
    
    clone in class Object
  - getParameters
```
public List<Parameter> getParameters()
```
    Description copied from interface: Parameterized
    
    Returns the list of parameters that can be altered for this learner.
    
    Specified by:
    
    getParameters in interface Parameterized
    
    Returns:
    
    the list of parameters that can be altered for this learner.
  - getParameter
```
public Parameter getParameter(String paramName)
```
    Description copied from interface: Parameterized
    
    Returns the parameter with the given name. Two different strings may map to a single Parameter object. An ASCII only string, and a Unicode style string.
    
    Specified by:
    
    getParameter in interface Parameterized
    
    Parameters:
    
    paramName - the name of the parameter to obtain
    
    Returns:
    
    the Parameter in question, or null if no such named Parameter exists.

Class StochasticMultinomialLogisticRegression

Nested Class Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Constructor Detail

StochasticMultinomialLogisticRegression

StochasticMultinomialLogisticRegression

StochasticMultinomialLogisticRegression

StochasticMultinomialLogisticRegression

Method Detail

setUseBias

isUseBias

setEpochs

getEpochs

setAlpha

getAlpha

setClipping

isClipping

setInitialLearningRate

getInitialLearningRate

setLearningRateDecay

getLearningRateDecay

setRegularization

getRegularization

setPrior

getPrior

setTolerance

getTolerance

setStandardized

isStandardized

setMiniBatchSize

getMiniBatchSize

getRawWeight

getBias

numWeightsVecs

classify

trainC

trainC

supportsWeightedData

getCoefficientVector

clone

getParameters

getParameter