jsat.classifiers.linear

Class SMIDAS

java.lang.Object

jsat.classifiers.linear.StochasticSTLinearL1

jsat.classifiers.linear.SMIDAS

All Implemented Interfaces:

Serializable, Cloneable, Classifier, Parameterized, Regressor, SimpleWeightVectorModel, SingleWeightVectorModel

public class SMIDAS
extends StochasticSTLinearL1

Implements the iterative and single threaded stochastic solver for L₁ regularized linear regression problems SMIDAS (Stochastic Mirror Descent Algorithm mAde Sparse). It performs very well when the number of features is large relative to or greater than the number of data points. It also works decently on smaller sparse data sets.
Using the squared loss is equivalent to LASSO regression, and the LOG loss is equivalent to logistic regression.

Note: This implementation requires all feature values to be in the range [-1, 1]. By default scaling is performed to [0,1] to preserve sparseness. If your data is dense or has negative and positive feature values, scaling to [-1, 1] may perform better. See StochasticSTLinearL1.setReScale(boolean)

See:
Shalev-Shwartz, S.,&Tewari, A. (2009). Stochastic Methods for L₁-regularized Loss Minimization. 26th International Conference on Machine Learning (Vol. 12, pp. 929–936).

Author:

Edward Raff

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class jsat.classifiers.linear.StochasticSTLinearL1

StochasticSTLinearL1.Loss

Field Summary

Fields inherited from class jsat.classifiers.linear.StochasticSTLinearL1

bias, DEFAULT_EPOCHS, DEFAULT_LOSS, DEFAULT_REG, epochs, lambda, loss, maxScaled, minScaled, obvMax, obvMin, reScale, w

Constructor Summary

Constructors

Constructor and Description
SMIDAS(double eta) Creates a new SMIDAS learner
SMIDAS(double eta, int epochs, double lambda, StochasticSTLinearL1.Loss loss) Creates a new SMIDAS learner
SMIDAS(double eta, int epochs, double lambda, StochasticSTLinearL1.Loss loss, boolean reScale) Creates a new SMIDAS learner

Method Summary

Modifier and Type	Method and Description
CategoricalResults	classify(DataPoint data) Performs classification on the given data point.
SMIDAS	clone()
double	getEta() Returns the current learning rate used during training
double	regress(DataPoint data)
void	setEta(double eta) Sets the learning rate used during training
boolean	supportsWeightedData() Indicates whether the model knows how to train using weighted data points.
void	train(RegressionDataSet dataSet)
void	train(RegressionDataSet dataSet, ExecutorService threadPool)
void	trainC(ClassificationDataSet dataSet) Trains the classifier and constructs a model for classification using the given data set.
void	trainC(ClassificationDataSet dataSet, ExecutorService threadPool) Trains the classifier and constructs a model for classification using the given data set.

Methods inherited from class jsat.classifiers.linear.StochasticSTLinearL1

getBias, getBias, getEpochs, getLambda, getLoss, getMaxScaled, getMinScaled, getParameter, getParameters, getRawWeight, getRawWeight, getW, getWRaw, isReScale, numWeightsVecs, setEpochs, setLambda, setLoss, setMaxScaled, setMinScaled, setReScale, wDot

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SMIDAS

public SMIDAS(double eta)

Creates a new SMIDAS learner

Parameters:

eta - the learning rate for each iteration

SMIDAS

public SMIDAS(double eta,
              int epochs,
              double lambda,
              StochasticSTLinearL1.Loss loss)

Creates a new SMIDAS learner

Parameters:

eta - the learning rate for each iteration

epochs - the number of learning iterations

lambda - the regularization penalty

loss - the loss function to use

SMIDAS

public SMIDAS(double eta,
              int epochs,
              double lambda,
              StochasticSTLinearL1.Loss loss,
              boolean reScale)

Creates a new SMIDAS learner

Parameters:

eta - the learning rate for each iteration

epochs - the number of learning iterations

lambda - the regularization penalty

loss - the loss function to use

reScale - whether or not to rescale the feature values

Method Detail

setEta

public void setEta(double eta)

Sets the learning rate used during training

Parameters:

eta - the learning rate to use

getEta

public double getEta()

Returns the current learning rate used during training

Returns:

the learning rate in use

classify

public CategoricalResults classify(DataPoint data)

Description copied from interface: Classifier

Performs classification on the given data point.

Parameters:

data - the data point to classify

Returns:

the results of the classification.

regress

public double regress(DataPoint data)

trainC

public void trainC(ClassificationDataSet dataSet,
                   ExecutorService threadPool)

Description copied from interface: Classifier

Trains the classifier and constructs a model for classification using the given data set. If the training method knows how, it will used the threadPool to conduct training in parallel. This method will block until the training has completed.

Parameters:

dataSet - the data set to train on

threadPool - the source of threads to use.

trainC

public void trainC(ClassificationDataSet dataSet)

Description copied from interface: Classifier

Trains the classifier and constructs a model for classification using the given data set.

Parameters:

dataSet - the data set to train on

train

public void train(RegressionDataSet dataSet,
                  ExecutorService threadPool)

train

public void train(RegressionDataSet dataSet)

supportsWeightedData

public boolean supportsWeightedData()

Description copied from interface: Classifier

Indicates whether the model knows how to train using weighted data points. If it does, the model will train assuming the weights. The values returned by this method may change depending on the parameters set for the model.

Returns:

true if the model supports weighted data, false otherwise

clone

public SMIDAS clone()

Specified by:

clone in interface Classifier

Specified by:

clone in interface Regressor

Specified by:

clone in class StochasticSTLinearL1