jsat.classifiers.linear

Class LinearL1SCD

java.lang.Object

jsat.classifiers.linear.StochasticSTLinearL1

jsat.classifiers.linear.LinearL1SCD

All Implemented Interfaces:

Serializable, Cloneable, Classifier, Parameterized, Regressor, SimpleWeightVectorModel, SingleWeightVectorModel

public class LinearL1SCD
extends StochasticSTLinearL1

Implements an iterative and single threaded form of fast Stochastic Coordinate Decent for optimizing L₁ regularized linear regression problems. It performs very well when the number of data points is very large, especially when the feature count is smaller in comparison. It also works well on sparse data sets.
Using the squared loss is equivalent to LASSO regression, and the LOG loss is equivalent to logistic regression.

Note: This implementation requires all feature values to be in the range [-1, 1]. By default scaling is performed to [0,1]. If your data is dense or has negative and positive feature values, scaling to [-1, 1] may perform better. See StochasticSTLinearL1.setReScale(boolean)

See:
Shalev-Shwartz, S.,&Tewari, A. (2009). Stochastic Methods for L₁-regularized Loss Minimization. 26th International Conference on Machine Learning (Vol. 12, pp. 929–936).

Author:

Edward Raff

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class jsat.classifiers.linear.StochasticSTLinearL1

StochasticSTLinearL1.Loss

Field Summary

Fields inherited from class jsat.classifiers.linear.StochasticSTLinearL1

bias, DEFAULT_EPOCHS, DEFAULT_LOSS, DEFAULT_REG, epochs, lambda, loss, maxScaled, minScaled, obvMax, obvMin, reScale, w

Constructor Summary

Constructors

Constructor and Description
LinearL1SCD() Creates a new SCD L1 learner using default settings.
LinearL1SCD(int epochs, double lambda, StochasticSTLinearL1.Loss loss) Creates a new SCD L1 learner.
LinearL1SCD(int epochs, double lambda, StochasticSTLinearL1.Loss loss, boolean reScale) Creates a new SCD L1 learner.

Method Summary

Modifier and Type	Method and Description
CategoricalResults	classify(DataPoint data) Performs classification on the given data point.
LinearL1SCD	clone()
double	regress(DataPoint data)
boolean	supportsWeightedData() Indicates whether the model knows how to train using weighted data points.
void	train(RegressionDataSet dataSet)
void	train(RegressionDataSet dataSet, ExecutorService threadPool)
void	trainC(ClassificationDataSet dataSet) Trains the classifier and constructs a model for classification using the given data set.
void	trainC(ClassificationDataSet dataSet, ExecutorService threadPool) Trains the classifier and constructs a model for classification using the given data set.

Methods inherited from class jsat.classifiers.linear.StochasticSTLinearL1

getBias, getBias, getEpochs, getLambda, getLoss, getMaxScaled, getMinScaled, getParameter, getParameters, getRawWeight, getRawWeight, getW, getWRaw, isReScale, numWeightsVecs, setEpochs, setLambda, setLoss, setMaxScaled, setMinScaled, setReScale, wDot

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

LinearL1SCD

public LinearL1SCD()

Creates a new SCD L₁ learner using default settings.

LinearL1SCD

public LinearL1SCD(int epochs,
                   double lambda,
                   StochasticSTLinearL1.Loss loss)

Creates a new SCD L₁ learner.

Parameters:

epochs - the number of learning iterations

lambda - the regularization penalty

loss - the loss function to use

LinearL1SCD

public LinearL1SCD(int epochs,
                   double lambda,
                   StochasticSTLinearL1.Loss loss,
                   boolean reScale)

Creates a new SCD L₁ learner.

Parameters:

epochs - the number of learning iterations

lambda - the regularization penalty

loss - the loss function to use

reScale - whether or not to rescale the feature values

Method Detail

classify

public CategoricalResults classify(DataPoint data)

Description copied from interface: Classifier

Performs classification on the given data point.

Parameters:

data - the data point to classify

Returns:

the results of the classification.

regress

public double regress(DataPoint data)

train

public void train(RegressionDataSet dataSet,
                  ExecutorService threadPool)

train

public void train(RegressionDataSet dataSet)

trainC

public void trainC(ClassificationDataSet dataSet,
                   ExecutorService threadPool)

Description copied from interface: Classifier

Trains the classifier and constructs a model for classification using the given data set. If the training method knows how, it will used the threadPool to conduct training in parallel. This method will block until the training has completed.

Parameters:

dataSet - the data set to train on

threadPool - the source of threads to use.

trainC

public void trainC(ClassificationDataSet dataSet)

Description copied from interface: Classifier

Trains the classifier and constructs a model for classification using the given data set.

Parameters:

dataSet - the data set to train on

supportsWeightedData

public boolean supportsWeightedData()

Description copied from interface: Classifier

Indicates whether the model knows how to train using weighted data points. If it does, the model will train assuming the weights. The values returned by this method may change depending on the parameters set for the model.

Returns:

true if the model supports weighted data, false otherwise

clone

public LinearL1SCD clone()

Specified by:

clone in interface Classifier

Specified by:

clone in interface Regressor

Specified by:

clone in class StochasticSTLinearL1