LinearSGD (Java Statistical Analysis Tool 0.0.8 API)

java.lang.Object
- jsat.classifiers.BaseUpdateableClassifier
- - jsat.classifiers.linear.LinearSGD

All Implemented Interfaces:

Serializable, Cloneable, Classifier, UpdateableClassifier, Parameterized, Regressor, UpdateableRegressor, SimpleWeightVectorModel
```
public class LinearSGD
extends BaseUpdateableClassifier
implements UpdateableRegressor, Parameterized, SimpleWeightVectorModel
```
LinearSGD learns either a classification or regression problem depending on the loss function ℓ(w,x) used. The solution attempts to minimize ∑_i ℓ(w,x_i) + λ₀/2 ||w||₂² + λ₁ ||w||₁, and is trained by Stochastic Gradient Descent.

NOTE: To support L₁ regularization with sparse results and online learning at the same time, the normalization of the regularization penalty by the number of data points is not present in the implementation at this time. Setting λ₁ to the desired value divided by the number of unique data points in the whole set will result in the correct regularization penalty being applied. See:
- Tsuruoka, Y., Tsujii, J.,&Ananiadou, S. (2009). Stochastic gradient descent training for L1-regularized log-linear models with cumulative penalty. Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, 1, 477. doi:10.3115/1687878.1687946
Author:

Edward Raff

See Also:

Serialized Form

Constructor Summary

Constructors
Constructor and Description
`LinearSGD()` Creates a new LinearSGD learner for multi-class classification problems.
`LinearSGD(LinearSGD toClone)` Copy constructor
`LinearSGD(LossFunc loss, double eta, DecayRate decay, double lambda0, double lambda1)` Creates a new LinearSGD learner.
`LinearSGD(LossFunc loss, double lambda0, double lambda1)` Creates a new LinearSGD learner

Method Summary

All Methods Static Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`CategoricalResults`	`classify(DataPoint data)` Performs classification on the given data point.
`LinearSGD`	`clone()`
`double`	`getBias(int index)` Returns the bias term used with the weight vector for the given class index.
`double`	`getEta()` Returns the current learning rate in use
`DecayRate`	`getEtaDecay()` Returns the decay rate in use
`GradientUpdater`	`getGradientUpdater()`
`double`	`getLambda0()` Returns the L₂ regularization term in use
`double`	`getLambda1()` Returns the L₁ regularization term in use
`LossFunc`	`getLoss()` Returns the loss function in use
`Parameter`	`getParameter(String paramName)` Returns the parameter with the given name.
`List<Parameter>`	`getParameters()` Returns the list of parameters that can be altered for this learner.
`Vec`	`getRawWeight(int index)` Returns the raw weight vector associated with the given class index.
`static Distribution`	`guessLambda0(DataSet d)` Guess the distribution to use for the regularization term `λ₀` .
`static Distribution`	`guessLambda1(DataSet d)` Guess the distribution to use for the regularization term `λ₁` .
`boolean`	`isUseBias()` Returns whether or not an implicit bias term is in use
`int`	`numWeightsVecs()` Returns the number of weight vectors that can be returned.
`double`	`regress(DataPoint data)`
`void`	`setEta(double eta)` Sets the initial learning rate η to use.
`void`	`setEtaDecay(DecayRate decay)` Sets the rate at which `η` is decayed at each update.
`void`	`setGradientUpdater(GradientUpdater gradientUpdater)` Sets the method that will be used to update the weight vectors given their gradient information.
`void`	`setLambda0(double lambda0)` λ₀ controls the L₂ regularization penalty.
`void`	`setLambda1(double lambda1)` λ₁ controls the L₁ regularization penalty.
`void`	`setLoss(LossFunc loss)` Sets the loss function used for the model.
`void`	`setUp(CategoricalData[] categoricalAttributes, int numericAttributes)` Prepares the classifier to begin learning from its `UpdateableRegressor.update(jsat.classifiers.DataPoint, double)` method.
`void`	`setUp(CategoricalData[] categoricalAttributes, int numericAttributes, CategoricalData predicting)` Prepares the classifier to begin learning from its `UpdateableClassifier.update(jsat.classifiers.DataPoint, int)` method.
`void`	`setUseBias(boolean useBias)` Sets whether or not an implicit bias term will be added to the data set
`boolean`	`supportsWeightedData()` Indicates whether the model knows how to train using weighted data points.
`void`	`train(RegressionDataSet dataSet)`
`void`	`train(RegressionDataSet dataSet, ExecutorService threadPool)`
`void`	`update(DataPoint dataPoint, double targetValue)` Updates the classifier by giving it a new data point to learn from.
`void`	`update(DataPoint dataPoint, int targetClass)` Updates the classifier by giving it a new data point to learn from.

Methods inherited from class jsat.classifiers.BaseUpdateableClassifier
getEpochs, setEpochs, trainC, trainC, trainEpochs

Methods inherited from class java.lang.Object
equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Constructor Detail
  - LinearSGD
```
public LinearSGD()
```
    Creates a new LinearSGD learner for multi-class classification problems.
  - LinearSGD
```
public LinearSGD(LossFunc loss,
                 double lambda0,
                 double lambda1)
```
    Creates a new LinearSGD learner
    
    Parameters:
    
    loss - the loss function to use
    
    lambda0 - the L₂ regularization term
    
    lambda1 - the L₁ regularization term
  - LinearSGD
```
public LinearSGD(LossFunc loss,
                 double eta,
                 DecayRate decay,
                 double lambda0,
                 double lambda1)
```
    Creates a new LinearSGD learner.
    
    Parameters:
    
    loss - the loss function to use
    
    eta - the initial learning rate
    
    decay - the decay rate for η
    
    lambda0 - the L₂ regularization term
    
    lambda1 - the L₁ regularization term
  - LinearSGD
```
public LinearSGD(LinearSGD toClone)
```
    Copy constructor
    
    Parameters:
    
    toClone - the object to copy
- Method Detail
  - setGradientUpdater
```
public void setGradientUpdater(GradientUpdater gradientUpdater)
```
    Sets the method that will be used to update the weight vectors given their gradient information.
    
    Parameters:
    
    gradientUpdater - the method to use for updating the weight vectors from the gradient
  - getGradientUpdater
```
public GradientUpdater getGradientUpdater()
```
    Returns:
    
    the method to use for updating the weight vectors from the gradient
  - setEtaDecay
```
public void setEtaDecay(DecayRate decay)
```
    Sets the rate at which η is decayed at each update.
    
    Parameters:
    
    decay - the decay rate to use
  - getEtaDecay
```
public DecayRate getEtaDecay()
```
    Returns the decay rate in use
    
    Returns:
    
    the decay rate in use
  - setEta
```
public void setEta(double eta)
```
    Sets the initial learning rate η to use. It should generally be in (0, 1), but any positive value is acceptable.
    
    Parameters:
    
    eta - the learning rate to use.
  - getEta
```
public double getEta()
```
    Returns the current learning rate in use
    
    Returns:
    
    the current learning rate in use
  - setLoss
```
public void setLoss(LossFunc loss)
```
    Sets the loss function used for the model. The loss function controls whether or not regression, binary classification, or multi-class classification is supported.
    
    Parameters:
    
    loss - the loss function to use
  - getLoss
```
public LossFunc getLoss()
```
    Returns the loss function in use
    
    Returns:
    
    the loss function in use
  - setLambda0
```
public void setLambda0(double lambda0)
```
    λ₀ controls the L₂ regularization penalty.
    
    Parameters:
    
    lambda0 - the L₂ regularization penalty to use
  - getLambda0
```
public double getLambda0()
```
    Returns the L₂ regularization term in use
    
    Returns:
    
    the L₂ regularization term in use
  - setLambda1
```
public void setLambda1(double lambda1)
```
    λ₁ controls the L₁ regularization penalty.
    
    Parameters:
    
    lambda1 - the L₁ regularization penalty to use
  - getLambda1
```
public double getLambda1()
```
    Returns the L₁ regularization term in use
    
    Returns:
    
    the L₁ regularization term in use
  - setUseBias
```
public void setUseBias(boolean useBias)
```
    Sets whether or not an implicit bias term will be added to the data set
    
    Parameters:
    
    useBias - true to add an implicit bias term
  - isUseBias
```
public boolean isUseBias()
```
    Returns whether or not an implicit bias term is in use
    
    Returns:
    
    true if a bias term is in use
  - clone
```
public LinearSGD clone()
```
    Specified by:
    
    clone in interface Classifier
    
    Specified by:
    
    clone in interface UpdateableClassifier
    
    Specified by:
    
    clone in interface Regressor
    
    Specified by:
    
    clone in interface UpdateableRegressor
    
    Specified by:
    
    clone in class BaseUpdateableClassifier
  - setUp
```
public void setUp(CategoricalData[] categoricalAttributes,
                  int numericAttributes,
                  CategoricalData predicting)
```
    Description copied from interface: UpdateableClassifier
    
    Prepares the classifier to begin learning from its UpdateableClassifier.update(jsat.classifiers.DataPoint, int) method.
    
    Specified by:
    
    setUp in interface UpdateableClassifier
    
    Parameters:
    
    categoricalAttributes - an array containing the categorical attributes that will be in each data point
    
    numericAttributes - the number of numeric attributes that will be in each data point
    
    predicting - the information for the target class that will be predicted
  - setUp
```
public void setUp(CategoricalData[] categoricalAttributes,
                  int numericAttributes)
```
    Description copied from interface: UpdateableRegressor
    
    Prepares the classifier to begin learning from its UpdateableRegressor.update(jsat.classifiers.DataPoint, double) method.
    
    Specified by:
    
    setUp in interface UpdateableRegressor
    
    Parameters:
    
    categoricalAttributes - an array containing the categorical attributes that will be in each data point
    
    numericAttributes - the number of numeric attributes that will be in each data point
  - update
```
public void update(DataPoint dataPoint,
                   int targetClass)
```
    Description copied from interface: UpdateableClassifier
    
    Updates the classifier by giving it a new data point to learn from.
    
    Specified by:
    
    update in interface UpdateableClassifier
    
    Parameters:
    
    dataPoint - the data point to learn
    
    targetClass - the target class of the data point
  - update
```
public void update(DataPoint dataPoint,
                   double targetValue)
```
    Description copied from interface: UpdateableRegressor
    
    Updates the classifier by giving it a new data point to learn from.
    
    Specified by:
    
    update in interface UpdateableRegressor
    
    Parameters:
    
    dataPoint - the data point to learn
    
    targetValue - the target value of the data point
  - classify
```
public CategoricalResults classify(DataPoint data)
```
    Description copied from interface: Classifier
    
    Performs classification on the given data point.
    
    Specified by:
    
    classify in interface Classifier
    
    Parameters:
    
    data - the data point to classify
    
    Returns:
    
    the results of the classification.
  - regress
```
public double regress(DataPoint data)
```
    Specified by:
    
    regress in interface Regressor
  - supportsWeightedData
```
public boolean supportsWeightedData()
```
    Description copied from interface: Classifier
    
    Indicates whether the model knows how to train using weighted data points. If it does, the model will train assuming the weights. The values returned by this method may change depending on the parameters set for the model.
    
    Specified by:
    
    supportsWeightedData in interface Classifier
    
    Specified by:
    
    supportsWeightedData in interface Regressor
    
    Returns:
    
    true if the model supports weighted data, false otherwise
  - train
```
public void train(RegressionDataSet dataSet,
                  ExecutorService threadPool)
```
    Specified by:
    
    train in interface Regressor
  - train
```
public void train(RegressionDataSet dataSet)
```
    Specified by:
    
    train in interface Regressor
  - getParameters
```
public List<Parameter> getParameters()
```
    Description copied from interface: Parameterized
    
    Returns the list of parameters that can be altered for this learner.
    
    Specified by:
    
    getParameters in interface Parameterized
    
    Returns:
    
    the list of parameters that can be altered for this learner.
  - getParameter
```
public Parameter getParameter(String paramName)
```
    Description copied from interface: Parameterized
    
    Returns the parameter with the given name. Two different strings may map to a single Parameter object. An ASCII only string, and a Unicode style string.
    
    Specified by:
    
    getParameter in interface Parameterized
    
    Parameters:
    
    paramName - the name of the parameter to obtain
    
    Returns:
    
    the Parameter in question, or null if no such named Parameter exists.
  - getRawWeight
```
public Vec getRawWeight(int index)
```
    Description copied from interface: SimpleWeightVectorModel
    
    Returns the raw weight vector associated with the given class index. If the given class is an implicit zero vector, a ConstantVector object may be returned.
    Do not alter the returned weight vector, as it will change the model's values.
    
    If a regression problem, only index = 0 should be used
    
    Specified by:
    
    getRawWeight in interface SimpleWeightVectorModel
    
    Parameters:
    
    index - the class index to get the weight vector for
    
    Returns:
    
    the weight vector used for the specified class
  - getBias
```
public double getBias(int index)
```
    Description copied from interface: SimpleWeightVectorModel
    
    Returns the bias term used with the weight vector for the given class index. If the model does not support or was not trained with bias weights, 0 will be returned.
    
    If a regression problem, only index = 0 should be used
    
    Specified by:
    
    getBias in interface SimpleWeightVectorModel
    
    Parameters:
    
    index - the class index to get the weight vector for
    
    Returns:
    
    the bias term for the specified class
  - numWeightsVecs
```
public int numWeightsVecs()
```
    Description copied from interface: SimpleWeightVectorModel
    
    Returns the number of weight vectors that can be returned. For binary classification problems the value may be 1 if only a single weight vector's sign is used to determine the class. For multi-class problems, the weight vector count includes the implicit zero vector (if one is being used).
    
    Specified by:
    
    numWeightsVecs in interface SimpleWeightVectorModel
    
    Returns:
    
    the number of weight vectors for which SimpleWeightVectorModel.getRawWeight(int) can be called.
  - guessLambda0
```
public static Distribution guessLambda0(DataSet d)
```
    Guess the distribution to use for the regularization term λ₀ .
    
    Parameters:
    
    d - the data set to get the guess for
    
    Returns:
    
    the guess for the λ₀ parameter
  - guessLambda1
```
public static Distribution guessLambda1(DataSet d)
```
    Guess the distribution to use for the regularization term λ₁ .
    
    Parameters:
    
    d - the data set to get the guess for
    
    Returns:
    
    the guess for the λ₁ parameter

Class LinearSGD

Constructor Summary

Method Summary

Methods inherited from class jsat.classifiers.BaseUpdateableClassifier

Methods inherited from class java.lang.Object

Constructor Detail

LinearSGD

LinearSGD

LinearSGD

LinearSGD

Method Detail

setGradientUpdater

getGradientUpdater

setEtaDecay

getEtaDecay

setEta

getEta

setLoss

getLoss

setLambda0

getLambda0

setLambda1

getLambda1

setUseBias

isUseBias

clone

setUp

setUp

update

update

classify

regress

supportsWeightedData

train

train

getParameters

getParameter

getRawWeight

getBias

numWeightsVecs

guessLambda0

guessLambda1