jsat.classifiers.svm

Class PegasosK

java.lang.Object

jsat.classifiers.svm.SupportVectorLearner

jsat.classifiers.svm.PegasosK

All Implemented Interfaces:

Serializable, Cloneable, BinaryScoreClassifier, Classifier, Parameterized

public class PegasosK
extends SupportVectorLearner
implements BinaryScoreClassifier, Parameterized

Implements the kernelized version of the Pegasos algorithm for SVMs. Unlike standard SVM algorithms, this one relies on randomness and has no guarantee to reach the optimal solution, however it is very fast to train. Each iteration of the algorithm randomly selects one datapoint to potentially update the coefficient of.
The resulting set of support vectors may be more or less sparse than a normal SVM implementation.
Because the Pegasos algorithm is stochastic and the kernelized updates on errors given regularization, the kernelized version may have more difficulty with noisy or overlapping class distributions.

See: Shalev-Shwartz, S., Singer, Y.,&Srebro, N. (2007). Pegasos : Primal Estimated sub-GrAdient SOlver for SVM. 24th international conference on Machine learning (pp. 807–814). New York, NY: ACM. doi:10.1145/1273496.1273598

Author:

Edward Raff

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class jsat.classifiers.svm.SupportVectorLearner

SupportVectorLearner.CacheMode

Field Summary

Fields inherited from class jsat.classifiers.svm.SupportVectorLearner

accelCache, alphas, cacheEvictions, evalCount, vecs

Constructor Summary

Constructors

Constructor and Description
PegasosK(double regularization, int iterations, KernelTrick kernel) Creates a new kernelized Pegasos SVM solver
PegasosK(double regularization, int iterations, KernelTrick kernel, SupportVectorLearner.CacheMode cacheMode) Creates a new kernelized Pegasos SVM solver

Method Summary

Modifier and Type	Method and Description
CategoricalResults	classify(DataPoint data) Performs classification on the given data point.
PegasosK	clone()
int	getIterations() Returns the number of iterations used during training
Parameter	getParameter(String paramName) Returns the parameter with the given name.
List<Parameter>	getParameters() Returns the list of parameters that can be altered for this learner.
double	getRegularization() Returns the amount of regularization used
double	getScore(DataPoint dp) Returns the numeric score for predicting a class of a given data point, where the sign of the value indicates which class the data point is predicted to belong to.
void	setIterations(int iterations) Sets the number of iterations of the algorithm to perform.
void	setRegularization(double regularization) Sets the amount of regularization to apply.
boolean	supportsWeightedData() Indicates whether the model knows how to train using weighted data points.
void	trainC(ClassificationDataSet dataSet) Trains the classifier and constructs a model for classification using the given data set.
void	trainC(ClassificationDataSet dataSet, ExecutorService threadPool) Trains the classifier and constructs a model for classification using the given data set.

Methods inherited from class jsat.classifiers.svm.SupportVectorLearner

accessingRow, getCacheMode, getCacheValue, getKernel, k, kEval, kEval, kEvalSum, setAlphas, setCacheMode, setCacheSize, setCacheValue, setKernel, sparsify

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

PegasosK

public PegasosK(double regularization,
                int iterations,
                KernelTrick kernel)

Creates a new kernelized Pegasos SVM solver

Parameters:

regularization - the amount of regularization to apply, normally a very small positive value

iterations - the number of update iterations to perform

kernel - the kernel to use

PegasosK

public PegasosK(double regularization,
                int iterations,
                KernelTrick kernel,
                SupportVectorLearner.CacheMode cacheMode)

Creates a new kernelized Pegasos SVM solver

Parameters:

regularization - the amount of regularization to apply, normally a very small positive value

iterations - the number of update iterations to perform

kernel - the kernel to use

cacheMode - what type of kernel caching to use

Method Detail

setIterations

public void setIterations(int iterations)

Sets the number of iterations of the algorithm to perform. Each iteration may or may not update a single coefficient for a specific data point.

Parameters:

iterations - the number of learning iterations to perform

getIterations

public int getIterations()

Returns the number of iterations used during training

Returns:

the number of iterations used in training

setRegularization

public void setRegularization(double regularization)

Sets the amount of regularization to apply. The regularization must be a positive value

Parameters:

regularization - the amount of regularization to apply

getRegularization

public double getRegularization()

Returns the amount of regularization used

Returns:

the amount of regularization used

clone

public PegasosK clone()

Specified by:

clone in interface BinaryScoreClassifier

Specified by:

clone in interface Classifier

Overrides:

clone in class Object

classify

public CategoricalResults classify(DataPoint data)

Description copied from interface: Classifier

Performs classification on the given data point.

Specified by:

classify in interface Classifier

Parameters:

data - the data point to classify

Returns:

the results of the classification.

getScore

public double getScore(DataPoint dp)

Description copied from interface: BinaryScoreClassifier

Returns the numeric score for predicting a class of a given data point, where the sign of the value indicates which class the data point is predicted to belong to.

Specified by:

getScore in interface BinaryScoreClassifier

Parameters:

dp - the data point to predict the class label of

Returns:

the score for the given data point

trainC

public void trainC(ClassificationDataSet dataSet,
                   ExecutorService threadPool)

Description copied from interface: Classifier

Trains the classifier and constructs a model for classification using the given data set. If the training method knows how, it will used the threadPool to conduct training in parallel. This method will block until the training has completed.

Specified by:

trainC in interface Classifier

Parameters:

dataSet - the data set to train on

threadPool - the source of threads to use.

trainC

public void trainC(ClassificationDataSet dataSet)

Description copied from interface: Classifier

Trains the classifier and constructs a model for classification using the given data set.

Specified by:

trainC in interface Classifier

Parameters:

dataSet - the data set to train on

supportsWeightedData

public boolean supportsWeightedData()

Description copied from interface: Classifier

Indicates whether the model knows how to train using weighted data points. If it does, the model will train assuming the weights. The values returned by this method may change depending on the parameters set for the model.

Specified by:

supportsWeightedData in interface Classifier

Returns:

true if the model supports weighted data, false otherwise

getParameters

public List<Parameter> getParameters()

Description copied from interface: Parameterized

Returns the list of parameters that can be altered for this learner.

Specified by:

getParameters in interface Parameterized

Returns:

the list of parameters that can be altered for this learner.

getParameter

public Parameter getParameter(String paramName)

Description copied from interface: Parameterized

Returns the parameter with the given name. Two different strings may map to a single Parameter object. An ASCII only string, and a Unicode style string.

Specified by:

getParameter in interface Parameterized

Parameters:

paramName - the name of the parameter to obtain

Returns:

the Parameter in question, or null if no such named Parameter exists.