jsat.classifiers.boosting

Class Stacking

java.lang.Object

jsat.classifiers.boosting.Stacking

All Implemented Interfaces:

Serializable, Cloneable, Classifier, Regressor

public class Stacking
extends Object
implements Classifier, Regressor

This provides an implementation of the Stacking ensemble method. Stacking learns several base classifiers and a top level classifier learns to predict the target based on the outputs of all the ensambled models. Historically a linear model (such as LinearBatch) is used, which translates to learning a weighted vote of the classifier outputs. However any classifier may be used so long as it supports the desired target type.

Note, that Stacking tends to work best when the base classifiers produce reasonable probability estimates.
Stacking supports weighted data instances if the aggregating model does.
See: Wolpert, D. H. (1992). Stacked generalization. Neural Networks, 5, 241–259.

Author:

Edward Raff

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
static int	DEFAULT_FOLDS

Constructor Summary

Constructors

Constructor and Description
Stacking(Classifier aggregatingClassifier, Classifier... baseClassifiers) Creates a new Stacking classifier that uses 3 folds of cross validation
Stacking(Classifier aggregatingClassifier, List<Classifier> baseClassifiers) Creates a new Stacking classifier that uses 3 folds of cross validation
Stacking(int folds, Classifier aggregatingClassifier, Classifier... baseClassifiers) Creates a new Stacking classifier
Stacking(int folds, Classifier aggregatingClassifier, List<Classifier> baseClassifiers) Creates a new Stacking classifier
Stacking(int folds, Regressor aggregatingRegressor, List<Regressor> baseRegressors) Creates a new Stacking regressor
Stacking(int folds, Regressor aggregatingRegressor, Regressor... baseRegressors) Creates a new Stacking regressor
Stacking(Regressor aggregatingRegressor, List<Regressor> baseRegressors) Creates a new Stacking regressor that uses 3 folds of cross validation
Stacking(Regressor aggregatingRegressor, Regressor... baseRegressors) Creates a new Stacking regressor that uses 3 folds of cross validation
Stacking(Stacking toCopy) Copy constructor

Method Summary

Modifier and Type	Method and Description
CategoricalResults	classify(DataPoint data) Performs classification on the given data point.
Stacking	clone()
int	getFolds()
double	regress(DataPoint data)
void	setFolds(int folds) Sets the number of folds of cross validation to use when creating the new set of weights that will be feed into the aggregating model.
boolean	supportsWeightedData() Indicates whether the model knows how to train using weighted data points.
void	train(RegressionDataSet dataSet)
void	train(RegressionDataSet dataSet, ExecutorService threadPool)
void	trainC(ClassificationDataSet dataSet) Trains the classifier and constructs a model for classification using the given data set.
void	trainC(ClassificationDataSet dataSet, ExecutorService threadPool) Trains the classifier and constructs a model for classification using the given data set.

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DEFAULT_FOLDS

public static final int DEFAULT_FOLDS

See Also:

Constant Field Values

Constructor Detail

Stacking

public Stacking(int folds,
                Classifier aggregatingClassifier,
                List<Classifier> baseClassifiers)

Creates a new Stacking classifier

Parameters:

folds - the number of cross validation folds for learning the base model

aggregatingClassifier - the classifier used to merge the results of all the input classifiers

baseClassifiers - the list of base classifiers to ensemble

Stacking

public Stacking(int folds,
                Classifier aggregatingClassifier,
                Classifier... baseClassifiers)

Creates a new Stacking classifier

Parameters:

folds - the number of cross validation folds for learning the base model

aggregatingClassifier - the classifier used to merge the results of all the input classifiers

baseClassifiers - the array of base classifiers to ensemble

Stacking

public Stacking(Classifier aggregatingClassifier,
                List<Classifier> baseClassifiers)

Creates a new Stacking classifier that uses 3 folds of cross validation

Parameters:

aggregatingClassifier - the classifier used to merge the results of all the input classifiers

baseClassifiers - the list of base classifiers to ensemble

Stacking

public Stacking(Classifier aggregatingClassifier,
                Classifier... baseClassifiers)

Creates a new Stacking classifier that uses 3 folds of cross validation

Parameters:

aggregatingClassifier - the classifier used to merge the results of all the input classifiers

baseClassifiers - the array of base classifiers to ensemble

Stacking

public Stacking(int folds,
                Regressor aggregatingRegressor,
                List<Regressor> baseRegressors)

Creates a new Stacking regressor

Parameters:

folds - the number of cross validation folds for learning the base model

aggregatingRegressor - the regressor used to merge the results of all the input classifiers

baseRegressors - the list of base regressors to ensemble

Stacking

public Stacking(int folds,
                Regressor aggregatingRegressor,
                Regressor... baseRegressors)

Creates a new Stacking regressor

Parameters:

folds - the number of cross validation folds for learning the base model

aggregatingRegressor - the regressor used to merge the results of all the input classifiers

baseRegressors - the array of base regressors to ensemble

Stacking

public Stacking(Regressor aggregatingRegressor,
                List<Regressor> baseRegressors)

Creates a new Stacking regressor that uses 3 folds of cross validation

Parameters:

aggregatingRegressor - the regressor used to merge the results of all the input classifiers

baseRegressors - the list of base regressors to ensemble

Stacking

public Stacking(Regressor aggregatingRegressor,
                Regressor... baseRegressors)

Creates a new Stacking regressor that uses 3 folds of cross validation

Parameters:

aggregatingRegressor - the regressor used to merge the results of all the input classifiers

baseRegressors - the array of base regressors to ensemble

Stacking

public Stacking(Stacking toCopy)

Copy constructor

Parameters:

toCopy - the object to copy

Method Detail

setFolds

public void setFolds(int folds)

Sets the number of folds of cross validation to use when creating the new set of weights that will be feed into the aggregating model.
Note that the number of folds may be 1, and will run significantly faster since models do not need to be re-trained. However it will be more prone to overfitting.

Parameters:

folds - the number of cross validation folds to use

getFolds

public int getFolds()

Returns:

the number of CV folds used for training

classify

public CategoricalResults classify(DataPoint data)

Description copied from interface: Classifier

Performs classification on the given data point.

Specified by:

classify in interface Classifier

Parameters:

data - the data point to classify

Returns:

the results of the classification.

trainC

public void trainC(ClassificationDataSet dataSet,
                   ExecutorService threadPool)

Description copied from interface: Classifier

Trains the classifier and constructs a model for classification using the given data set. If the training method knows how, it will used the threadPool to conduct training in parallel. This method will block until the training has completed.

Specified by:

trainC in interface Classifier

Parameters:

dataSet - the data set to train on

threadPool - the source of threads to use.

trainC

public void trainC(ClassificationDataSet dataSet)

Description copied from interface: Classifier

Trains the classifier and constructs a model for classification using the given data set.

Specified by:

trainC in interface Classifier

Parameters:

dataSet - the data set to train on

supportsWeightedData

public boolean supportsWeightedData()

Description copied from interface: Classifier

Indicates whether the model knows how to train using weighted data points. If it does, the model will train assuming the weights. The values returned by this method may change depending on the parameters set for the model.

Specified by:

supportsWeightedData in interface Classifier

Specified by:

supportsWeightedData in interface Regressor

Returns:

true if the model supports weighted data, false otherwise

regress

public double regress(DataPoint data)

Specified by:

regress in interface Regressor

train

public void train(RegressionDataSet dataSet,
                  ExecutorService threadPool)

Specified by:

train in interface Regressor

train

public void train(RegressionDataSet dataSet)

Specified by:

train in interface Regressor

clone

public Stacking clone()

Specified by:

clone in interface Classifier

Specified by:

clone in interface Regressor

Overrides:

clone in class Object