jsat.classifiers.boosting

Class Bagging

java.lang.Object

jsat.classifiers.boosting.Bagging

All Implemented Interfaces:

Serializable, Cloneable, Classifier, Parameterized, Regressor

public class Bagging
extends Object
implements Classifier, Regressor, Parameterized

An implementation of Bootstrap Aggregating, as described by LEO BREIMAN in "Bagging Predictors".

Bagging is an ensemble learner, it takes a weak learner and trains several to create a better over result. Bagging is particularly useful when the base classifier has some amount of predictive power, but is hindered by variance in the output (small change in input causes large change in output), or variances in input (handles noisy data badly or is has a brittle learning algorithm). It is common to perform bagging on Decision Trees, because they meet these strengths and weaknesses.
Bagging produces little to no improvement when using learners that have low variance and robust learning methods. NearestNeighbour is an example of a particularly bad method to bag.
Bagging has many similarities to boosting.

Author:

Edward Raff

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
static int	DEFAULT_EXTRA_SAMPLES The number of extra samples to take when bagging in each round used by default in the constructor: 0
static int	DEFAULT_ROUNDS The number of rounds of bagging that will be used by default in the constructor: 20
static boolean	DEFAULT_SIMULTANIOUS_TRAINING The default behavior for parallel training, as specified by setSimultaniousTraining(boolean) is true

Constructor Summary

Constructors

Constructor and Description
Bagging(Classifier baseClassifier) Creates a new Bagger for classification.
Bagging(Classifier baseClassifier, int extraSamples, boolean simultaniousTraining) Creates a new Bagger for classification.
Bagging(Classifier baseClassifier, int extraSamples, boolean simultaniousTraining, int rounds, Random random) Creates a new Bagger for classification.
Bagging(Regressor baseRegressor) Creates a new Bagger for regression.
Bagging(Regressor baseRegressor, int extraSamples, boolean simultaniousTraining) Creates a new Bagger for regression.
Bagging(Regressor baseRegressor, int extraSamples, boolean simultaniousTraining, int rounds, Random random) Creates a new Bagger for regression.

Method Summary

Modifier and Type	Method and Description
CategoricalResults	classify(DataPoint data) Performs classification on the given data point.
Bagging	clone()
int	getExtraSamples()
Parameter	getParameter(String paramName) Returns the parameter with the given name.
List<Parameter>	getParameters() Returns the list of parameters that can be altered for this learner.
int	getRounds() Returns the number of rounds of boosting that will be done, which is also the number of base learners that will be trained
static ClassificationDataSet	getSampledDataSet(ClassificationDataSet dataSet, int[] sampledCounts) Creates a new data set from the given sample counts.
static RegressionDataSet	getSampledDataSet(RegressionDataSet dataSet, int[] sampledCounts) Creates a new data set from the given sample counts.
static ClassificationDataSet	getWeightSampledDataSet(ClassificationDataSet dataSet, int[] sampledCounts) Creates a new data set from the given sample counts.
static RegressionDataSet	getWeightSampledDataSet(RegressionDataSet dataSet, int[] sampledCounts) Creates a new data set from the given sample counts.
double	regress(DataPoint data)
static void	sampleWithReplacement(int[] sampleCounts, int samples, Random rand) Performs the sampling based on the number of data points, storing the counts in an array to be constructed from XXXX
void	setExtraSamples(int i) Bagging samples from the training set with replacement, and draws a sampleWithReplacement at least as large as the training set.
void	setRounds(int rounds) Sets the number of rounds that bagging is done, meaning how many base learners are trained
void	setSimultaniousTraining(boolean simultaniousTraining) Bagging produces multiple base learners.
boolean	supportsWeightedData() Indicates whether the model knows how to train using weighted data points.
void	train(RegressionDataSet dataSet)
void	train(RegressionDataSet dataSet, ExecutorService threadPool)
void	trainC(ClassificationDataSet dataSet) Trains the classifier and constructs a model for classification using the given data set.
void	trainC(ClassificationDataSet dataSet, ExecutorService threadPool) Trains the classifier and constructs a model for classification using the given data set.

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DEFAULT_ROUNDS

public static final int DEFAULT_ROUNDS

The number of rounds of bagging that will be used by default in the constructor: 20

See Also:

Constant Field Values

DEFAULT_EXTRA_SAMPLES

public static final int DEFAULT_EXTRA_SAMPLES

The number of extra samples to take when bagging in each round used by default in the constructor: 0

See Also:

Constant Field Values

DEFAULT_SIMULTANIOUS_TRAINING

public static final boolean DEFAULT_SIMULTANIOUS_TRAINING

The default behavior for parallel training, as specified by setSimultaniousTraining(boolean) is true

See Also:

Constant Field Values

Constructor Detail

Bagging

public Bagging(Classifier baseClassifier)

Creates a new Bagger for classification. This can not be changed after construction.

Parameters:

baseClassifier - the base learner to use.

Bagging

public Bagging(Classifier baseClassifier,
               int extraSamples,
               boolean simultaniousTraining)

Creates a new Bagger for classification. This can not be changed after construction.

Parameters:

baseClassifier - the base learner to use.

extraSamples - how many extra samples past the training size to take

simultaniousTraining - controls whether base learners are trained sequentially or simultaneously

Bagging

public Bagging(Classifier baseClassifier,
               int extraSamples,
               boolean simultaniousTraining,
               int rounds,
               Random random)

Creates a new Bagger for classification. This can not be changed after construction.

Parameters:

baseClassifier - the base learner to use.

extraSamples - how many extra samples past the training size to take

simultaniousTraining - controls whether base learners are trained sequentially or simultaneously

rounds - how many rounds of bagging to perform.

random - the source of randomness for sampling

Bagging

public Bagging(Regressor baseRegressor)

Creates a new Bagger for regression. This can not be changed after construction.

Parameters:

baseRegressor - the base learner to use.

Bagging

public Bagging(Regressor baseRegressor,
               int extraSamples,
               boolean simultaniousTraining)

Creates a new Bagger for regression. This can not be changed after construction.

Parameters:

baseRegressor - the base learner to use.

extraSamples - how many extra samples past the training size to take

simultaniousTraining - controls whether base learners are trained sequentially or simultaneously

Bagging

public Bagging(Regressor baseRegressor,
               int extraSamples,
               boolean simultaniousTraining,
               int rounds,
               Random random)

Creates a new Bagger for regression. This can not be changed after construction.

Parameters:

baseRegressor - the base learner to use.

extraSamples - how many extra samples past the training size to take

simultaniousTraining - controls whether base learners are trained sequentially or simultaneously

rounds - how many rounds of bagging to perform.

random - the source of randomness for sampling

Method Detail

setExtraSamples

public void setExtraSamples(int i)

Bagging samples from the training set with replacement, and draws a sampleWithReplacement at least as large as the training set. This controls how many extra samples are taken. If negative, fewer samples will be taken. Using negative values is not recommended.

Parameters:

i - how many extra samples to take

getExtraSamples

public int getExtraSamples()

setRounds

public void setRounds(int rounds)

Sets the number of rounds that bagging is done, meaning how many base learners are trained

Parameters:

rounds - the number of base learners to train

Throws:

ArithmeticException - if the number specified is not a positive value

getRounds

public int getRounds()

Returns the number of rounds of boosting that will be done, which is also the number of base learners that will be trained

Returns:

the number of rounds of boosting that will be done, which is also the number of base learners that will be trained

setSimultaniousTraining

public void setSimultaniousTraining(boolean simultaniousTraining)

Bagging produces multiple base learners. These can all be trained at the same time, using more memory, or sequentially using the base learner's parallel training method. If set to true, the base learners will be trained simultaneously.

Parameters:

simultaniousTraining - true to train all learners at the same time, false to train them sequentially

classify

public CategoricalResults classify(DataPoint data)

Description copied from interface: Classifier

Performs classification on the given data point.

Specified by:

classify in interface Classifier

Parameters:

data - the data point to classify

Returns:

the results of the classification.

trainC

public void trainC(ClassificationDataSet dataSet,
                   ExecutorService threadPool)

Description copied from interface: Classifier

Trains the classifier and constructs a model for classification using the given data set. If the training method knows how, it will used the threadPool to conduct training in parallel. This method will block until the training has completed.

Specified by:

trainC in interface Classifier

Parameters:

dataSet - the data set to train on

threadPool - the source of threads to use.

trainC

public void trainC(ClassificationDataSet dataSet)

Description copied from interface: Classifier

Trains the classifier and constructs a model for classification using the given data set.

Specified by:

trainC in interface Classifier

Parameters:

dataSet - the data set to train on

getSampledDataSet

public static ClassificationDataSet getSampledDataSet(ClassificationDataSet dataSet,
                                                      int[] sampledCounts)

Creates a new data set from the given sample counts. Points sampled multiple times will have multiple entries in the data set.

Parameters:

dataSet - the data set that was sampled from

sampledCounts - the sampling values obtained from sampleWithReplacement(int[], int, java.util.Random)

Returns:

a new sampled classification data set

getWeightSampledDataSet

public static ClassificationDataSet getWeightSampledDataSet(ClassificationDataSet dataSet,
                                                            int[] sampledCounts)

Creates a new data set from the given sample counts. Points sampled multiple times will be added once to the data set with their weight multiplied by the number of times it was sampled.

Parameters:

dataSet - the data set that was sampled from

sampledCounts - the sampling values obtained from sampleWithReplacement(int[], int, java.util.Random)

Returns:

a new sampled classification data set

getSampledDataSet

public static RegressionDataSet getSampledDataSet(RegressionDataSet dataSet,
                                                  int[] sampledCounts)

Creates a new data set from the given sample counts. Points sampled multiple times will have multiple entries in the data set.

Parameters:

dataSet - the data set that was sampled from

sampledCounts - the sampling values obtained from sampleWithReplacement(int[], int, java.util.Random)

Returns:

a new sampled classification data set

getWeightSampledDataSet

public static RegressionDataSet getWeightSampledDataSet(RegressionDataSet dataSet,
                                                        int[] sampledCounts)

Creates a new data set from the given sample counts. Points sampled multiple times will be added once to the data set with their weight multiplied by the number of times it was sampled.

Parameters:

dataSet - the data set that was sampled from

sampledCounts - the sampling values obtained from sampleWithReplacement(int[], int, java.util.Random)

Returns:

a new sampled classification data set

sampleWithReplacement

public static void sampleWithReplacement(int[] sampleCounts,
                                         int samples,
                                         Random rand)

Performs the sampling based on the number of data points, storing the counts in an array to be constructed from XXXX

Parameters:

sampleCounts - an array to keep count of how many times each data point was sampled. The array will be filled with zeros before sampling starts

samples - the number of samples to take from the data set

rand - the source of randomness

supportsWeightedData

public boolean supportsWeightedData()

Description copied from interface: Classifier

Indicates whether the model knows how to train using weighted data points. If it does, the model will train assuming the weights. The values returned by this method may change depending on the parameters set for the model.

Specified by:

supportsWeightedData in interface Classifier

Specified by:

supportsWeightedData in interface Regressor

Returns:

true if the model supports weighted data, false otherwise

regress

public double regress(DataPoint data)

Specified by:

regress in interface Regressor

train

public void train(RegressionDataSet dataSet,
                  ExecutorService threadPool)

Specified by:

train in interface Regressor

train

public void train(RegressionDataSet dataSet)

Specified by:

train in interface Regressor

clone

public Bagging clone()

Specified by:

clone in interface Classifier

Specified by:

clone in interface Regressor

Overrides:

clone in class Object

getParameters

public List<Parameter> getParameters()

Description copied from interface: Parameterized

Returns the list of parameters that can be altered for this learner.

Specified by:

getParameters in interface Parameterized

Returns:

the list of parameters that can be altered for this learner.

getParameter

public Parameter getParameter(String paramName)

Description copied from interface: Parameterized

Returns the parameter with the given name. Two different strings may map to a single Parameter object. An ASCII only string, and a Unicode style string.

Specified by:

getParameter in interface Parameterized

Parameters:

paramName - the name of the parameter to obtain

Returns:

the Parameter in question, or null if no such named Parameter exists.