jsat.datatransform.featureselection

Class SBS

java.lang.Object

jsat.datatransform.RemoveAttributeTransform

jsat.datatransform.featureselection.SBS

All Implemented Interfaces:

Serializable, Cloneable, DataTransform

public class SBS
extends RemoveAttributeTransform

Sequential Backward Selection (SBS) is a greedy method of selecting a subset of features to use for prediction. It starts from the set of all features and attempts to remove the least informative feature from the set at each iteration

Author:

Edward Raff

See Also:

Serialized Form

Field Summary

Fields inherited from class jsat.datatransform.RemoveAttributeTransform

catIndexMap, numIndexMap

Constructor Summary

Constructors

Constructor and Description
SBS(int minFeatures, int maxFeatures, ClassificationDataSet cds, Classifier evaluater, int folds, double maxDecrease) Performs SBS feature selection for a classification problem
SBS(int minFeatures, int maxFeatures, Classifier evaluater, double maxDecrease) Performs SBS feature selection for a classification problem
SBS(int minFeatures, int maxFeatures, RegressionDataSet rds, Regressor evaluater, int folds, double maxDecrease) Performs SBS feature selection for a regression problem
SBS(int minFeatures, int maxFeatures, Regressor evaluater, double maxDecrease) Performs SBS feature selection for a regression problem

Method Summary

Modifier and Type	Method and Description
SBS	clone()
void	fit(DataSet data) Fits this transform to the given dataset.
int	getFolds()
double	getMaxDecrease() Returns the maximum allowable decrease in accuracy from one set of features to the next
int	getMaxFeatures() Returns the maximum number of features to find
int	getMinFeatures() Returns the minimum number of features to find
Set<Integer>	getSelectedCategorical() Returns a copy of the set of categorical features selected by the search algorithm
Set<Integer>	getSelectedNumerical() Returns a copy of the set of numerical features selected by the search algorithm.
protected static int	SBSRemoveFeature(Set<Integer> available, DataSet dataSet, Set<Integer> catToRemove, Set<Integer> numToRemove, Set<Integer> catSelecteed, Set<Integer> numSelected, Object evaluater, int folds, Random rand, int maxFeatures, double[] PbestScore, double maxDecrease) Attempts to remove one feature from the list while maintaining its accuracy
void	setFolds(int folds) Sets the number of folds to use for cross validation when estimating the error rate
void	setMaxDecrease(double maxDecrease) Sets the maximum allowable decrease in accuracy (increase in error) from the previous set of features to the new current set.
void	setMaxFeatures(int maxFeatures) Sets the maximum number of features that must be selected
void	setMinFeatures(int minFeatures) Sets the minimum number of features that must be selected

Methods inherited from class jsat.datatransform.RemoveAttributeTransform

consolidate, getKeptNominal, getKeptNumeric, getReverseNominalMap, getReverseNumericMap, setUp, transform

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SBS

public SBS(int minFeatures,
           int maxFeatures,
           Classifier evaluater,
           double maxDecrease)

Performs SBS feature selection for a classification problem

Parameters:

minFeatures - the minimum number of features to find

maxFeatures - the maximum number of features to find

evaluater - the classifier to use in determining accuracy given a feature subset

folds - the number of cross validation folds to use in selection

maxDecrease - the maximum tolerable decrease in accuracy in accuracy when a feature is removed

SBS

public SBS(int minFeatures,
           int maxFeatures,
           ClassificationDataSet cds,
           Classifier evaluater,
           int folds,
           double maxDecrease)

Performs SBS feature selection for a classification problem

Parameters:

minFeatures - the minimum number of features to find

maxFeatures - the maximum number of features to find

cds - the data set to perform feature selection on

evaluater - the classifier to use in determining accuracy given a feature subset

folds - the number of cross validation folds to use in selection

maxDecrease - the maximum tolerable decrease in accuracy in accuracy when a feature is removed

SBS

public SBS(int minFeatures,
           int maxFeatures,
           Regressor evaluater,
           double maxDecrease)

Performs SBS feature selection for a regression problem

Parameters:

minFeatures - the minimum number of features to find

maxFeatures - the maximum number of features to find

evaluater - the regressor to use in determining accuracy given a feature subset

maxDecrease - the maximum tolerable increase in the error rate when a feature is removed

SBS

public SBS(int minFeatures,
           int maxFeatures,
           RegressionDataSet rds,
           Regressor evaluater,
           int folds,
           double maxDecrease)

Performs SBS feature selection for a regression problem

Parameters:

minFeatures - the minimum number of features to find

maxFeatures - the maximum number of features to find

rds - the data set to perform feature selection on

evaluater - the regressor to use in determining accuracy given a feature subset

folds - the number of cross validation folds to use in selection

maxDecrease - the maximum tolerable increase in the error rate when a feature is removed

Method Detail

fit

public void fit(DataSet data)

Description copied from interface: DataTransform

Fits this transform to the given dataset. Some transforms can only be learned from classification or regression datasets. If an incompatible dataset type is given, a FailedToFitException exception may be thrown.

Specified by:

fit in interface DataTransform

Overrides:

fit in class RemoveAttributeTransform

Parameters:

data - the dataset to fir this transform to

clone

public SBS clone()

Specified by:

clone in interface DataTransform

Overrides:

clone in class RemoveAttributeTransform

getSelectedCategorical

public Set<Integer> getSelectedCategorical()

Returns a copy of the set of categorical features selected by the search algorithm

Returns:

the set of categorical features to use

getSelectedNumerical

public Set<Integer> getSelectedNumerical()

Returns a copy of the set of numerical features selected by the search algorithm.

Returns:

the set of numeric features to use

SBSRemoveFeature

protected static int SBSRemoveFeature(Set<Integer> available,
                                      DataSet dataSet,
                                      Set<Integer> catToRemove,
                                      Set<Integer> numToRemove,
                                      Set<Integer> catSelecteed,
                                      Set<Integer> numSelected,
                                      Object evaluater,
                                      int folds,
                                      Random rand,
                                      int maxFeatures,
                                      double[] PbestScore,
                                      double maxDecrease)

Attempts to remove one feature from the list while maintaining its accuracy

Parameters:

available - the set of available features from [0, n) to consider for removal

dataSet - the original data set to perform feature selection from

catToRemove - the current set of categorical features to remove

numToRemove - the current set of numerical features to remove

catSelecteed - the current set of categorical features we are keeping

numSelected - the current set of numerical features we are keeping

evaluater - the classifier or regressor to perform evaluations with

folds - the number of cross validation folds to determine performance

rand - the source of randomness

maxFeatures - the maximum allowable number of features

PbestScore - an array to behave as a pointer to the best score seen so far

maxDecrease - the maximum allowable decrease in accuracy from the best accuracy we see

Returns:

the feature that was selected to be removed, or -1 if none were removed

setMaxDecrease

public void setMaxDecrease(double maxDecrease)

Sets the maximum allowable decrease in accuracy (increase in error) from the previous set of features to the new current set.

Parameters:

maxDecrease - the maximum allowable decrease in the accuracy from removing a feature

getMaxDecrease

public double getMaxDecrease()

Returns the maximum allowable decrease in accuracy from one set of features to the next

Returns:

the maximum allowable decrease in accuracy from one set of features to the next

setMinFeatures

public void setMinFeatures(int minFeatures)

Sets the minimum number of features that must be selected

Parameters:

minFeatures - the minimum number of features to learn

getMinFeatures

public int getMinFeatures()

Returns the minimum number of features to find

Returns:

the minimum number of features to find

setMaxFeatures

public void setMaxFeatures(int maxFeatures)

Sets the maximum number of features that must be selected

Parameters:

maxFeatures - the maximum number of features to find

getMaxFeatures

public int getMaxFeatures()

Returns the maximum number of features to find

Returns:

the maximum number of features to find

setFolds

public void setFolds(int folds)

Sets the number of folds to use for cross validation when estimating the error rate

Parameters:

folds - the number of folds to use for cross validation when estimating the error rate

getFolds

public int getFolds()

Returns:

the number of folds to use for cross validation when estimating the error rate