jsat.datatransform.featureselection

Class SFS

java.lang.Object

jsat.datatransform.featureselection.SFS

All Implemented Interfaces:

Serializable, Cloneable, DataTransform

public class SFS
extends Object
implements DataTransform

Sequential Forward Selection (SFS) is a greedy method of selecting a subset of features to use for prediction. It starts from the set of no features and attempts to add the next best feature to the set at each iteration.

Author:

Edward Raff

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
SFS(int minFeatures, int maxFeatures, ClassificationDataSet dataSet, Classifier evaluater, int folds, double maxIncrease) Performs SFS feature selection for a classification problem
SFS(int minFeatures, int maxFeatures, Classifier evaluater, double maxIncrease) Performs SFS feature selection for a classification problem
SFS(int minFeatures, int maxFeatures, RegressionDataSet dataSet, Regressor regressor, int folds, double maxIncrease) Performs SFS feature selection for a regression problem
SFS(int minFeatures, int maxFeatures, Regressor regressor, double maxIncrease) Creates SFS feature selection for a regression problem

Method Summary

Modifier and Type	Method and Description
protected static void	addFeature(int curBest, int nCat, Set<Integer> catF, Set<Integer> numF)
SFS	clone()
void	fit(DataSet data) Fits this transform to the given dataset.
int	getFolds()
int	getMaxFeatures() Returns the maximum number of features to find
double	getMaxIncrease()
int	getMinFeatures() Returns the minimum number of features to find
protected static double	getScore(DataSet workOn, Object evaluater, int folds, Random rand) The score function for a data set and a learner by cross validation of a classifier
Set<Integer>	getSelectedCategorical() Returns a copy of the set of categorical features selected by the search algorithm
Set<Integer>	getSelectedNumerical() Returns a copy of the set of numerical features selected by the search algorithm.
protected static void	removeFeature(int feature, int nCat, Set<Integer> catF, Set<Integer> numF)
void	setFolds(int folds) Sets the number of folds to use for cross validation when estimating the error rate
void	setMaxFeatures(int maxFeatures) Sets the maximum number of features that must be selected
void	setMaxIncrease(double maxIncrease) Sets the maximum allowable the maximum tolerable increase in error when a feature is added
void	setMinFeatures(int minFeatures) Sets the minimum number of features that must be selected
protected static int	SFSSelectFeature(Set<Integer> available, DataSet dataSet, Set<Integer> catToRemove, Set<Integer> numToRemove, Set<Integer> catSelecteed, Set<Integer> numSelected, Object evaluater, int folds, Random rand, double[] PbestScore, int minFeatures) Attempts to add one feature to the list of features while increasing or maintaining the current accuracy
DataPoint	transform(DataPoint dp) Returns a new data point that is a transformation of the original data point.

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SFS

public SFS(int minFeatures,
           int maxFeatures,
           Classifier evaluater,
           double maxIncrease)

Performs SFS feature selection for a classification problem

Parameters:

minFeatures - the minimum number of features to find

maxFeatures - the maximum number of features to find

evaluater - the classifier to use in determining accuracy given a feature subset

maxIncrease - the maximum tolerable increase in error when a feature is added

SFS

public SFS(int minFeatures,
           int maxFeatures,
           ClassificationDataSet dataSet,
           Classifier evaluater,
           int folds,
           double maxIncrease)

Performs SFS feature selection for a classification problem

Parameters:

minFeatures - the minimum number of features to find

maxFeatures - the maximum number of features to find

dataSet - the data set to perform feature selection on

evaluater - the classifier to use in determining accuracy given a feature subset

folds - the number of cross validation folds to use in selection

maxIncrease - the maximum tolerable increase in error when a feature is added

SFS

public SFS(int minFeatures,
           int maxFeatures,
           Regressor regressor,
           double maxIncrease)

Creates SFS feature selection for a regression problem

Parameters:

minFeatures - the minimum number of features to find

maxFeatures - the maximum number of features to find

regressor - the regressor to use in determining accuracy given a feature subset

maxIncrease - the maximum tolerable increase in error when a feature is added

SFS

public SFS(int minFeatures,
           int maxFeatures,
           RegressionDataSet dataSet,
           Regressor regressor,
           int folds,
           double maxIncrease)

Performs SFS feature selection for a regression problem

Parameters:

minFeatures - the minimum number of features to find

maxFeatures - the maximum number of features to find

dataSet - the data set to perform feature selection on

regressor - the regressor to use in determining accuracy given a feature subset

folds - the number of cross validation folds to use in selection

maxIncrease - the maximum tolerable increase in error when a feature is added

Method Detail

fit

public void fit(DataSet data)

Description copied from interface: DataTransform

Fits this transform to the given dataset. Some transforms can only be learned from classification or regression datasets. If an incompatible dataset type is given, a FailedToFitException exception may be thrown.

Specified by:

fit in interface DataTransform

Parameters:

data - the dataset to fir this transform to

addFeature

protected static void addFeature(int curBest,
                                 int nCat,
                                 Set<Integer> catF,
                                 Set<Integer> numF)

Parameters:

curBest - the value of curBest

nCat - the value of nCat

catF - the value of catF

numF - the value of numF

removeFeature

protected static void removeFeature(int feature,
                                    int nCat,
                                    Set<Integer> catF,
                                    Set<Integer> numF)

Parameters:

feature - the value of feature

nCat - the value of nCat

catF - the value of catF

numF - the value of numF

transform

public DataPoint transform(DataPoint dp)

Description copied from interface: DataTransform

Returns a new data point that is a transformation of the original data point. This new data point is a different object, but may contain the same references as the original data point. It is not guaranteed that you can mutate the transformed point without having a side effect on the original point.

Specified by:

transform in interface DataTransform

Parameters:

dp - the data point to apply a transformation to

Returns:

a transformed data point

clone

public SFS clone()

Specified by:

clone in interface DataTransform

Overrides:

clone in class Object

getSelectedCategorical

public Set<Integer> getSelectedCategorical()

Returns a copy of the set of categorical features selected by the search algorithm

Returns:

the set of categorical features to use

getSelectedNumerical

public Set<Integer> getSelectedNumerical()

Returns a copy of the set of numerical features selected by the search algorithm.

Returns:

the set of numeric features to use

SFSSelectFeature

protected static int SFSSelectFeature(Set<Integer> available,
                                      DataSet dataSet,
                                      Set<Integer> catToRemove,
                                      Set<Integer> numToRemove,
                                      Set<Integer> catSelecteed,
                                      Set<Integer> numSelected,
                                      Object evaluater,
                                      int folds,
                                      Random rand,
                                      double[] PbestScore,
                                      int minFeatures)

Attempts to add one feature to the list of features while increasing or maintaining the current accuracy

Parameters:

available - the set of available features from [0, n) to consider for adding

dataSet - the original data set to perform feature selection from

catToRemove - the current set of categorical features to remove

numToRemove - the current set of numerical features to remove

catSelecteed - the current set of categorical features we are keeping

numSelected - the current set of numerical features we are keeping

evaluater - the classifier or regressor to perform evaluations with

folds - the number of cross validation folds to determine performance

rand - the source of randomness

PbestScore - an array to behave as a pointer to the best score seen so far

minFeatures - the minimum number of features needed

Returns:

the feature that was selected to add, or -1 if none were added.

getScore

protected static double getScore(DataSet workOn,
                                 Object evaluater,
                                 int folds,
                                 Random rand)

The score function for a data set and a learner by cross validation of a classifier

Parameters:

workOn - the transformed data set to test from with cross validation

evaluater - the learning algorithm to use

folds - the number of cross validation folds to perform

rand - the source of randomness

Returns:

the score value in terms of cross validated error

setMaxIncrease

public void setMaxIncrease(double maxIncrease)

Sets the maximum allowable the maximum tolerable increase in error when a feature is added

Parameters:

maxIncrease - the maximum allowable the maximum tolerable increase in error when a feature is added

getMaxIncrease

public double getMaxIncrease()

Returns:

the maximum allowable the maximum tolerable increase in error when a feature is added

setMinFeatures

public void setMinFeatures(int minFeatures)

Sets the minimum number of features that must be selected

Parameters:

minFeatures - the minimum number of features to learn

getMinFeatures

public int getMinFeatures()

Returns the minimum number of features to find

Returns:

the minimum number of features to find

setMaxFeatures

public void setMaxFeatures(int maxFeatures)

Sets the maximum number of features that must be selected

Parameters:

maxFeatures - the maximum number of features to find

getMaxFeatures

public int getMaxFeatures()

Returns the maximum number of features to find

Returns:

the maximum number of features to find

setFolds

public void setFolds(int folds)

Sets the number of folds to use for cross validation when estimating the error rate

Parameters:

folds - the number of folds to use for cross validation when estimating the error rate

getFolds

public int getFolds()

Returns:

the number of folds to use for cross validation when estimating the error rate