jsat.classifiers.calibration

Class BinaryCalibration

java.lang.Object

jsat.classifiers.calibration.BinaryCalibration

All Implemented Interfaces:

Serializable, Cloneable, Classifier, Parameterized

Direct Known Subclasses:

IsotonicCalibration, PlattCalibration

public abstract class BinaryCalibration
extends Object
implements Classifier, Parameterized

This abstract class provides the frame work for an algorithm to perform probability calibration based on the outputs of a base learning algorithm for binary classification problems.

Calibration can be performed directly on output values, though it may cause over-fitting. For this reason, the BinaryCalibration.CalibrationMode may be set to an alternative method.

The parameters include the calibration parameters, and any parameters that would be returned by the base model.

Author:

Edward Raff

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	BinaryCalibration.CalibrationMode Controls how the scores are obtained for producing a "training set" to calibrate the output of the underlying model.

Field Summary

Fields

Modifier and Type	Field and Description
protected BinaryScoreClassifier	base The base classifier to train and calibrate the outputs of
protected int	folds The number of CV folds
protected double	holdOut The proportion of the data set to hold out for calibration
protected BinaryCalibration.CalibrationMode	mode The calibration mode to use

Constructor Summary

Constructors

Constructor and Description
BinaryCalibration(BinaryScoreClassifier base, BinaryCalibration.CalibrationMode mode) Creates a new Binary Calibration object

Method Summary

Modifier and Type	Method and Description
protected abstract void	calibrate(boolean[] label, double[] scores, int len) This method perform the model calibration on the outputs verse the class labels.
abstract BinaryCalibration	clone()
int	getCalibrationFolds() Returns the number of cross validation folds to use
double	getCalibrationHoldOut() Returns the portion of the data set that will be held out for calibration
BinaryCalibration.CalibrationMode	getCalibrationMode() Returns the calibration mode used during training
Parameter	getParameter(String paramName) Returns the parameter with the given name.
List<Parameter>	getParameters() Returns the list of parameters that can be altered for this learner.
void	setCalibrationFolds(int folds) If the calibration mode is set to BinaryCalibration.CalibrationMode.CV, this controls how many folds of cross validation will be used.
void	setCalibrationHoldOut(double holdOut) If the calibration mode is set to BinaryCalibration.CalibrationMode.HOLD_OUT, this what portion of the data set is randomly selected to be the hold out set.
void	setCalibrationMode(BinaryCalibration.CalibrationMode mode) Sets which calibration mode will be used during training
void	trainC(ClassificationDataSet dataSet) Trains the classifier and constructs a model for classification using the given data set.
void	trainC(ClassificationDataSet dataSet, ExecutorService threadPool) Trains the classifier and constructs a model for classification using the given data set.

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface jsat.classifiers.Classifier

classify, supportsWeightedData

Field Detail

base

protected BinaryScoreClassifier base

The base classifier to train and calibrate the outputs of

folds

protected int folds

The number of CV folds

holdOut

protected double holdOut

The proportion of the data set to hold out for calibration

mode

protected BinaryCalibration.CalibrationMode mode

The calibration mode to use

Constructor Detail

BinaryCalibration

public BinaryCalibration(BinaryScoreClassifier base,
                         BinaryCalibration.CalibrationMode mode)

Creates a new Binary Calibration object

Parameters:

base - the base learning algorithm

mode - the calibration mode to use

Method Detail

trainC

public void trainC(ClassificationDataSet dataSet,
                   ExecutorService threadPool)

Description copied from interface: Classifier

Trains the classifier and constructs a model for classification using the given data set. If the training method knows how, it will used the threadPool to conduct training in parallel. This method will block until the training has completed.

Specified by:

trainC in interface Classifier

Parameters:

dataSet - the data set to train on

threadPool - the source of threads to use.

trainC

public void trainC(ClassificationDataSet dataSet)

Description copied from interface: Classifier

Trains the classifier and constructs a model for classification using the given data set.

Specified by:

trainC in interface Classifier

Parameters:

dataSet - the data set to train on

calibrate

protected abstract void calibrate(boolean[] label,
                                  double[] scores,
                                  int len)

This method perform the model calibration on the outputs verse the class labels.

Parameters:

label - the set of labels, where true indicates the positive class label, and false indicates the negative class label.

scores - the score associated with each label from the learning algorithm.

len - the number of values (from zero) of the label and scores array to use. This value may be less than the actual array size

setCalibrationFolds

public void setCalibrationFolds(int folds)

If the calibration mode is set to BinaryCalibration.CalibrationMode.CV, this controls how many folds of cross validation will be used. The default is 3.

Parameters:

folds - the number of cross validation folds to perform

getCalibrationFolds

public int getCalibrationFolds()

Returns the number of cross validation folds to use

Returns:

the number of cross validation folds to use

setCalibrationHoldOut

public void setCalibrationHoldOut(double holdOut)

If the calibration mode is set to BinaryCalibration.CalibrationMode.HOLD_OUT, this what portion of the data set is randomly selected to be the hold out set. The default is 0.3.

Parameters:

holdOut - the portion in (0, 1) to hold out

getCalibrationHoldOut

public double getCalibrationHoldOut()

Returns the portion of the data set that will be held out for calibration

Returns:

the portion of the data set that will be held out for calibration

setCalibrationMode

public void setCalibrationMode(BinaryCalibration.CalibrationMode mode)

Sets which calibration mode will be used during training

Parameters:

mode - the calibration mode to use during training.

getCalibrationMode

public BinaryCalibration.CalibrationMode getCalibrationMode()

Returns the calibration mode used during training

Returns:

the calibration mode used during training

clone

public abstract BinaryCalibration clone()

Specified by:

clone in interface Classifier

Overrides:

clone in class Object

getParameters

public List<Parameter> getParameters()

Description copied from interface: Parameterized

Returns the list of parameters that can be altered for this learner.

Specified by:

getParameters in interface Parameterized

Returns:

the list of parameters that can be altered for this learner.

getParameter

public Parameter getParameter(String paramName)

Description copied from interface: Parameterized

Returns the parameter with the given name. Two different strings may map to a single Parameter object. An ASCII only string, and a Unicode style string.

Specified by:

getParameter in interface Parameterized

Parameters:

paramName - the name of the parameter to obtain

Returns:

the Parameter in question, or null if no such named Parameter exists.