RegressionModelEvaluation (Java Statistical Analysis Tool 0.0.8 API)

java.lang.Object
- jsat.regression.RegressionModelEvaluation

```
public class RegressionModelEvaluation
extends Object
```
Provides a mechanism to quickly evaluate a regression model on a data set. This can be done by cross validation or with a separate testing set.

Author:

Edward Raff

Constructor Summary

Constructors
Constructor and Description
`RegressionModelEvaluation(Regressor regressor, RegressionDataSet dataSet)` Creates a new RegressionModelEvaluation that will perform serial training
`RegressionModelEvaluation(Regressor regressor, RegressionDataSet dataSet, ExecutorService threadpool)` Creates a new RegressionModelEvaluation that will perform parallel training.

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`void`	`addScorer(RegressionScore scorer)` Adds a new score object that will be used as part of the evaluation when calling `evaluateCrossValidation(int, java.util.Random)` or `evaluateTestSet(jsat.regression.RegressionDataSet)`.
`void`	`evaluateCrossValidation(int folds)` Performs an evaluation of the regressor using the training data set.
`void`	`evaluateCrossValidation(int folds, Random rand)` Performs an evaluation of the regressor using the training data set.
`void`	`evaluateCrossValidation(List<RegressionDataSet> lcds)` Performs an evaluation of the regressor using the training data set, where the folds of the training data set are provided by the user.
`void`	`evaluateCrossValidation(List<RegressionDataSet> lcds, List<RegressionDataSet> trainCombinations)` Note: Most people should never need to call this method.
`void`	`evaluateTestSet(RegressionDataSet testSet)` Performs an evaluation of the regressor using the initial data set to train, and testing on the given data set.
`double`	`getErrorStndDev()` Returns the standard deviation of the error from all runs
`Regressor[]`	`getKeptModels()` Returns the models that were kept after the last evaluation.
`double`	`getMaxError()` Returns the maximum squared error observed from all runs.
`double`	`getMeanError()` Returns the mean squared error from all runs.
`double`	`getMinError()` Returns the minimum squared error from all runs.
`Regressor`	`getRegressor()` Returns the regressor that was to be evaluated
`OnLineStatistics`	`getScoreStats(RegressionScore score)` Gets the statistics associated with the given score.
`long`	`getTotalClassificationTime()` Returns the total number of milliseconds spent performing regression on the testing set.
`long`	`getTotalTrainingTime()` Returns the total number of milliseconds spent training the regressor.
`boolean`	`isKeepModels()` This will keep the models trained when evaluating the model.
`void`	`prettyPrintRegressionScores()` Prints out the classification information in a convenient format.
`void`	`setDataTransformProcess(DataTransformProcess dtp)` Sets the data transform process to use when performing cross validation.
`void`	`setKeepModels(boolean keepModels)` Set this to `true` in order to keep the trained models after evaluation.
`void`	`setWarmModels(Regressor... warmModels)` Sets the models that will be used for warm starting training.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Constructor Detail
  - RegressionModelEvaluation
```
public RegressionModelEvaluation(Regressor regressor,
                                 RegressionDataSet dataSet,
                                 ExecutorService threadpool)
```
    Creates a new RegressionModelEvaluation that will perform parallel training.
    
    Parameters:
    
    regressor - the regressor model to evaluate
    
    dataSet - the data set to train or perform cross validation from
    
    threadpool - the source of threads for training of models
  - RegressionModelEvaluation
```
public RegressionModelEvaluation(Regressor regressor,
                                 RegressionDataSet dataSet)
```
    Creates a new RegressionModelEvaluation that will perform serial training
    
    Parameters:
    
    regressor - the regressor model to evaluate
    
    dataSet - the data set to train or perform cross validation from
- Method Detail
  - setKeepModels
```
public void setKeepModels(boolean keepModels)
```
    Set this to true in order to keep the trained models after evaluation. They can then be retrieved used the getKeptModels() methods. The default value is false.
    
    Parameters:
    
    keepModels - true to keep the trained models after evaluation, false to discard them.
  - isKeepModels
```
public boolean isKeepModels()
```
    This will keep the models trained when evaluating the model. The models can be obtained after an evaluation from getKeptModels().
    
    Returns:
    
    true if trained models will be kept after evaluation.
  - getKeptModels
```
public Regressor[] getKeptModels()
```
    Returns the models that were kept after the last evaluation. null will be returned instead if isKeepModels() returns false, which is the default.
    
    Returns:
    
    the models that were kept after the last evaluation. Or null if if models are not being kept.
  - setWarmModels
```
public void setWarmModels(Regressor... warmModels)
```
    Sets the models that will be used for warm starting training. If using cross-validation, the number of models given should match the number of folds. If using a test set, only one model should be given.
    
    Parameters:
    
    warmModels - the models to use for warm start training
  - setDataTransformProcess
```
public void setDataTransformProcess(DataTransformProcess dtp)
```
    Sets the data transform process to use when performing cross validation. By default, no transforms are applied
    
    Parameters:
    
    dtp - the transformation process to clone for use during evaluation
  - evaluateCrossValidation
```
public void evaluateCrossValidation(int folds)
```
    Performs an evaluation of the regressor using the training data set. The evaluation is done by performing cross validation.
    
    Parameters:
    
    folds - the number of folds for cross validation
    
    Throws:
    
    UntrainedModelException - if the number of folds given is less than 2
  - evaluateCrossValidation
```
public void evaluateCrossValidation(int folds,
                                    Random rand)
```
    Performs an evaluation of the regressor using the training data set. The evaluation is done by performing cross validation.
    
    Parameters:
    
    folds - the number of folds for cross validation
    
    rand - the source of randomness for generating the cross validation sets
    
    Throws:
    
    UntrainedModelException - if the number of folds given is less than 2
  - evaluateCrossValidation
```
public void evaluateCrossValidation(List<RegressionDataSet> lcds)
```
    Performs an evaluation of the regressor using the training data set, where the folds of the training data set are provided by the user. The folds do not need to be the same sizes, though it is assumed that they are all approximately the same size. It is the caller's responsibility to ensure that the folds are only from the original training data set.
    
    This method exists so that the user can provide very specific folds if they so desire. This can be useful when there is known bias in the data set, such as when caused by duplicate data point values. The caller can then manually make sure duplicate values all occur in the same fold to avoid over-estimating the accuracy of the model.
    
    Parameters:
    
    lcds - the training data set already split into folds
  - evaluateCrossValidation
```
public void evaluateCrossValidation(List<RegressionDataSet> lcds,
                                    List<RegressionDataSet> trainCombinations)
```
    Note: Most people should never need to call this method. Make sure you understand what you are doing before you do.
    
    Performs an evaluation of the regressor using the training data set, where the folds of the training data set, and their combinations, are provided by the user. The folds do not need to be the same sizes, though it is assumed that they are all approximately the same size - and the the training combination corresponding to each index will be the sum of the folds in the other indices. It is the caller's responsibility to ensure that the folds are only from the original training data set.
    
    This method exists so that the user can provide very specific folds if they so desire, and when the same folds will be used multiple times. Doing so allows the algorithms called to take advantage of any potential caching of results based on the data set and avoid all possible excessive memory movement. (For example, DataSet.getNumericColumns() may get re-used and benefit from its caching)
    The same behavior of this method can be obtained by calling evaluateCrossValidation(java.util.List).
    
    Parameters:
    
    lcds - training data set already split into folds
    
    trainCombinations - each index contains the training data sans the data stored in the fold associated with that index
  - evaluateTestSet
```
public void evaluateTestSet(RegressionDataSet testSet)
```
    Performs an evaluation of the regressor using the initial data set to train, and testing on the given data set.
    
    Parameters:
    
    testSet - the data set to perform testing on
  - addScorer
```
public void addScorer(RegressionScore scorer)
```
    Adds a new score object that will be used as part of the evaluation when calling evaluateCrossValidation(int, java.util.Random) or evaluateTestSet(jsat.regression.RegressionDataSet). The statistics for the given score are reset on every call, and the mean / standard deviation comes from multiple folds in cross validation.
    
    The score statistics can be obtained from getScoreStats(jsat.regression.evaluation.RegressionScore) after one of the evaluation methods have been called.
    
    Parameters:
    
    scorer - the score method to keep track of.
  - getScoreStats
```
public OnLineStatistics getScoreStats(RegressionScore score)
```
    Gets the statistics associated with the given score. If the score is not currently in the model evaluation null will be returned. The object passed in does not need to be the exact same object passed to addScorer(jsat.regression.evaluation.RegressionScore), it only needs to be equal to the object.
    
    Parameters:
    
    score - the score type to get the result statistics
    
    Returns:
    
    the result statistics for the given score, or null if the score is not in th evaluation set
  - prettyPrintRegressionScores
```
public void prettyPrintRegressionScores()
```
    Prints out the classification information in a convenient format. If no additional scores were added via the addScorer(RegressionScore) method, nothing will be printed.
  - getMinError
```
public double getMinError()
```
    Returns the minimum squared error from all runs.
    
    Returns:
    
    the minimum observed squared error
  - getMaxError
```
public double getMaxError()
```
    Returns the maximum squared error observed from all runs.
    
    Returns:
    
    the maximum observed squared error
  - getMeanError
```
public double getMeanError()
```
    Returns the mean squared error from all runs.
    
    Returns:
    
    the overall mean squared error
  - getErrorStndDev
```
public double getErrorStndDev()
```
    Returns the standard deviation of the error from all runs
    
    Returns:
    
    the overall standard deviation of the errors
  - getTotalTrainingTime
```
public long getTotalTrainingTime()
```
    Returns the total number of milliseconds spent training the regressor.
    
    Returns:
    
    the total number of milliseconds spent training the regressor.
  - getTotalClassificationTime
```
public long getTotalClassificationTime()
```
    Returns the total number of milliseconds spent performing regression on the testing set.
    
    Returns:
    
    the total number of milliseconds spent performing regression on the testing set.
  - getRegressor
```
public Regressor getRegressor()
```
    Returns the regressor that was to be evaluated
    
    Returns:
    
    the regressor original given

Class RegressionModelEvaluation

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Constructor Detail

RegressionModelEvaluation

RegressionModelEvaluation

Method Detail

setKeepModels

isKeepModels

getKeptModels

setWarmModels

setDataTransformProcess

evaluateCrossValidation

evaluateCrossValidation

evaluateCrossValidation

evaluateCrossValidation

evaluateTestSet

addScorer

getScoreStats

prettyPrintRegressionScores

getMinError

getMaxError

getMeanError

getErrorStndDev

getTotalTrainingTime

getTotalClassificationTime

getRegressor