StochasticGradientBoosting (Java Statistical Analysis Tool 0.0.8 API)

java.lang.Object
- jsat.regression.StochasticGradientBoosting

All Implemented Interfaces:

Serializable, Cloneable, Parameterized, Regressor
```
public class StochasticGradientBoosting
extends Object
implements Regressor, Parameterized
```
An implementation of Stochastic Gradient Boosting (SGB) for the Squared Error loss. SGB is also known as Gradient Boosting Machine. There is a specialized version of SGB known as TreeBoost, that is not implemented by this method. SGB is a boosting method derived for regression. It uses many weak learners by attempting to estimate the residual error of all previous learners. It can also use an initial strong learner and use the weak learners to refine the initial estimate.

See papers:
Friedman, J. H. (2002). Stochastic gradient boosting. Computational Statistics&Data Analysis, 38(4), 367–378.

Mohan, A., Chen, Z.,&Weinberger, K. (2011). Web-search ranking with initialized gradient boosted regression trees. Journal of Machine Learning Research, 14,

Author:

Edward Raff

See Also:

Serialized Form

Field Summary

Fields
Modifier and Type	Field and Description
`static double`	`DEFAULT_LEARNING_RATE` The default value for the `setLearningRate(double)` is 0.1
`static double`	`DEFAULT_TRAINING_PROPORTION` The default value for the `training proportion` is 0.5.

Constructor Summary

Constructors
Constructor and Description
`StochasticGradientBoosting(Regressor weakLearner, int maxIterations)` Creates a new SGB learner that is initialized using the weak learner.
`StochasticGradientBoosting(Regressor weakLearner, int maxIterations, double learningRate)` Creates a new SGB learner that is initialized using the weak learner.
`StochasticGradientBoosting(Regressor weakLearner, int maxIterations, double learningRate, double trainingPortion)` Creates a new SGB learner that is initialized using the weak learner.
`StochasticGradientBoosting(Regressor strongLearner, Regressor weakLearner, int maxIterations, double learningRate, double trainingPortion)` Creates a new initialized SGB learner.

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`StochasticGradientBoosting`	`clone()`
`double`	`getLearningRate()` Returns the learning rate of the algorithm used to control overfitting.
`int`	`getMaxIterations()` Returns the maximum number of iterations used in SGB
`Parameter`	`getParameter(String paramName)` Returns the parameter with the given name.
`List<Parameter>`	`getParameters()` Returns the list of parameters that can be altered for this learner.
`double`	`getTrainingProportion()` Returns the fraction of the data points used during each iteration of the training algorithm.
`double`	`regress(DataPoint data)`
`void`	`setLearningRate(double learningRate)` Sets the learning rate of the algorithm.
`void`	`setMaxIterations(int maxIterations)` Sets the maximum number of iterations used in SGB.
`void`	`setTrainingProportion(double trainingProportion)` The GB version uses the whole data set at each iteration.
`boolean`	`supportsWeightedData()`
`void`	`train(RegressionDataSet dataSet)`
`void`	`train(RegressionDataSet dataSet, ExecutorService threadPool)`

Methods inherited from class java.lang.Object
equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - DEFAULT_TRAINING_PROPORTION
```
public static final double DEFAULT_TRAINING_PROPORTION
```
    The default value for the training proportion is 0.5.
    
    See Also:
    
    Constant Field Values
  - DEFAULT_LEARNING_RATE
```
public static final double DEFAULT_LEARNING_RATE
```
    The default value for the setLearningRate(double) is 0.1
    
    See Also:
    
    Constant Field Values
- Constructor Detail
  - StochasticGradientBoosting
```
public StochasticGradientBoosting(Regressor strongLearner,
                                  Regressor weakLearner,
                                  int maxIterations,
                                  double learningRate,
                                  double trainingPortion)
```
    Creates a new initialized SGB learner.
    
    Parameters:
    
    strongLearner - the powerful learner to refine with weak learners
    
    weakLearner - the weak learner to fit to the residuals in each iteration
    
    maxIterations - the maximum number of algorithm iterations to perform
    
    learningRate - the multiplier to apply to the weak learners
    
    trainingPortion - the proportion of the data set to use for each iteration of learning
  - StochasticGradientBoosting
```
public StochasticGradientBoosting(Regressor weakLearner,
                                  int maxIterations,
                                  double learningRate,
                                  double trainingPortion)
```
    Creates a new SGB learner that is initialized using the weak learner.
    
    Parameters:
    
    weakLearner - the weak learner to fit to the residuals in each iteration
    
    maxIterations - the maximum number of algorithm iterations to perform
    
    learningRate - the multiplier to apply to the weak learners
    
    trainingPortion - the proportion of the data set to use for each iteration of learning
  - StochasticGradientBoosting
```
public StochasticGradientBoosting(Regressor weakLearner,
                                  int maxIterations,
                                  double learningRate)
```
    Creates a new SGB learner that is initialized using the weak learner.
    
    Parameters:
    
    weakLearner - the weak learner to fit to the residuals in each iteration
    
    maxIterations - the maximum number of algorithm iterations to perform
    
    learningRate - the multiplier to apply to the weak learners
  - StochasticGradientBoosting
```
public StochasticGradientBoosting(Regressor weakLearner,
                                  int maxIterations)
```
    Creates a new SGB learner that is initialized using the weak learner.
    
    Parameters:
    
    weakLearner - the weak learner to fit to the residuals in each iteration
    
    maxIterations - the maximum number of algorithm iterations to perform
- Method Detail
  - setMaxIterations
```
public void setMaxIterations(int maxIterations)
```
    Sets the maximum number of iterations used in SGB.
    
    Parameters:
    
    maxIterations - the maximum number of algorithm iterations to perform
  - getMaxIterations
```
public int getMaxIterations()
```
    Returns the maximum number of iterations used in SGB
    
    Returns:
    
    the maximum number of algorithm iterations to perform
  - setLearningRate
```
public void setLearningRate(double learningRate)
```
    Sets the learning rate of the algorithm. The GB version uses a learning rate of 1. SGB uses a learning rate in (0,1) to avoid overfitting. The learning rate is multiplied by the output of each weak learner to reduce its contribution.
    
    Parameters:
    
    learningRate - the multiplier to apply to the weak learners
    
    Throws:
    
    ArithmeticException - if the learning rate is not in the range (0, 1]
  - getLearningRate
```
public double getLearningRate()
```
    Returns the learning rate of the algorithm used to control overfitting.
    
    Returns:
    
    the learning rate multiplier applied to the weak learner outputs
  - setTrainingProportion
```
public void setTrainingProportion(double trainingProportion)
```
    The GB version uses the whole data set at each iteration. SGB can use a fraction of the data set at each iteration in order to reduce overfitting and add randomness.
    
    Parameters:
    
    trainingProportion - the fraction of training the data set to use for each iteration of SGB
    
    Throws:
    
    ArithmeticException - if the trainingPortion is not a valid fraction in (0, 1]
  - getTrainingProportion
```
public double getTrainingProportion()
```
    Returns the fraction of the data points used during each iteration of the training algorithm.
    
    Returns:
    
    the fraction of the training data set to use for each iteration of SGB
  - regress
```
public double regress(DataPoint data)
```
    Specified by:
    
    regress in interface Regressor
  - train
```
public void train(RegressionDataSet dataSet,
                  ExecutorService threadPool)
```
    Specified by:
    
    train in interface Regressor
  - train
```
public void train(RegressionDataSet dataSet)
```
    Specified by:
    
    train in interface Regressor
  - supportsWeightedData
```
public boolean supportsWeightedData()
```
    Specified by:
    
    supportsWeightedData in interface Regressor
  - clone
```
public StochasticGradientBoosting clone()
```
    Specified by:
    
    clone in interface Regressor
    
    Overrides:
    
    clone in class Object
  - getParameters
```
public List<Parameter> getParameters()
```
    Description copied from interface: Parameterized
    
    Returns the list of parameters that can be altered for this learner.
    
    Specified by:
    
    getParameters in interface Parameterized
    
    Returns:
    
    the list of parameters that can be altered for this learner.
  - getParameter
```
public Parameter getParameter(String paramName)
```
    Description copied from interface: Parameterized
    
    Returns the parameter with the given name. Two different strings may map to a single Parameter object. An ASCII only string, and a Unicode style string.
    
    Specified by:
    
    getParameter in interface Parameterized
    
    Parameters:
    
    paramName - the name of the parameter to obtain
    
    Returns:
    
    the Parameter in question, or null if no such named Parameter exists.

Class StochasticGradientBoosting

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

DEFAULT_TRAINING_PROPORTION

DEFAULT_LEARNING_RATE

Constructor Detail

StochasticGradientBoosting

StochasticGradientBoosting

StochasticGradientBoosting

StochasticGradientBoosting

Method Detail

setMaxIterations

getMaxIterations

setLearningRate

getLearningRate

setTrainingProportion

getTrainingProportion

regress

train

train

supportsWeightedData

clone

getParameters

getParameter