jsat.classifiers.neuralnetwork.activations

Class SoftSignLayer

java.lang.Object

jsat.classifiers.neuralnetwork.activations.SoftSignLayer

All Implemented Interfaces:

Serializable, ActivationLayer

public class SoftSignLayer
extends Object
implements ActivationLayer

This provides the Soft Sign activation function f(x) = x/(1+abs(x)), which is similar to the tanh activation and has a min/max of -1 and 1. However it is significantly faster to compute.

See: Glorot, X., & Bengio, Y. (2010). Understanding the difficulty of training deep feedforward neural networks. Journal of Machine Learning Research - Proceedings Track, 9, 249–256. Retrieved from here

Author:

Edward Raff

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
SoftSignLayer()

Method Summary

Modifier and Type	Method and Description
void	activate(Matrix input, Matrix output, boolean rowMajor) Computes the activation function of this layer on the given input.
void	activate(Vec input, Vec output) Computes the activation function of this layer on the given input.
void	backprop(Matrix input, Matrix output, Matrix delta_partial, Matrix errout, boolean rowMajor) This method computes the backpropagated error to a given layer.
void	backprop(Vec input, Vec output, Vec delta_partial, Vec errout) This method computes the backpropagated error to a given layer.
SoftSignLayer	clone()

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SoftSignLayer

public SoftSignLayer()

Method Detail

activate

public void activate(Vec input,
                     Vec output)

Description copied from interface: ActivationLayer

Computes the activation function of this layer on the given input.

Specified by:

activate in interface ActivationLayer

Parameters:

input - the raw input to compute the activation for

output - the location to store the activation in

activate

public void activate(Matrix input,
                     Matrix output,
                     boolean rowMajor)

Description copied from interface: ActivationLayer

Computes the activation function of this layer on the given input.

Specified by:

activate in interface ActivationLayer

Parameters:

input - the raw input to compute the activation for

output - the location to store the activation in

rowMajor - true if the information per input is stored in rows, false if the inputs were stored by column. This parameter does not indicate if the matrices themselves are backed by a row or column major implementation

backprop

public void backprop(Vec input,
                     Vec output,
                     Vec delta_partial,
                     Vec errout)

Description copied from interface: ActivationLayer

This method computes the backpropagated error to a given layer. Often denoted as δ^l = w^{l+1 T} δ^l+1 ⊗ ∂ f(x^l), where ∂ is the Hadamard product and ∂ f(x^l) is the derivative of this activation function on the input that was feed into this activation.
delta_partial and errout may point to the same vector object

Specified by:

backprop in interface ActivationLayer

Parameters:

input - the input to this layer that was feed in to be activated

output - the activation that was produced for this layer

delta_partial - the error assigned to this layer from the above layer, sans the hamard product with the derivative of the layer activation. Often denoted as w^{l+1 T} δ^l+1

errout - the delta value or error produced for this layer

backprop

public void backprop(Matrix input,
                     Matrix output,
                     Matrix delta_partial,
                     Matrix errout,
                     boolean rowMajor)

Description copied from interface: ActivationLayer

This method computes the backpropagated error to a given layer. Often denoted as δ^l = w^{l+1 T} δ^l+1 ⊗ ∂ f(x^l), where ∂ is the Hadamard product and ∂ f(x^l) is the derivative of this activation function on the input that was feed into this activation.
delta_partial and errout may point to the same vector object

Specified by:

backprop in interface ActivationLayer

Parameters:

input - the input to this layer that was feed in to be activated

output - the activation that was produced for this layer

delta_partial - the error assigned to this layer from the above layer, sans the hamard product with the derivative of the layer activation. Often denoted as w^{l+1 T} δ^l+1

errout - the delta value or error produced for this layer

rowMajor - true if the information per input is stored in rows, false if the inputs were stored by column. This parameter does not indicate if the matrices themselves are backed by a row or column major implementation

clone

public SoftSignLayer clone()

Specified by:

clone in interface ActivationLayer

Overrides:

clone in class Object