jsat.text

Class HashedTextVectorCreator

java.lang.Object

jsat.text.HashedTextVectorCreator

All Implemented Interfaces:

Serializable, TextVectorCreator

public class HashedTextVectorCreator
extends Object
implements TextVectorCreator

Hashed Text Vector Creator exists to convert a text string into a Vec using feature hashing. The tokenization and word weighting method must be provided and already set up. When constructed the user should make sure the WordWeighting.setWeight(java.util.List, java.util.List) method has already been called, or is a stateless weighting (such as BinaryWordPresent).

Author:

Edward Raff

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
HashedTextVectorCreator(int dimensionSize, Tokenizer tokenizer, WordWeighting weighting) Creates a new text vector creator that works with hash-trick features

Method Summary

Modifier and Type	Method and Description
Vec	newText(String input) Converts the given input text into a vector representation.
Vec	newText(String input, StringBuilder workSpace, List<String> storageSpace) Converts the given input text into a vector representation

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HashedTextVectorCreator

public HashedTextVectorCreator(int dimensionSize,
                               Tokenizer tokenizer,
                               WordWeighting weighting)

Creates a new text vector creator that works with hash-trick features

Parameters:

dimensionSize - the dimension size of the feature space

tokenizer - the tokenizer to apply to incoming strings

weighting - the weighting process to apply to each loaded document.

Method Detail

newText

public Vec newText(String input)

Description copied from interface: TextVectorCreator

Converts the given input text into a vector representation.

Specified by:

newText in interface TextVectorCreator

Parameters:

input - the input string

Returns:

a vector representation

newText

public Vec newText(String input,
                   StringBuilder workSpace,
                   List<String> storageSpace)

Description copied from interface: TextVectorCreator

Converts the given input text into a vector representation

Specified by:

newText in interface TextVectorCreator

Parameters:

input - the input string

workSpace - an already allocated (but empty) string builder than can be used as a temporary work space.

storageSpace - an already allocated (but empty) list to place the tokens into

Returns:

a vector representation