jsat.text.wordweighting

Class OkapiBM25

java.lang.Object

jsat.math.IndexFunction

jsat.text.wordweighting.WordWeighting

jsat.text.wordweighting.OkapiBM25

All Implemented Interfaces:

Serializable, Function

public class OkapiBM25
extends WordWeighting

Implements the Okapi BM25 word weighting scheme.

Author:

EdwardRaff

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
OkapiBM25() Creates a new Okapi object
OkapiBM25(double k1, double b) Creates a new Okapi object

Method Summary

Modifier and Type	Method and Description
void	applyTo(Vec vec) The implementation may want to pre compute come values based on the vector it is about to be applied to.
double	indexFunc(double value, int index) An index function, meant to be applied to vectors where the value to be computed may vary based on the position in the vector of the value.
void	setWeight(List<? extends Vec> allDocuments, List<Integer> df) Prepares the word weighting to be performed on a data set.

Methods inherited from class jsat.math.IndexFunction

f, f

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

OkapiBM25

public OkapiBM25()

Creates a new Okapi object

OkapiBM25

public OkapiBM25(double k1,
                 double b)

Creates a new Okapi object

Parameters:

k1 - the non negative coefficient to apply to the term frequency

b - the coefficient to apply to the document length in the range [0,1]

Method Detail

setWeight

public void setWeight(List<? extends Vec> allDocuments,
                      List<Integer> df)

Description copied from class: WordWeighting

Prepares the word weighting to be performed on a data set. This should be called once before being applied to any vectors. Different WordWeightings may require different amounts of computation to set up.

Specified by:

setWeight in class WordWeighting

Parameters:

allDocuments - the list of all vectors that make up the set of documents. The word vectors should be unmodified, containing the value of how many times a word appeared in the document for each index.

df - a list mapping each integer index of a word to how many times that word occurred in total

applyTo

public void applyTo(Vec vec)

Description copied from class: WordWeighting

The implementation may want to pre compute come values based on the vector it is about to be applied to. This should be called in place of Vec.applyIndexFunction(jsat.math.IndexFunction) . The vector should be in a bag-of-words form where each index value indicates how many times the word for that index occurred in the document represented by the vector.

Specified by:

applyTo in class WordWeighting

Parameters:

vec - the vector to set up for and then alter by invoking Vec.applyIndexFunction(jsat.math.IndexFunction) on

indexFunc

public double indexFunc(double value,
                        int index)

Description copied from class: IndexFunction

An index function, meant to be applied to vectors where the value to be computed may vary based on the position in the vector of the value.

Specified by:

indexFunc in class IndexFunction

Parameters:

value - the value at the specified index

index - the index the value is from

Returns:

the computed result. If a negative index was given, the function should return 0.0 if indexFunc(0,indx) would return zero for all valid indices. If this is not the case, any non zero value should be returned.