jsat.io

Class CSV

java.lang.Object

jsat.io.CSV

public class CSV
extends Object

Provides a reader and writer for CSV style datasets. This CSV reader supports comments in CSVs (must begin with a single character) and categorical features (columns must be specified when calling). Any number of newlines will be treated as a single newline separating two rows.

When reading and writing a CSV, if the delimiter or comment markers are not specified - the defaults will be used DEFAULT_DELIMITER and DEFAULT_COMMENT respectively.

The CSV loader will treat empty columns as missing values for both numeric and categorical features. A value of "NaN" in a numeric column will also be treated as a missing value. Once loaded, missing values for numeric features are encoded as Double.NaN and as -1 for categorical features.

Author:

Edward Raff

Field Summary

Fields

Modifier and Type	Field and Description
static char	DEFAULT_COMMENT
static char	DEFAULT_DELIMITER

Method Summary

Modifier and Type	Method and Description
static SimpleDataSet	read(Path path, char delimiter, int lines_to_skip, char comment, Set<Integer> cat_cols) Reads in the given CSV dataset as a simple CSV file
static SimpleDataSet	read(Path path, int lines_to_skip, Set<Integer> cat_cols) Reads in the given CSV dataset as a simple CSV file
static SimpleDataSet	read(Reader reader, char delimiter, int lines_to_skip, char comment, Set<Integer> cat_cols) Reads in the given CSV dataset as a simple CSV file
static SimpleDataSet	read(Reader reader, int lines_to_skip, Set<Integer> cat_cols) Reads in the given CSV dataset as a simple CSV file
static ClassificationDataSet	readC(int classification_target, Path path, char delimiter, int lines_to_skip, char comment, Set<Integer> cat_cols) Reads in a CSV dataset as a classification dataset.
static ClassificationDataSet	readC(int classification_target, Path path, int lines_to_skip, Set<Integer> cat_cols) Reads in a CSV dataset as a classification dataset.
static ClassificationDataSet	readC(int classification_target, Reader reader, char delimiter, int lines_to_skip, char comment, Set<Integer> cat_cols) Reads in a CSV dataset as a classification dataset.
static ClassificationDataSet	readC(int classification_target, Reader reader, int lines_to_skip, Set<Integer> cat_cols) Reads in a CSV dataset as a classification dataset.
static RegressionDataSet	readR(int numeric_target_column, Path path, char delimiter, int lines_to_skip, char comment, Set<Integer> cat_cols) Reads in a CSV dataset as a regression dataset.
static RegressionDataSet	readR(int numeric_target_column, Path path, int lines_to_skip, Set<Integer> cat_cols) Reads in a CSV dataset as a regression dataset.
static RegressionDataSet	readR(int numeric_target_column, Reader reader, char delimiter, int lines_to_skip, char comment, Set<Integer> cat_cols) Reads in a CSV dataset as a regression dataset.
static RegressionDataSet	readR(int numeric_target_column, Reader reader, int lines_to_skip, Set<Integer> cat_cols) Reads in a CSV dataset as a regression dataset.
static void	write(DataSet<?> data, Path path) Writes out the given dataset as a CSV file.
static void	write(DataSet<?> data, Path path, char delimiter) Writes out the given dataset as a CSV file.
static void	write(DataSet<?> data, Writer writer) Writes out the given dataset as a CSV file.
static void	write(DataSet<?> data, Writer writer, char delimiter) Writes out the given dataset as a CSV file.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DEFAULT_DELIMITER

public static final char DEFAULT_DELIMITER

See Also:

Constant Field Values

DEFAULT_COMMENT

public static final char DEFAULT_COMMENT

See Also:

Constant Field Values

Method Detail

readR

public static RegressionDataSet readR(int numeric_target_column,
                                      Path path,
                                      int lines_to_skip,
                                      Set<Integer> cat_cols)
                               throws IOException

Reads in a CSV dataset as a regression dataset.

Parameters:

numeric_target_column - the column index (starting from zero) of the feature that will be the target regression value

path - the reader for the CSV content

lines_to_skip - the number of lines to skip when reading in the CSV (used to skip header information)

cat_cols - a set of the indices to treat as categorical features.

Returns:

the regression dataset from the given CSV file

Throws:

IOException

readR

public static RegressionDataSet readR(int numeric_target_column,
                                      Reader reader,
                                      int lines_to_skip,
                                      Set<Integer> cat_cols)
                               throws IOException

Reads in a CSV dataset as a regression dataset.

Parameters:

numeric_target_column - the column index (starting from zero) of the feature that will be the target regression value

reader - the reader for the CSV content

lines_to_skip - the number of lines to skip when reading in the CSV (used to skip header information)

cat_cols - a set of the indices to treat as categorical features.

Returns:

the regression dataset from the given CSV file

Throws:

IOException

readR

public static RegressionDataSet readR(int numeric_target_column,
                                      Path path,
                                      char delimiter,
                                      int lines_to_skip,
                                      char comment,
                                      Set<Integer> cat_cols)
                               throws IOException

Reads in a CSV dataset as a regression dataset.

Parameters:

numeric_target_column - the column index (starting from zero) of the feature that will be the target regression value

path - the CSV file to read

delimiter - the delimiter to separate columns, usually a comma

lines_to_skip - the number of lines to skip when reading in the CSV (used to skip header information)

comment - the character used to indicate the start of a comment. Once this character is reached, anything at and after the character will be ignored.

cat_cols - a set of the indices to treat as categorical features.

Returns:

the regression dataset from the given CSV file

Throws:

IOException

readR

public static RegressionDataSet readR(int numeric_target_column,
                                      Reader reader,
                                      char delimiter,
                                      int lines_to_skip,
                                      char comment,
                                      Set<Integer> cat_cols)
                               throws IOException

Reads in a CSV dataset as a regression dataset.

Parameters:

numeric_target_column - the column index (starting from zero) of the feature that will be the target regression value

reader - the reader for the CSV content

delimiter - the delimiter to separate columns, usually a comma

lines_to_skip - the number of lines to skip when reading in the CSV (used to skip header information)

comment - the character used to indicate the start of a comment. Once this character is reached, anything at and after the character will be ignored.

cat_cols - a set of the indices to treat as categorical features.

Returns:

the regression dataset from the given CSV file

Throws:

IOException

readC

public static ClassificationDataSet readC(int classification_target,
                                          Path path,
                                          int lines_to_skip,
                                          Set<Integer> cat_cols)
                                   throws IOException

Reads in a CSV dataset as a classification dataset. Comments assumed to start with the "#" symbol.

Parameters:

classification_target - the column index (starting from zero) of the feature that will be the categorical target value

path - the CSV file to read

lines_to_skip - the number of lines to skip when reading in the CSV (used to skip header information)

cat_cols - a set of the indices to treat as categorical features.

Returns:

the classification dataset from the given CSV file

Throws:

IOException

readC

public static ClassificationDataSet readC(int classification_target,
                                          Reader reader,
                                          int lines_to_skip,
                                          Set<Integer> cat_cols)
                                   throws IOException

Reads in a CSV dataset as a classification dataset. Comments assumed to start with the "#" symbol.

Parameters:

classification_target - the column index (starting from zero) of the feature that will be the categorical target value

reader - the reader for the CSV content

lines_to_skip - the number of lines to skip when reading in the CSV (used to skip header information)

cat_cols - a set of the indices to treat as categorical features.

Returns:

the classification dataset from the given CSV file

Throws:

IOException

readC

public static ClassificationDataSet readC(int classification_target,
                                          Reader reader,
                                          char delimiter,
                                          int lines_to_skip,
                                          char comment,
                                          Set<Integer> cat_cols)
                                   throws IOException

Reads in a CSV dataset as a classification dataset.

Parameters:

classification_target - the column index (starting from zero) of the feature that will be the categorical target value

reader - the reader for the CSV content

delimiter - the delimiter to separate columns, usually a comma

lines_to_skip - the number of lines to skip when reading in the CSV (used to skip header information)

comment - the character used to indicate the start of a comment. Once this character is reached, anything at and after the character will be ignored.

cat_cols - a set of the indices to treat as categorical features.

Returns:

the classification dataset from the given CSV file

Throws:

IOException

readC

public static ClassificationDataSet readC(int classification_target,
                                          Path path,
                                          char delimiter,
                                          int lines_to_skip,
                                          char comment,
                                          Set<Integer> cat_cols)
                                   throws IOException

Reads in a CSV dataset as a classification dataset.

Parameters:

classification_target - the column index (starting from zero) of the feature that will be the categorical target value

path - the CSV file

delimiter - the delimiter to separate columns, usually a comma

lines_to_skip - the number of lines to skip when reading in the CSV (used to skip header information)

comment - the character used to indicate the start of a comment. Once this character is reached, anything at and after the character will be ignored.

cat_cols - a set of the indices to treat as categorical features.

Returns:

the classification dataset from the given CSV file

Throws:

IOException

read

public static SimpleDataSet read(Path path,
                                 int lines_to_skip,
                                 Set<Integer> cat_cols)
                          throws IOException

Reads in the given CSV dataset as a simple CSV file

Parameters:

path - the CSV file

lines_to_skip - the number of lines to skip when reading in the CSV (used to skip header information)

cat_cols - a set of the indices to treat as categorical features.

Returns:

a simple dataset of the given CSV file

Throws:

IOException

read

public static SimpleDataSet read(Reader reader,
                                 int lines_to_skip,
                                 Set<Integer> cat_cols)
                          throws IOException

Reads in the given CSV dataset as a simple CSV file

Parameters:

reader - the reader for the CSV content

lines_to_skip - the number of lines to skip when reading in the CSV (used to skip header information)

cat_cols - a set of the indices to treat as categorical features.

Returns:

a simple dataset of the given CSV file

Throws:

IOException

read

public static SimpleDataSet read(Path path,
                                 char delimiter,
                                 int lines_to_skip,
                                 char comment,
                                 Set<Integer> cat_cols)
                          throws IOException

Reads in the given CSV dataset as a simple CSV file

Parameters:

path - the CSV file to read

delimiter - the delimiter to separate columns, usually a comma

lines_to_skip - the number of lines to skip when reading in the CSV (used to skip header information)

comment - the character used to indicate the start of a comment. Once this character is reached, anything at and after the character will be ignored.

cat_cols - a set of the indices to treat as categorical features.

Returns:

a simple dataset of the given CSV file

Throws:

IOException

read

public static SimpleDataSet read(Reader reader,
                                 char delimiter,
                                 int lines_to_skip,
                                 char comment,
                                 Set<Integer> cat_cols)
                          throws IOException

Reads in the given CSV dataset as a simple CSV file

Parameters:

reader - the reader for the CSV content

delimiter - the delimiter to separate columns, usually a comma

lines_to_skip - the number of lines to skip when reading in the CSV (used to skip header information)

comment - the character used to indicate the start of a comment. Once this character is reached, anything at and after the character will be ignored.

cat_cols - a set of the indices to treat as categorical features.

Returns:

a simple dataset of the given CSV file

Throws:

IOException

write

public static void write(DataSet<?> data,
                         Path path)
                  throws IOException

Writes out the given dataset as a CSV file. If the given dataset is a regression or classification dataset, the target feature that is being predicted will always be written out as the first index in the CSV.
After that, all numeric features will be written out in order, followed by the categorical features.

Parameters:

data - the dataset object to save as a CSV file

path - the path to write the CSV to

Throws:

IOException

write

public static void write(DataSet<?> data,
                         Writer writer)
                  throws IOException

Writes out the given dataset as a CSV file. If the given dataset is a regression or classification dataset, the target feature that is being predicted will always be written out as the first index in the CSV.
After that, all numeric features will be written out in order, followed by the categorical features.

Parameters:

data - the dataset object to save as a CSV file

writer - the output writer to write the CSV to

Throws:

IOException

write

public static void write(DataSet<?> data,
                         Path path,
                         char delimiter)
                  throws IOException

Writes out the given dataset as a CSV file. If the given dataset is a regression or classification dataset, the target feature that is being predicted will always be written out as the first index in the CSV.
After that, all numeric features will be written out in order, followed by the categorical features.

Parameters:

data - the dataset object to save as a CSV file

path - the path to write the CSV to

delimiter - the delimiter between column values, normally a comma

Throws:

IOException

write

public static void write(DataSet<?> data,
                         Writer writer,
                         char delimiter)
                  throws IOException

Writes out the given dataset as a CSV file. If the given dataset is a regression or classification dataset, the target feature that is being predicted will always be written out as the first index in the CSV.
After that, all numeric features will be written out in order, followed by the categorical features.

Parameters:

data - the dataset object to save as a CSV file

writer - the output writer to write the CSV to

delimiter - the delimiter between column values, normally a comma

Throws:

IOException