smile.io

Class Parquet

java.lang.Object

smile.io.Parquet

public class Parquet
extends java.lang.Object

Apache Parquet is a columnar storage format that supports nested data structures. It uses the record shredding and assembly algorithm described in the Dremel paper.

Method Summary

Modifier and Type	Method and Description
static smile.data.DataFrame	read(org.apache.parquet.io.InputFile file) Reads a parquet file.
static smile.data.DataFrame	read(org.apache.parquet.io.InputFile file, int limit) Reads a limited number of records from a parquet file.
static smile.data.DataFrame	read(java.nio.file.Path path) Reads a local parquet file.
static smile.data.DataFrame	read(java.nio.file.Path path, int limit) Reads a local parquet file.
static smile.data.DataFrame	read(java.lang.String path) Reads a HDFS parquet file.
static smile.data.DataFrame	read(java.lang.String path, int limit) Reads a HDFS parquet file.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

read

public static smile.data.DataFrame read(java.nio.file.Path path)
                                 throws java.io.IOException

Reads a local parquet file.

Parameters:

path - an Apache Parquet file path.

Throws:

java.io.IOException

read

public static smile.data.DataFrame read(java.nio.file.Path path,
                                        int limit)
                                 throws java.io.IOException

Reads a local parquet file.

Parameters:

path - an Apache Parquet file path.

limit - reads a limited number of records.

Throws:

java.io.IOException

read

public static smile.data.DataFrame read(java.lang.String path)
                                 throws java.io.IOException,
                                        java.net.URISyntaxException

Reads a HDFS parquet file.

Parameters:

path - an Apache Parquet file path.

Throws:

java.io.IOException

java.net.URISyntaxException

read

public static smile.data.DataFrame read(java.lang.String path,
                                        int limit)
                                 throws java.io.IOException,
                                        java.net.URISyntaxException

Reads a HDFS parquet file.

Parameters:

path - an Apache Parquet file path.

limit - reads a limited number of records.

Throws:

java.io.IOException

java.net.URISyntaxException

read

public static smile.data.DataFrame read(org.apache.parquet.io.InputFile file)
                                 throws java.io.IOException

Reads a parquet file.

Parameters:

file - an interface with the methods needed by Parquet to read data files. See HadoopInputFile for example.

Throws:

java.io.IOException

read

public static smile.data.DataFrame read(org.apache.parquet.io.InputFile file,
                                        int limit)
                                 throws java.io.IOException

Reads a limited number of records from a parquet file.

Parameters:

file - an interface with the methods needed by Parquet to read data files. See HadoopInputFile for example.

limit - reads a limited number of records.

Throws:

java.io.IOException