org.deeplearning4j.api.loader.impl

Class RecordReaderFileBatchLoader

java.lang.Object

org.deeplearning4j.api.loader.impl.RecordReaderFileBatchLoader

All Implemented Interfaces:

Serializable, DataSetLoader, org.nd4j.api.loader.Loader<DataSet>

public class RecordReaderFileBatchLoader
extends Object
implements DataSetLoader

A dataset loader for use with FileBatch objects. The API (constructor arguments) mirrors RecordReaderDataSetIterator which it uses internally. Can be used in the context of Spark - see SparkDataUtils methods for this purpose

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
RecordReaderFileBatchLoader(RecordReader recordReader, int batchSize, int labelIndex, int numClasses) Main constructor for classification.
RecordReaderFileBatchLoader(RecordReader recordReader, int batchSize, int labelIndexFrom, int labelIndexTo, boolean regression) Main constructor for multi-label regression (i.e., regression with multiple outputs).
RecordReaderFileBatchLoader(RecordReader recordReader, int batchSize, int labelIndexFrom, int labelIndexTo, int numPossibleLabels, boolean regression, DataSetPreProcessor preProcessor) Main constructor

Method Summary

Modifier and Type	Method and Description
DataSet	load(org.nd4j.api.loader.Source source)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

RecordReaderFileBatchLoader

public RecordReaderFileBatchLoader(RecordReader recordReader,
                                   int batchSize,
                                   int labelIndex,
                                   int numClasses)

Main constructor for classification. This will convert the input class index (at position labelIndex, with integer values 0 to numPossibleLabels-1 inclusive) to the appropriate one-hot output/labels representation.

Parameters:

recordReader - RecordReader: provides the source of the data

batchSize - Batch size (number of examples) for the output DataSet objects

labelIndex - Index of the label Writable (usually an IntWritable), as obtained by recordReader.next()

numClasses - Number of classes (possible labels) for classification

RecordReaderFileBatchLoader

public RecordReaderFileBatchLoader(RecordReader recordReader,
                                   int batchSize,
                                   int labelIndexFrom,
                                   int labelIndexTo,
                                   boolean regression)

Main constructor for multi-label regression (i.e., regression with multiple outputs). Can also be used for single output regression with labelIndexFrom == labelIndexTo

Parameters:

recordReader - RecordReader to get data from

labelIndexFrom - Index of the first regression target

labelIndexTo - Index of the last regression target, inclusive

batchSize - Minibatch size

regression - Require regression = true. Mainly included to avoid clashing with other constructors previously defined :/

RecordReaderFileBatchLoader

public RecordReaderFileBatchLoader(RecordReader recordReader,
                                   int batchSize,
                                   int labelIndexFrom,
                                   int labelIndexTo,
                                   int numPossibleLabels,
                                   boolean regression,
                                   DataSetPreProcessor preProcessor)

Main constructor

Parameters:

recordReader - the recordreader to use

batchSize - Minibatch size - number of examples returned for each call of .next()

labelIndexFrom - the index of the label (for classification), or the first index of the labels for multi-output regression

labelIndexTo - only used if regression == true. The last index inclusive of the multi-output regression

numPossibleLabels - the number of possible labels for classification. Not used if regression == true

regression - if true: regression. If false: classification (assume labelIndexFrom is the class it belongs to)

preProcessor - Optional DataSetPreProcessor. May be null.

Method Detail

load

public DataSet load(org.nd4j.api.loader.Source source)
             throws IOException

Specified by:

load in interface org.nd4j.api.loader.Loader<DataSet>

Throws:

IOException