org.deeplearning4j.datasets.iterator

Class JointMultiDataSetIterator

java.lang.Object

org.deeplearning4j.datasets.iterator.JointMultiDataSetIterator

All Implemented Interfaces:

Serializable, Iterator<MultiDataSet>, MultiDataSetIterator

public class JointMultiDataSetIterator
extends Object
implements MultiDataSetIterator

This dataset iterator combines multiple DataSetIterators into 1 MultiDataSetIterator. Values from each iterator are joined on a per-example basis - i.e., the values from each DataSet are combined as different feature arrays for a multi-input neural network. Labels can come from either one of the underlying DataSetIteartors only (if 'outcome' is >= 0) or from all iterators (if outcome is < 0)

Author:

raver119@gmail.com

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
protected Collection<DataSetIterator>	iterators
protected int	outcome
protected MultiDataSetPreProcessor	preProcessor

Constructor Summary

Constructors

Constructor and Description
JointMultiDataSetIterator(DataSetIterator... iterators)
JointMultiDataSetIterator(int outcome, DataSetIterator... iterators)

Method Summary

Modifier and Type	Method and Description
boolean	asyncSupported() Does this MultiDataSetIterator support asynchronous prefetching of multiple MultiDataSet objects? Most MultiDataSetIterators do, but in some cases it may not make sense to wrap this iterator in an iterator that does asynchronous prefetching.
MultiDataSetPreProcessor	getPreProcessor() Get the MultiDataSetPreProcessor, if one has previously been set.
boolean	hasNext() Returns true if the iteration has more elements.
MultiDataSet	next() Returns the next element in the iteration.
MultiDataSet	next(int num) Fetch the next 'num' examples.
void	remove() PLEASE NOTE: This method is NOT implemented
void	reset() Resets the iterator back to the beginning
boolean	resetSupported() Is resetting supported by this DataSetIterator? Many DataSetIterators do support resetting, but some don't
void	setPreProcessor(MultiDataSetPreProcessor preProcessor) Set the preprocessor to be applied to each MultiDataSet, before each MultiDataSet is returned.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.util.Iterator

forEachRemaining

Field Detail

preProcessor

protected MultiDataSetPreProcessor preProcessor

iterators

protected Collection<DataSetIterator> iterators

outcome

protected int outcome

Constructor Detail

JointMultiDataSetIterator

public JointMultiDataSetIterator(DataSetIterator... iterators)

Parameters:

iterators - Underlying iterators to wrap

JointMultiDataSetIterator

public JointMultiDataSetIterator(int outcome,
                                 DataSetIterator... iterators)

Parameters:

outcome - Index to get the label from. If < 0, labels from all iterators will be used to create the final MultiDataSet

iterators - Underlying iterators to wrap

Method Detail

next

public MultiDataSet next(int num)

Fetch the next 'num' examples. Similar to the next method, but returns a specified number of examples

Specified by:

next in interface MultiDataSetIterator

Parameters:

num - Number of examples to fetch

setPreProcessor

public void setPreProcessor(MultiDataSetPreProcessor preProcessor)

Set the preprocessor to be applied to each MultiDataSet, before each MultiDataSet is returned.

Specified by:

setPreProcessor in interface MultiDataSetIterator

Parameters:

preProcessor - MultiDataSetPreProcessor. May be null.

getPreProcessor

public MultiDataSetPreProcessor getPreProcessor()

Get the MultiDataSetPreProcessor, if one has previously been set. Returns null if no preprocessor has been set

Specified by:

getPreProcessor in interface MultiDataSetIterator

Returns:

Preprocessor

resetSupported

public boolean resetSupported()

Is resetting supported by this DataSetIterator? Many DataSetIterators do support resetting, but some don't

Specified by:

resetSupported in interface MultiDataSetIterator

Returns:

true if reset method is supported; false otherwise

asyncSupported

public boolean asyncSupported()

Does this MultiDataSetIterator support asynchronous prefetching of multiple MultiDataSet objects? Most MultiDataSetIterators do, but in some cases it may not make sense to wrap this iterator in an iterator that does asynchronous prefetching. For example, it would not make sense to use asynchronous prefetching for the following types of iterators: (a) Iterators that store their full contents in memory already (b) Iterators that re-use features/labels arrays (as future next() calls will overwrite past contents) (c) Iterators that already implement some level of asynchronous prefetching (d) Iterators that may return different data depending on when the next() method is called

Specified by:

asyncSupported in interface MultiDataSetIterator

Returns:

true if asynchronous prefetching from this iterator is OK; false if asynchronous prefetching should not be used with this iterator

reset

public void reset()

Resets the iterator back to the beginning

Specified by:

reset in interface MultiDataSetIterator

hasNext

public boolean hasNext()

Returns true if the iteration has more elements. (In other words, returns true if next(int) would return an element rather than throwing an exception.)

Specified by:

hasNext in interface Iterator<MultiDataSet>

Returns:

true if the iteration has more elements

next

public MultiDataSet next()

Returns the next element in the iteration.

Specified by:

next in interface Iterator<MultiDataSet>

Returns:

the next element in the iteration

remove

public void remove()

PLEASE NOTE: This method is NOT implemented

Specified by:

remove in interface Iterator<MultiDataSet>

Throws:

UnsupportedOperationException - if the remove operation is not supported by this iterator

IllegalStateException - if the next method has not yet been called, or the remove method has already been called after the last call to the next method