org.deeplearning4j.parallelism

Class MagicQueue<T>

java.lang.Object

org.deeplearning4j.parallelism.MagicQueue<T>

All Implemented Interfaces:

Iterable<T>, Collection<T>, BlockingQueue<T>, Queue<T>

Deprecated.

@Deprecated
public class MagicQueue<T>
extends Object
implements BlockingQueue<T>

Limited Queue implementation, suited for multi-gpu prefetch. Basic idea is simple: DataSets are coming from DataSetIterator, and their device location is unknown. So, for better performance DataSets should be transparently moved to the devices where they will be used later, and this should be done in background. PLEASE NOTE: This class is pending removal, since better behavior was implemented as InterleavedCallback for AsyncDataSetIterator

Author:

raver119@gmail.com

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	MagicQueue.Builder Deprecated.
static class	MagicQueue.Mode Deprecated.
static class	MagicQueue.Type Deprecated.

Field Summary

Fields

Modifier and Type	Field and Description
protected List<LinkedBlockingQueue<T>>	backingQueues Deprecated.
protected int	capacity Deprecated.
protected AtomicLong	cntGet Deprecated.
protected AtomicLong	cntPut Deprecated.
protected List<org.deeplearning4j.parallelism.MagicQueue.QueueHandler>	handlers Deprecated.
protected AtomicInteger	interleavedCounter Deprecated.
protected AtomicInteger	interleavedPutter Deprecated.
protected MagicQueue.Mode	mode Deprecated.
protected AtomicInteger	nextBucket Deprecated.
protected int	numberOfBuckets Deprecated.
protected MagicQueue.Type	type Deprecated.

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	MagicQueue(int numberOfFlows, int capacity, MagicQueue.Type type) Deprecated.

Method Summary

Modifier and Type	Method and Description
boolean	add(T dataSet) Deprecated.
boolean	addAll(Collection<? extends T> c) Deprecated.
void	clear() Deprecated.
boolean	contains(Object o) Deprecated. This method isn't supported
boolean	containsAll(Collection<?> c) Deprecated. This method isn't supported
int	drainTo(Collection<? super T> c) Deprecated.
int	drainTo(Collection<? super T> c, int maxElements) Deprecated.
T	element() Deprecated.
boolean	isEmpty() Deprecated.
Iterator<T>	iterator() Deprecated. This method isn't supported
boolean	offer(T dataSet) Deprecated.
boolean	offer(T dataSet, long timeout, TimeUnit unit) Deprecated.
T	peek() Deprecated.
T	poll() Deprecated. This method is supposed to be called from managed thread, attached to specific device.
T	poll(long time, TimeUnit timeUnit) Deprecated. This method is supposed to be called from managed thread, attached to specific device.
void	put(T dataSet) Deprecated.
int	remainingCapacity() Deprecated.
T	remove() Deprecated.
boolean	remove(Object o) Deprecated. This method isn't supported
boolean	removeAll(Collection<?> c) Deprecated. This method isn't supported
boolean	retainAll(Collection<?> c) Deprecated. This method isn't supported
int	size() Deprecated. This method returns average queue size for all devices
protected int	size(int deviceId) Deprecated.
T	take() Deprecated.
Object[]	toArray() Deprecated. This method isn't supported
<T> T[]	toArray(T[] a) Deprecated. This method isn't supported

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.util.Collection

equals, hashCode, parallelStream, removeIf, spliterator, stream

Methods inherited from interface java.lang.Iterable

forEach

Field Detail

backingQueues

protected final List<LinkedBlockingQueue<T>> backingQueues

Deprecated.

nextBucket

protected final AtomicInteger nextBucket

Deprecated.

numberOfBuckets

protected final int numberOfBuckets

Deprecated.

handlers

protected final List<org.deeplearning4j.parallelism.MagicQueue.QueueHandler> handlers

Deprecated.

capacity

protected int capacity

Deprecated.

mode

protected MagicQueue.Mode mode

Deprecated.

type

protected MagicQueue.Type type

Deprecated.

interleavedCounter

protected AtomicInteger interleavedCounter

Deprecated.

interleavedPutter

protected AtomicInteger interleavedPutter

Deprecated.

cntPut

protected AtomicLong cntPut

Deprecated.

cntGet

protected AtomicLong cntGet

Deprecated.

Constructor Detail

MagicQueue

protected MagicQueue(int numberOfFlows,
                     int capacity,
                     MagicQueue.Type type)

Deprecated.

Method Detail

size

public int size()

Deprecated.

This method returns average queue size for all devices

Specified by:

size in interface Collection<T>

Returns:

size

protected int size(int deviceId)

Deprecated.

isEmpty

public boolean isEmpty()

Deprecated.

Specified by:

isEmpty in interface Collection<T>

contains

public boolean contains(Object o)

Deprecated.

This method isn't supported

Specified by:

contains in interface Collection<T>

Specified by:

contains in interface BlockingQueue<T>

Parameters:

o -

Returns:

drainTo

public int drainTo(Collection<? super T> c)

Deprecated.

Specified by:

drainTo in interface BlockingQueue<T>

drainTo

public int drainTo(Collection<? super T> c,
                   int maxElements)

Deprecated.

Specified by:

drainTo in interface BlockingQueue<T>

iterator

public Iterator<T> iterator()

Deprecated.

This method isn't supported

Specified by:

iterator in interface Iterable<T>

Specified by:

iterator in interface Collection<T>

Returns:

toArray

public Object[] toArray()

Deprecated.

This method isn't supported

Specified by:

toArray in interface Collection<T>

Returns:

toArray

public <T> T[] toArray(T[] a)

Deprecated.

This method isn't supported

Specified by:

toArray in interface Collection<T>

Type Parameters:

T -

Parameters:

a -

Returns:

add

public boolean add(T dataSet)

Deprecated.

Specified by:

add in interface Collection<T>

Specified by:

add in interface BlockingQueue<T>

Specified by:

add in interface Queue<T>

remove

public boolean remove(Object o)

Deprecated.

This method isn't supported

Specified by:

remove in interface Collection<T>

Specified by:

remove in interface BlockingQueue<T>

Parameters:

o -

Returns:

containsAll

public boolean containsAll(Collection<?> c)

Deprecated.

This method isn't supported

Specified by:

containsAll in interface Collection<T>

Parameters:

c -

Returns:

addAll

public boolean addAll(Collection<? extends T> c)

Deprecated.

Specified by:

addAll in interface Collection<T>

removeAll

public boolean removeAll(Collection<?> c)

Deprecated.

This method isn't supported

Specified by:

removeAll in interface Collection<T>

Parameters:

c -

Returns:

retainAll

public boolean retainAll(Collection<?> c)

Deprecated.

This method isn't supported

Specified by:

retainAll in interface Collection<T>

Parameters:

c -

Returns:

clear

public void clear()

Deprecated.

Specified by:

clear in interface Collection<T>

offer

public boolean offer(T dataSet)

Deprecated.

Specified by:

offer in interface BlockingQueue<T>

Specified by:

offer in interface Queue<T>

put

public void put(T dataSet)
         throws InterruptedException

Deprecated.

Specified by:

put in interface BlockingQueue<T>

Throws:

InterruptedException

offer

public boolean offer(T dataSet,
                     long timeout,
                     TimeUnit unit)
              throws InterruptedException

Deprecated.

Specified by:

offer in interface BlockingQueue<T>

Throws:

InterruptedException

take

public T take()
       throws InterruptedException

Deprecated.

Specified by:

take in interface BlockingQueue<T>

Throws:

InterruptedException

remove

public T remove()

Deprecated.

Specified by:

remove in interface Queue<T>

poll

public T poll(long time,
              TimeUnit timeUnit)
       throws InterruptedException

Deprecated.

This method is supposed to be called from managed thread, attached to specific device. It returns 1 DataSet element from head of the queue, and deletes that element from Queue. If queue is empty, Please note: if there's nothing available in Queue - NULL will be returned

Specified by:

poll in interface BlockingQueue<T>

Parameters:

time - time to wait for something appear in queue

timeUnit - TimeUnit for time param

Returns:

Throws:

InterruptedException

remainingCapacity

public int remainingCapacity()

Deprecated.

Specified by:

remainingCapacity in interface BlockingQueue<T>

poll

public T poll()

Deprecated.

This method is supposed to be called from managed thread, attached to specific device. It returns 1 DataSet element from head of the queue, and deletes that element from Queue Please note: if there's nothing available in Queue - NULL will be returned

Specified by:

poll in interface Queue<T>

Returns:

element

public T element()

Deprecated.

Specified by:

element in interface Queue<T>

peek

public T peek()

Deprecated.

Specified by:

peek in interface Queue<T>