org.apache.tez.runtime.library.common.sort.impl

Class PipelinedSorter

java.lang.Object

org.apache.tez.runtime.library.common.sort.impl.ExternalSorter

org.apache.tez.runtime.library.common.sort.impl.PipelinedSorter

public class PipelinedSorter
extends ExternalSorter

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected class	PipelinedSorter.InputByteBuffer

Nested classes/interfaces inherited from class org.apache.tez.runtime.library.common.sort.impl.ExternalSorter

ExternalSorter.MapBufferTooSmallException

Field Summary

Fields

Modifier and Type	Field and Description
static int	MAP_OUTPUT_INDEX_RECORD_LENGTH The size of each record in the index file for the map-outputs.

Fields inherited from class org.apache.tez.runtime.library.common.sort.impl.ExternalSorter

additionalSpillBytesRead, additionalSpillBytesWritten, availableMemoryMb, codec, combiner, comparator, conf, fileOutputByteCounter, ifileBufferSize, ifileReadAhead, ifileReadAheadLength, keyClass, keySerializer, mapOutputByteCounter, mapOutputFile, mapOutputRecordCounter, nullProgressable, numAdditionalSpills, outputBytesWithOverheadCounter, outputContext, partitioner, partitions, rfs, serializationFactory, sorter, spilledRecordsCounter, valClass, valSerializer

Constructor Summary

Constructors

Constructor and Description
PipelinedSorter(org.apache.tez.runtime.api.OutputContext outputContext, org.apache.hadoop.conf.Configuration conf, int numOutputs, long initialMemoryAvailable)

Method Summary

Methods

Modifier and Type	Method and Description
void	close()
void	flush()
void	sort()
void	spill()
void	write(Object key, Object value)

Methods inherited from class org.apache.tez.runtime.library.common.sort.impl.ExternalSorter

getInitialMemoryRequirement, getMapOutput, getShuffleHeader, getSortedStream, runCombineProcessor, sameVolRename, write

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

MAP_OUTPUT_INDEX_RECORD_LENGTH

public static final int MAP_OUTPUT_INDEX_RECORD_LENGTH

The size of each record in the index file for the map-outputs.

See Also:

Constant Field Values

Constructor Detail

PipelinedSorter

public PipelinedSorter(org.apache.tez.runtime.api.OutputContext outputContext,
               org.apache.hadoop.conf.Configuration conf,
               int numOutputs,
               long initialMemoryAvailable)
                throws IOException

Throws:

IOException

Method Detail

sort

public void sort()
          throws IOException

Throws:

IOException

write

public void write(Object key,
         Object value)
           throws IOException

Specified by:

write in class ExternalSorter

Throws:

IOException

spill

public void spill()
           throws IOException

Throws:

IOException

flush

public void flush()
           throws IOException

Specified by:

flush in class ExternalSorter

Throws:

IOException

close

public void close()

Specified by:

close in class ExternalSorter