org.apache.tez.runtime.library.common.task.local.output

Class TezLocalTaskOutputFiles

java.lang.Object

org.apache.tez.runtime.library.common.task.local.output.TezTaskOutput

org.apache.tez.runtime.library.common.task.local.output.TezLocalTaskOutputFiles

@InterfaceAudience.Private
@InterfaceStability.Unstable
public class TezLocalTaskOutputFiles
extends TezTaskOutput

Manipulate the working area for the transient store for maps and reduces. This class is used by map and reduce tasks to identify the directories that they need to write to/read from for intermediate files. The callers of these methods are from the Child running the Task.

Field Summary

Fields inherited from class org.apache.tez.runtime.library.common.task.local.output.TezTaskOutput

conf, uniqueId

Constructor Summary

Constructors

Constructor and Description
TezLocalTaskOutputFiles(org.apache.hadoop.conf.Configuration conf, String uniqueId)

Method Summary

Methods

Modifier and Type	Method and Description
org.apache.hadoop.fs.Path	getInputFile(InputAttemptIdentifier mapId) Return a local reduce input file created earlier
org.apache.hadoop.fs.Path	getInputFileForWrite(int taskId, long size) Create a local reduce input file name.
org.apache.hadoop.fs.Path	getOutputFile() Return the path to local map output file created earlier
org.apache.hadoop.fs.Path	getOutputFileForWrite() Create a local map output file name.
org.apache.hadoop.fs.Path	getOutputFileForWrite(long size) Create a local map output file name.
org.apache.hadoop.fs.Path	getOutputFileForWriteInVolume(org.apache.hadoop.fs.Path existing) Create a local map output file name on the same volume.
org.apache.hadoop.fs.Path	getOutputIndexFile() Return the path to a local map output index file created earlier
org.apache.hadoop.fs.Path	getOutputIndexFileForWrite(long size) Create a local map output index file name.
org.apache.hadoop.fs.Path	getOutputIndexFileForWriteInVolume(org.apache.hadoop.fs.Path existing) Create a local map output index file name on the same volume.
org.apache.hadoop.fs.Path	getSpillFile(int spillNumber) Return a local map spill file created earlier.
org.apache.hadoop.fs.Path	getSpillFileForWrite(int spillNumber, long size) Create a local map spill file name.
String	getSpillFileName(int spillNum) Construct a spill file name, given a spill number
org.apache.hadoop.fs.Path	getSpillIndexFile(int spillNumber) Return a local map spill index file created earlier
org.apache.hadoop.fs.Path	getSpillIndexFileForWrite(int spillNumber, long size) Create a local map spill index file name.
void	removeAll() Removes all of the files related to a task.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TezLocalTaskOutputFiles

public TezLocalTaskOutputFiles(org.apache.hadoop.conf.Configuration conf,
                       String uniqueId)

Method Detail

getOutputFile

public org.apache.hadoop.fs.Path getOutputFile()
                                        throws IOException

Return the path to local map output file created earlier

Specified by:

getOutputFile in class TezTaskOutput

Returns:

path

Throws:

IOException

getOutputFileForWrite

public org.apache.hadoop.fs.Path getOutputFileForWrite(long size)
                                                throws IOException

Create a local map output file name.

Specified by:

getOutputFileForWrite in class TezTaskOutput

Parameters:

size - the size of the file

Returns:

path

Throws:

IOException

getOutputFileForWrite

public org.apache.hadoop.fs.Path getOutputFileForWrite()
                                                throws IOException

Create a local map output file name. This should *only* be used if the size of the file is not known. Otherwise use the equivalent which accepts a size parameter.

Specified by:

getOutputFileForWrite in class TezTaskOutput

Returns:

path

Throws:

IOException

getOutputFileForWriteInVolume

public org.apache.hadoop.fs.Path getOutputFileForWriteInVolume(org.apache.hadoop.fs.Path existing)

Create a local map output file name on the same volume.

Specified by:

getOutputFileForWriteInVolume in class TezTaskOutput

getOutputIndexFile

public org.apache.hadoop.fs.Path getOutputIndexFile()
                                             throws IOException

Return the path to a local map output index file created earlier

Specified by:

getOutputIndexFile in class TezTaskOutput

Returns:

path

Throws:

IOException

getOutputIndexFileForWrite

public org.apache.hadoop.fs.Path getOutputIndexFileForWrite(long size)
                                                     throws IOException

Create a local map output index file name.

Specified by:

getOutputIndexFileForWrite in class TezTaskOutput

Parameters:

size - the size of the file

Returns:

path

Throws:

IOException

getOutputIndexFileForWriteInVolume

public org.apache.hadoop.fs.Path getOutputIndexFileForWriteInVolume(org.apache.hadoop.fs.Path existing)

Create a local map output index file name on the same volume.

Specified by:

getOutputIndexFileForWriteInVolume in class TezTaskOutput

getSpillFile

public org.apache.hadoop.fs.Path getSpillFile(int spillNumber)
                                       throws IOException

Return a local map spill file created earlier.

Specified by:

getSpillFile in class TezTaskOutput

Parameters:

spillNumber - the number

Returns:

path

Throws:

IOException

getSpillFileForWrite

public org.apache.hadoop.fs.Path getSpillFileForWrite(int spillNumber,
                                             long size)
                                               throws IOException

Create a local map spill file name.

Specified by:

getSpillFileForWrite in class TezTaskOutput

Parameters:

spillNumber - the number

size - the size of the file

Returns:

path

Throws:

IOException

getSpillIndexFile

public org.apache.hadoop.fs.Path getSpillIndexFile(int spillNumber)
                                            throws IOException

Return a local map spill index file created earlier

Specified by:

getSpillIndexFile in class TezTaskOutput

Parameters:

spillNumber - the number

Returns:

path

Throws:

IOException

getSpillIndexFileForWrite

public org.apache.hadoop.fs.Path getSpillIndexFileForWrite(int spillNumber,
                                                  long size)
                                                    throws IOException

Create a local map spill index file name.

Specified by:

getSpillIndexFileForWrite in class TezTaskOutput

Parameters:

spillNumber - the number

size - the size of the file

Returns:

path

Throws:

IOException

getInputFile

public org.apache.hadoop.fs.Path getInputFile(InputAttemptIdentifier mapId)
                                       throws IOException

Return a local reduce input file created earlier

Specified by:

getInputFile in class TezTaskOutput

Parameters:

mapId - a map task id

Returns:

path

Throws:

IOException

getInputFileForWrite

public org.apache.hadoop.fs.Path getInputFileForWrite(int taskId,
                                             long size)
                                               throws IOException

Create a local reduce input file name.

Specified by:

getInputFileForWrite in class TezTaskOutput

Parameters:

taskId - a task id

size - the size of the file

Returns:

path

Throws:

IOException

getSpillFileName

public String getSpillFileName(int spillNum)

Description copied from class: TezTaskOutput

Construct a spill file name, given a spill number

Specified by:

getSpillFileName in class TezTaskOutput

Returns:

removeAll

public void removeAll()
               throws IOException

Removes all of the files related to a task.

Specified by:

removeAll in class TezTaskOutput

Throws:

IOException