org.apache.tez.mapreduce.hadoop.mapreduce

Class JobContextImpl

java.lang.Object

org.apache.tez.mapreduce.hadoop.mapreduce.JobContextImpl

All Implemented Interfaces:

org.apache.hadoop.mapreduce.JobContext, org.apache.hadoop.mapreduce.MRJobConfig

@InterfaceAudience.Private
@InterfaceStability.Unstable
public class JobContextImpl
extends Object
implements org.apache.hadoop.mapred.JobContext

A read-only view of the job that is provided to the tasks while they are running.

Field Summary

Fields

Modifier and Type	Field and Description
protected org.apache.hadoop.mapred.JobConf	conf
protected org.apache.hadoop.security.Credentials	credentials

Fields inherited from interface org.apache.hadoop.mapreduce.MRJobConfig

APPLICATION_ATTEMPT_ID, APPLICATION_MASTER_CLASS, CACHE_ARCHIVES, CACHE_ARCHIVES_SIZES, CACHE_ARCHIVES_TIMESTAMPS, CACHE_ARCHIVES_VISIBILITIES, CACHE_FILE_TIMESTAMPS, CACHE_FILE_VISIBILITIES, CACHE_FILES, CACHE_FILES_SIZES, CACHE_LOCALARCHIVES, CACHE_LOCALFILES, CACHE_SYMLINK, CLASSPATH_ARCHIVES, CLASSPATH_FILES, COMBINE_CLASS_ATTR, COMBINE_RECORDS_BEFORE_PROGRESS, COMBINER_GROUP_COMPARATOR_CLASS, COMPLETED_MAPS_FOR_REDUCE_SLOWSTART, COUNTER_GROUP_NAME_MAX_DEFAULT, COUNTER_GROUP_NAME_MAX_KEY, COUNTER_GROUPS_MAX_DEFAULT, COUNTER_GROUPS_MAX_KEY, COUNTER_NAME_MAX_DEFAULT, COUNTER_NAME_MAX_KEY, COUNTERS_MAX_DEFAULT, COUNTERS_MAX_KEY, DEFAULT_JOB_ACL_MODIFY_JOB, DEFAULT_JOB_ACL_VIEW_JOB, DEFAULT_JOB_AM_ACCESS_DISABLED, DEFAULT_JOB_TOKEN_TRACKING_IDS_ENABLED, DEFAULT_LOG_LEVEL, DEFAULT_MAP_CPU_VCORES, DEFAULT_MAP_MEMORY_MB, DEFAULT_MAPRED_ADMIN_JAVA_OPTS, DEFAULT_MAPRED_ADMIN_USER_ENV, DEFAULT_MAPREDUCE_APPLICATION_CLASSPATH, DEFAULT_MAPREDUCE_CROSS_PLATFORM_APPLICATION_CLASSPATH, DEFAULT_MAPREDUCE_JOB_EMIT_TIMELINE_DATA, DEFAULT_MAX_ALLOWED_FETCH_FAILURES_FRACTION, DEFAULT_MAX_FETCH_FAILURES_NOTIFICATIONS, DEFAULT_MAX_SHUFFLE_FETCH_HOST_FAILURES, DEFAULT_MAX_SHUFFLE_FETCH_RETRY_DELAY, DEFAULT_MR_AM_ADMIN_COMMAND_OPTS, DEFAULT_MR_AM_COMMAND_OPTS, DEFAULT_MR_AM_COMMIT_WINDOW_MS, DEFAULT_MR_AM_COMMITTER_CANCEL_TIMEOUT_MS, DEFAULT_MR_AM_CONTAINERLAUNCHER_THREAD_COUNT_LIMIT, DEFAULT_MR_AM_CPU_VCORES, DEFAULT_MR_AM_HISTORY_COMPLETE_EVENT_FLUSH_TIMEOUT_MS, DEFAULT_MR_AM_HISTORY_JOB_COMPLETE_UNFLUSHED_MULTIPLIER, DEFAULT_MR_AM_HISTORY_MAX_UNFLUSHED_COMPLETE_EVENTS, DEFAULT_MR_AM_HISTORY_USE_BATCHED_FLUSH_QUEUE_SIZE_THRESHOLD, DEFAULT_MR_AM_IGNORE_BLACKLISTING_BLACKLISTED_NODE_PERCENT, DEFAULT_MR_AM_JOB_CLIENT_THREAD_COUNT, DEFAULT_MR_AM_JOB_REDUCE_PREEMPTION_LIMIT, DEFAULT_MR_AM_JOB_REDUCE_RAMP_UP_LIMIT, DEFAULT_MR_AM_LOG_BACKUPS, DEFAULT_MR_AM_LOG_KB, DEFAULT_MR_AM_LOG_LEVEL, DEFAULT_MR_AM_MAX_ATTEMPTS, DEFAULT_MR_AM_NUM_PROGRESS_SPLITS, DEFAULT_MR_AM_PROFILE, DEFAULT_MR_AM_STAGING_DIR, DEFAULT_MR_AM_TASK_ESTIMATOR_SMOOTH_LAMBDA_MS, DEFAULT_MR_AM_TASK_LISTENER_THREAD_COUNT, DEFAULT_MR_AM_TO_RM_HEARTBEAT_INTERVAL_MS, DEFAULT_MR_AM_TO_RM_WAIT_INTERVAL_MS, DEFAULT_MR_AM_VMEM_MB, DEFAULT_MR_CLIENT_MAX_RETRIES, DEFAULT_MR_CLIENT_TO_AM_IPC_MAX_RETRIES, DEFAULT_MR_CLIENT_TO_AM_IPC_MAX_RETRIES_ON_TIMEOUTS, DEFAULT_MR_ENCRYPTED_INTERMEDIATE_DATA, DEFAULT_MR_ENCRYPTED_INTERMEDIATE_DATA_BUFFER_KB, DEFAULT_MR_ENCRYPTED_INTERMEDIATE_DATA_KEY_SIZE_BITS, DEFAULT_MR_JOB_END_NOTIFICATION_TIMEOUT, DEFAULT_MR_JOB_REDUCER_PREEMPT_DELAY_SEC, DEFAULT_REDUCE_CPU_VCORES, DEFAULT_REDUCE_MEMORY_MB, DEFAULT_SHELL, DEFAULT_SHUFFLE_FETCH_RETRY_INTERVAL_MS, DEFAULT_SHUFFLE_INPUT_BUFFER_PERCENT, DEFAULT_SPLIT_METAINFO_MAXSIZE, DEFAULT_TASK_LOG_BACKUPS, DEFAULT_TASK_PROFILE_PARAMS, GROUP_COMPARATOR_CLASS, HADOOP_WORK_DIR, ID, INDEX_CACHE_MEMORY_LIMIT, INPUT_FORMAT_CLASS_ATTR, IO_SORT_FACTOR, IO_SORT_MB, JAR, JAR_UNPACK_PATTERN, JOB_ACL_MODIFY_JOB, JOB_ACL_VIEW_JOB, JOB_AM_ACCESS_DISABLED, JOB_CANCEL_DELEGATION_TOKEN, JOB_CONF_FILE, JOB_JAR, JOB_JOBTRACKER_ID, JOB_LOCAL_DIR, JOB_NAME, JOB_NAMENODES, JOB_SPLIT, JOB_SPLIT_METAINFO, JOB_SUBMIT_DIR, JOB_SUBMITHOST, JOB_SUBMITHOSTADDR, JOB_TAGS, JOB_TOKEN_TRACKING_IDS, JOB_TOKEN_TRACKING_IDS_ENABLED, JOB_UBERTASK_ENABLE, JOB_UBERTASK_MAXBYTES, JOB_UBERTASK_MAXMAPS, JOB_UBERTASK_MAXREDUCES, JVM_NUMTASKS_TORUN, KEY_COMPARATOR, MAP_CLASS_ATTR, MAP_COMBINE_MIN_SPILLS, MAP_CPU_VCORES, MAP_DEBUG_SCRIPT, MAP_ENV, MAP_FAILURES_MAX_PERCENT, MAP_INPUT_FILE, MAP_INPUT_PATH, MAP_INPUT_START, MAP_JAVA_OPTS, MAP_LOG_LEVEL, MAP_MAX_ATTEMPTS, MAP_MEMORY_MB, MAP_OUTPUT_COLLECTOR_CLASS_ATTR, MAP_OUTPUT_COMPRESS, MAP_OUTPUT_COMPRESS_CODEC, MAP_OUTPUT_KEY_CLASS, MAP_OUTPUT_KEY_FIELD_SEPERATOR, MAP_OUTPUT_VALUE_CLASS, MAP_SKIP_INCR_PROC_COUNT, MAP_SKIP_MAX_RECORDS, MAP_SORT_SPILL_PERCENT, MAP_SPECULATIVE, MAPRED_ADMIN_USER_ENV, MAPRED_ADMIN_USER_SHELL, MAPRED_MAP_ADMIN_JAVA_OPTS, MAPRED_REDUCE_ADMIN_JAVA_OPTS, MAPREDUCE_APPLICATION_CLASSPATH, MAPREDUCE_APPLICATION_FRAMEWORK_PATH, MAPREDUCE_JOB_CLASSLOADER, MAPREDUCE_JOB_CLASSLOADER_SYSTEM_CLASSES, MAPREDUCE_JOB_CREDENTIALS_BINARY, MAPREDUCE_JOB_DIR, MAPREDUCE_JOB_EMIT_TIMELINE_DATA, MAPREDUCE_JOB_LOG4J_PROPERTIES_FILE, MAPREDUCE_JOB_SHUFFLE_PROVIDER_SERVICES, MAPREDUCE_JOB_USER_CLASSPATH_FIRST, MAPREDUCE_V2_CHILD_CLASS, MAX_ALLOWED_FETCH_FAILURES_FRACTION, MAX_FETCH_FAILURES_NOTIFICATIONS, MAX_SHUFFLE_FETCH_HOST_FAILURES, MAX_SHUFFLE_FETCH_RETRY_DELAY, MAX_TASK_FAILURES_PER_TRACKER, MR_AM_ADMIN_COMMAND_OPTS, MR_AM_ADMIN_USER_ENV, MR_AM_COMMAND_OPTS, MR_AM_COMMIT_WINDOW_MS, MR_AM_COMMITTER_CANCEL_TIMEOUT_MS, MR_AM_CONTAINERLAUNCHER_THREAD_COUNT_LIMIT, MR_AM_CPU_VCORES, MR_AM_CREATE_JH_INTERMEDIATE_BASE_DIR, MR_AM_ENV, MR_AM_HISTORY_COMPLETE_EVENT_FLUSH_TIMEOUT_MS, MR_AM_HISTORY_JOB_COMPLETE_UNFLUSHED_MULTIPLIER, MR_AM_HISTORY_MAX_UNFLUSHED_COMPLETE_EVENTS, MR_AM_HISTORY_USE_BATCHED_FLUSH_QUEUE_SIZE_THRESHOLD, MR_AM_IGNORE_BLACKLISTING_BLACKLISTED_NODE_PERECENT, MR_AM_JOB_CLIENT_PORT_RANGE, MR_AM_JOB_CLIENT_THREAD_COUNT, MR_AM_JOB_NODE_BLACKLISTING_ENABLE, MR_AM_JOB_RECOVERY_ENABLE, MR_AM_JOB_RECOVERY_ENABLE_DEFAULT, MR_AM_JOB_REDUCE_PREEMPTION_LIMIT, MR_AM_JOB_REDUCE_RAMPUP_UP_LIMIT, MR_AM_JOB_SPECULATOR, MR_AM_LOG_BACKUPS, MR_AM_LOG_KB, MR_AM_LOG_LEVEL, MR_AM_MAX_ATTEMPTS, MR_AM_NUM_PROGRESS_SPLITS, MR_AM_PREFIX, MR_AM_PROFILE, MR_AM_PROFILE_PARAMS, MR_AM_SECURITY_SERVICE_AUTHORIZATION_CLIENT, MR_AM_SECURITY_SERVICE_AUTHORIZATION_TASK_UMBILICAL, MR_AM_STAGING_DIR, MR_AM_TASK_ESTIMATOR, MR_AM_TASK_ESTIMATOR_EXPONENTIAL_RATE_ENABLE, MR_AM_TASK_ESTIMATOR_SMOOTH_LAMBDA_MS, MR_AM_TASK_LISTENER_THREAD_COUNT, MR_AM_TO_RM_HEARTBEAT_INTERVAL_MS, MR_AM_TO_RM_WAIT_INTERVAL_MS, MR_AM_VMEM_MB, MR_APPLICATION_TYPE, MR_CLIENT_MAX_RETRIES, MR_CLIENT_TO_AM_IPC_MAX_RETRIES, MR_CLIENT_TO_AM_IPC_MAX_RETRIES_ON_TIMEOUTS, MR_ENCRYPTED_INTERMEDIATE_DATA, MR_ENCRYPTED_INTERMEDIATE_DATA_BUFFER_KB, MR_ENCRYPTED_INTERMEDIATE_DATA_KEY_SIZE_BITS, MR_JOB_END_NOTIFICATION_MAX_ATTEMPTS, MR_JOB_END_NOTIFICATION_MAX_RETRY_INTERVAL, MR_JOB_END_NOTIFICATION_PROXY, MR_JOB_END_NOTIFICATION_TIMEOUT, MR_JOB_END_NOTIFICATION_URL, MR_JOB_END_RETRY_ATTEMPTS, MR_JOB_END_RETRY_INTERVAL, MR_JOB_REDUCER_PREEMPT_DELAY_SEC, MR_PREFIX, NUM_MAP_PROFILES, NUM_MAPS, NUM_REDUCE_PROFILES, NUM_REDUCES, OUTPUT, OUTPUT_FORMAT_CLASS_ATTR, OUTPUT_KEY_CLASS, OUTPUT_VALUE_CLASS, PARTITIONER_CLASS_ATTR, PRESERVE_FAILED_TASK_FILES, PRESERVE_FILES_PATTERN, PRIORITY, QUEUE_NAME, RECORDS_BEFORE_PROGRESS, REDUCE_CLASS_ATTR, REDUCE_CPU_VCORES, REDUCE_DEBUG_SCRIPT, REDUCE_ENV, REDUCE_FAILURES_MAXPERCENT, REDUCE_INPUT_BUFFER_PERCENT, REDUCE_JAVA_OPTS, REDUCE_LOG_LEVEL, REDUCE_MARKRESET_BUFFER_PERCENT, REDUCE_MARKRESET_BUFFER_SIZE, REDUCE_MAX_ATTEMPTS, REDUCE_MEMORY_MB, REDUCE_MEMORY_TOTAL_BYTES, REDUCE_MEMTOMEM_ENABLED, REDUCE_MEMTOMEM_THRESHOLD, REDUCE_MERGE_INMEM_THRESHOLD, REDUCE_SKIP_INCR_PROC_COUNT, REDUCE_SKIP_MAXGROUPS, REDUCE_SPECULATIVE, RESERVATION_ID, SETUP_CLEANUP_NEEDED, SHUFFLE_CONNECT_TIMEOUT, SHUFFLE_FETCH_FAILURES, SHUFFLE_FETCH_RETRY_ENABLED, SHUFFLE_FETCH_RETRY_INTERVAL_MS, SHUFFLE_FETCH_RETRY_TIMEOUT_MS, SHUFFLE_INPUT_BUFFER_PERCENT, SHUFFLE_MEMORY_LIMIT_PERCENT, SHUFFLE_MERGE_PERCENT, SHUFFLE_NOTIFY_READERROR, SHUFFLE_PARALLEL_COPIES, SHUFFLE_READ_TIMEOUT, SKIP_OUTDIR, SKIP_RECORDS, SKIP_START_ATTEMPTS, SPECULATIVE_SLOWNODE_THRESHOLD, SPECULATIVE_SLOWTASK_THRESHOLD, SPECULATIVECAP, SPLIT_FILE, SPLIT_METAINFO_MAXSIZE, STDERR_LOGFILE_ENV, STDOUT_LOGFILE_ENV, TASK_ATTEMPT_ID, TASK_CLEANUP_NEEDED, TASK_DEBUGOUT_LINES, TASK_ID, TASK_ISMAP, TASK_LOG_BACKUPS, TASK_MAP_PROFILE_PARAMS, TASK_OUTPUT_DIR, TASK_PARTITION, TASK_PROFILE, TASK_PROFILE_PARAMS, TASK_REDUCE_PROFILE_PARAMS, TASK_TEMP_DIR, TASK_TIMEOUT, TASK_TIMEOUT_CHECK_INTERVAL_MS, TASK_USERLOG_LIMIT, USER_LOG_RETAIN_HOURS, USER_NAME, WORKDIR, WORKFLOW_ADJACENCY_PREFIX_PATTERN, WORKFLOW_ADJACENCY_PREFIX_STRING, WORKFLOW_ID, WORKFLOW_NAME, WORKFLOW_NODE_NAME, WORKFLOW_TAGS, WORKING_DIR

Constructor Summary

Constructors

Constructor and Description
JobContextImpl(org.apache.hadoop.conf.Configuration conf, org.apache.tez.dag.records.TezDAGID dagId)
JobContextImpl(org.apache.hadoop.conf.Configuration conf, org.apache.tez.dag.records.TezDAGID dagId, org.apache.hadoop.util.Progressable progress)

Method Summary

Methods

Modifier and Type	Method and Description
org.apache.hadoop.fs.Path[]	getArchiveClassPaths() Get the archive entries in classpath as an array of Path
String[]	getArchiveTimestamps() Get the timestamps of the archives.
URI[]	getCacheArchives() Get cache archives set in the Configuration
URI[]	getCacheFiles() Get cache files set in the Configuration
Class<? extends org.apache.hadoop.mapreduce.Reducer<?,?,?,?>>	getCombinerClass() Get the combiner class for the job.
org.apache.hadoop.io.RawComparator<?>	getCombinerKeyGroupingComparator() Get the user defined RawComparator comparator for grouping keys of inputs to the combiner.
org.apache.hadoop.conf.Configuration	getConfiguration() Return the configuration for the job.
org.apache.hadoop.security.Credentials	getCredentials()
org.apache.hadoop.fs.Path[]	getFileClassPaths() Get the file entries in classpath as an array of Path
String[]	getFileTimestamps() Get the timestamps of the files.
org.apache.hadoop.io.RawComparator<?>	getGroupingComparator() Get the user defined RawComparator comparator for grouping keys of inputs to the reduce.
Class<? extends org.apache.hadoop.mapreduce.InputFormat<?,?>>	getInputFormatClass() Get the InputFormat class for the job.
String	getJar() Get the pathname of the job's jar.
org.apache.hadoop.mapred.JobConf	getJobConf()
org.apache.hadoop.mapreduce.JobID	getJobID() Get the unique ID for the job.
String	getJobName() Get the user-specified job name.
boolean	getJobSetupCleanupNeeded() Get whether job-setup and job-cleanup is needed for the job
org.apache.hadoop.fs.Path[]	getLocalCacheArchives() Return the path array of the localized caches
org.apache.hadoop.fs.Path[]	getLocalCacheFiles() Return the path array of the localized files
Class<?>	getMapOutputKeyClass() Get the key class for the map output data.
Class<?>	getMapOutputValueClass() Get the value class for the map output data.
Class<? extends org.apache.hadoop.mapreduce.Mapper<?,?,?,?>>	getMapperClass() Get the Mapper class for the job.
int	getMaxMapAttempts() Get the configured number of maximum attempts that will be made to run a map task, as specified by the mapred.map.max.attempts property.
int	getMaxReduceAttempts() Get the configured number of maximum attempts that will be made to run a reduce task, as specified by the mapred.reduce.max.attempts property.
int	getNumReduceTasks() Get configured the number of reduce tasks for this job.
Class<? extends org.apache.hadoop.mapreduce.OutputFormat<?,?>>	getOutputFormatClass() Get the OutputFormat class for the job.
Class<?>	getOutputKeyClass() Get the key class for the job output data.
Class<?>	getOutputValueClass() Get the value class for job outputs.
Class<? extends org.apache.hadoop.mapreduce.Partitioner<?,?>>	getPartitionerClass() Get the Partitioner class for the job.
boolean	getProfileEnabled() Get whether the task profiling is enabled.
String	getProfileParams() Get the profiler configuration arguments.
org.apache.hadoop.conf.Configuration.IntegerRanges	getProfileTaskRange(boolean isMap) Get the range of maps or reduces to profile.
org.apache.hadoop.util.Progressable	getProgressible()
Class<? extends org.apache.hadoop.mapreduce.Reducer<?,?,?,?>>	getReducerClass() Get the Reducer class for the job.
org.apache.hadoop.io.RawComparator<?>	getSortComparator() Get the RawComparator comparator used to compare keys.
boolean	getSymlink() This method checks to see if symlinks are to be create for the localized cache files in the current working directory
boolean	getTaskCleanupNeeded() Get whether task-cleanup is needed for the job
String	getUser() Get the reported username for this job.
org.apache.hadoop.fs.Path	getWorkingDirectory() Get the current working directory for the default file system.
void	setJobID(org.apache.hadoop.mapreduce.JobID jobId) Set the JobID.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

conf

protected final org.apache.hadoop.mapred.JobConf conf

credentials

protected final org.apache.hadoop.security.Credentials credentials

Constructor Detail

JobContextImpl

public JobContextImpl(org.apache.hadoop.conf.Configuration conf,
              org.apache.tez.dag.records.TezDAGID dagId)

JobContextImpl

public JobContextImpl(org.apache.hadoop.conf.Configuration conf,
              org.apache.tez.dag.records.TezDAGID dagId,
              org.apache.hadoop.util.Progressable progress)

Method Detail

getConfiguration

public org.apache.hadoop.conf.Configuration getConfiguration()

Return the configuration for the job.

Specified by:

getConfiguration in interface org.apache.hadoop.mapreduce.JobContext

Returns:

the shared configuration object

getJobID

public org.apache.hadoop.mapreduce.JobID getJobID()

Get the unique ID for the job.

Specified by:

getJobID in interface org.apache.hadoop.mapreduce.JobContext

Returns:

the object with the job id

setJobID

public void setJobID(org.apache.hadoop.mapreduce.JobID jobId)

Set the JobID.

getNumReduceTasks

public int getNumReduceTasks()

Get configured the number of reduce tasks for this job. Defaults to 1.

Specified by:

getNumReduceTasks in interface org.apache.hadoop.mapreduce.JobContext

Returns:

the number of reduce tasks for this job.

getWorkingDirectory

public org.apache.hadoop.fs.Path getWorkingDirectory()
                                              throws IOException

Get the current working directory for the default file system.

Specified by:

getWorkingDirectory in interface org.apache.hadoop.mapreduce.JobContext

Returns:

the directory name.

Throws:

IOException

getOutputKeyClass

public Class<?> getOutputKeyClass()

Get the key class for the job output data.

Specified by:

getOutputKeyClass in interface org.apache.hadoop.mapreduce.JobContext

Returns:

the key class for the job output data.

getOutputValueClass

public Class<?> getOutputValueClass()

Get the value class for job outputs.

Specified by:

getOutputValueClass in interface org.apache.hadoop.mapreduce.JobContext

Returns:

the value class for job outputs.

getMapOutputKeyClass

public Class<?> getMapOutputKeyClass()

Get the key class for the map output data. If it is not set, use the (final) output key class. This allows the map output key class to be different than the final output key class.

Specified by:

getMapOutputKeyClass in interface org.apache.hadoop.mapreduce.JobContext

Returns:

the map output key class.

getMapOutputValueClass

public Class<?> getMapOutputValueClass()

Get the value class for the map output data. If it is not set, use the (final) output value class This allows the map output value class to be different than the final output value class.

Specified by:

getMapOutputValueClass in interface org.apache.hadoop.mapreduce.JobContext

Returns:

the map output value class.

getJobName

public String getJobName()

Get the user-specified job name. This is only used to identify the job to the user.

Specified by:

getJobName in interface org.apache.hadoop.mapreduce.JobContext

Returns:

the job's name, defaulting to "".

getInputFormatClass

public Class<? extends org.apache.hadoop.mapreduce.InputFormat<?,?>> getInputFormatClass()
                                                                                  throws ClassNotFoundException

Get the InputFormat class for the job.

Specified by:

getInputFormatClass in interface org.apache.hadoop.mapreduce.JobContext

Returns:

the InputFormat class for the job.

Throws:

ClassNotFoundException

getMapperClass

public Class<? extends org.apache.hadoop.mapreduce.Mapper<?,?,?,?>> getMapperClass()
                                                                            throws ClassNotFoundException

Get the Mapper class for the job.

Specified by:

getMapperClass in interface org.apache.hadoop.mapreduce.JobContext

Returns:

the Mapper class for the job.

Throws:

ClassNotFoundException

getCombinerClass

public Class<? extends org.apache.hadoop.mapreduce.Reducer<?,?,?,?>> getCombinerClass()
                                                                               throws ClassNotFoundException

Get the combiner class for the job.

Specified by:

getCombinerClass in interface org.apache.hadoop.mapreduce.JobContext

Returns:

the combiner class for the job.

Throws:

ClassNotFoundException

getReducerClass

public Class<? extends org.apache.hadoop.mapreduce.Reducer<?,?,?,?>> getReducerClass()
                                                                              throws ClassNotFoundException

Get the Reducer class for the job.

Specified by:

getReducerClass in interface org.apache.hadoop.mapreduce.JobContext

Returns:

the Reducer class for the job.

Throws:

ClassNotFoundException

getOutputFormatClass

public Class<? extends org.apache.hadoop.mapreduce.OutputFormat<?,?>> getOutputFormatClass()
                                                                                    throws ClassNotFoundException

Get the OutputFormat class for the job.

Specified by:

getOutputFormatClass in interface org.apache.hadoop.mapreduce.JobContext

Returns:

the OutputFormat class for the job.

Throws:

ClassNotFoundException

getPartitionerClass

public Class<? extends org.apache.hadoop.mapreduce.Partitioner<?,?>> getPartitionerClass()
                                                                                  throws ClassNotFoundException

Get the Partitioner class for the job.

Specified by:

getPartitionerClass in interface org.apache.hadoop.mapreduce.JobContext

Returns:

the Partitioner class for the job.

Throws:

ClassNotFoundException

getSortComparator

public org.apache.hadoop.io.RawComparator<?> getSortComparator()

Get the RawComparator comparator used to compare keys.

Specified by:

getSortComparator in interface org.apache.hadoop.mapreduce.JobContext

Returns:

the RawComparator comparator used to compare keys.

getJar

public String getJar()

Get the pathname of the job's jar.

Specified by:

getJar in interface org.apache.hadoop.mapreduce.JobContext

Returns:

the pathname

getCombinerKeyGroupingComparator

public org.apache.hadoop.io.RawComparator<?> getCombinerKeyGroupingComparator()

Get the user defined RawComparator comparator for grouping keys of inputs to the combiner.

Specified by:

getCombinerKeyGroupingComparator in interface org.apache.hadoop.mapreduce.JobContext

Returns:

comparator set by the user for grouping values.

See Also:

for details.

getGroupingComparator

public org.apache.hadoop.io.RawComparator<?> getGroupingComparator()

Get the user defined RawComparator comparator for grouping keys of inputs to the reduce.

Specified by:

getGroupingComparator in interface org.apache.hadoop.mapreduce.JobContext

Returns:

comparator set by the user for grouping values.

See Also:

for details.

getJobSetupCleanupNeeded

public boolean getJobSetupCleanupNeeded()

Get whether job-setup and job-cleanup is needed for the job

Specified by:

getJobSetupCleanupNeeded in interface org.apache.hadoop.mapreduce.JobContext

Returns:

boolean

getTaskCleanupNeeded

public boolean getTaskCleanupNeeded()

Get whether task-cleanup is needed for the job

Specified by:

getTaskCleanupNeeded in interface org.apache.hadoop.mapreduce.JobContext

Returns:

boolean

getSymlink

public boolean getSymlink()

This method checks to see if symlinks are to be create for the localized cache files in the current working directory

Specified by:

getSymlink in interface org.apache.hadoop.mapreduce.JobContext

Returns:

true if symlinks are to be created- else return false

getArchiveClassPaths

public org.apache.hadoop.fs.Path[] getArchiveClassPaths()

Get the archive entries in classpath as an array of Path

Specified by:

getArchiveClassPaths in interface org.apache.hadoop.mapreduce.JobContext

getCacheArchives

public URI[] getCacheArchives()
                       throws IOException

Get cache archives set in the Configuration

Specified by:

getCacheArchives in interface org.apache.hadoop.mapreduce.JobContext

Returns:

A URI array of the caches set in the Configuration

Throws:

IOException

getCacheFiles

public URI[] getCacheFiles()
                    throws IOException

Get cache files set in the Configuration

Specified by:

getCacheFiles in interface org.apache.hadoop.mapreduce.JobContext

Returns:

A URI array of the files set in the Configuration

Throws:

IOException

getLocalCacheArchives

public org.apache.hadoop.fs.Path[] getLocalCacheArchives()
                                                  throws IOException

Return the path array of the localized caches

Specified by:

getLocalCacheArchives in interface org.apache.hadoop.mapreduce.JobContext

Returns:

A path array of localized caches

Throws:

IOException

getLocalCacheFiles

public org.apache.hadoop.fs.Path[] getLocalCacheFiles()
                                               throws IOException

Return the path array of the localized files

Specified by:

getLocalCacheFiles in interface org.apache.hadoop.mapreduce.JobContext

Returns:

A path array of localized files

Throws:

IOException

getFileClassPaths

public org.apache.hadoop.fs.Path[] getFileClassPaths()

Get the file entries in classpath as an array of Path

Specified by:

getFileClassPaths in interface org.apache.hadoop.mapreduce.JobContext

getArchiveTimestamps

public String[] getArchiveTimestamps()

Get the timestamps of the archives. Used by internal DistributedCache and MapReduce code.

Specified by:

getArchiveTimestamps in interface org.apache.hadoop.mapreduce.JobContext

Returns:

a string array of timestamps

Throws:

IOException

getFileTimestamps

public String[] getFileTimestamps()

Get the timestamps of the files. Used by internal DistributedCache and MapReduce code.

Specified by:

getFileTimestamps in interface org.apache.hadoop.mapreduce.JobContext

Returns:

a string array of timestamps

Throws:

IOException

getMaxMapAttempts

public int getMaxMapAttempts()

Get the configured number of maximum attempts that will be made to run a map task, as specified by the mapred.map.max.attempts property. If this property is not already set, the default is 4 attempts.

Specified by:

getMaxMapAttempts in interface org.apache.hadoop.mapreduce.JobContext

Returns:

the max number of attempts per map task.

getMaxReduceAttempts

public int getMaxReduceAttempts()

Get the configured number of maximum attempts that will be made to run a reduce task, as specified by the mapred.reduce.max.attempts property. If this property is not already set, the default is 4 attempts.

Specified by:

getMaxReduceAttempts in interface org.apache.hadoop.mapreduce.JobContext

Returns:

the max number of attempts per reduce task.

getProfileEnabled

public boolean getProfileEnabled()

Get whether the task profiling is enabled.

Specified by:

getProfileEnabled in interface org.apache.hadoop.mapreduce.JobContext

Returns:

true if some tasks will be profiled

getProfileParams

public String getProfileParams()

Get the profiler configuration arguments. The default value for this property is "-agentlib:hprof=cpu=samples,heap=sites,force=n,thread=y,verbose=n,file=%s"

Specified by:

getProfileParams in interface org.apache.hadoop.mapreduce.JobContext

Returns:

the parameters to pass to the task child to configure profiling

getProfileTaskRange

public org.apache.hadoop.conf.Configuration.IntegerRanges getProfileTaskRange(boolean isMap)

Get the range of maps or reduces to profile.

Specified by:

getProfileTaskRange in interface org.apache.hadoop.mapreduce.JobContext

Parameters:

isMap - is the task a map?

Returns:

the task ranges

getUser

public String getUser()

Get the reported username for this job.

Specified by:

getUser in interface org.apache.hadoop.mapreduce.JobContext

Returns:

the username

getCredentials

public org.apache.hadoop.security.Credentials getCredentials()

Specified by:

getCredentials in interface org.apache.hadoop.mapreduce.JobContext

getJobConf

public org.apache.hadoop.mapred.JobConf getJobConf()

getProgressible

public org.apache.hadoop.util.Progressable getProgressible()