org.apache.hadoop.hbase.mapreduce.hadoopbackport
Class InputSampler<K,V>

java.lang.Object
  org.apache.hadoop.conf.Configured
      org.apache.hadoop.hbase.mapreduce.hadoopbackport.InputSampler<K,V>

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, org.apache.hadoop.util.Tool

public class InputSampler<K,V>
extends org.apache.hadoop.conf.Configured
implements org.apache.hadoop.util.Tool

Utility for collecting samples and writing a partition file for TotalOrderPartitioner. This is an identical copy of o.a.h.mapreduce.lib.partition.TotalOrderPartitioner from Hadoop trunk at r961542, with the exception of replacing TaskAttemptContextImpl with TaskAttemptContext.

Nested Class Summary

static class	InputSampler.IntervalSampler<K,V> Sample from s splits at regular intervals.
static class	InputSampler.RandomSampler<K,V> Sample from random points in the input.
static interface	InputSampler.Sampler<K,V> Interface to sample using an InputFormat.
static class	InputSampler.SplitSampler<K,V> Samples the first n records from s splits.

Constructor Summary

InputSampler(org.apache.hadoop.conf.Configuration conf)

Method Summary

static void	main(java.lang.String[] args)
int	run(java.lang.String[] args) Driver for InputSampler from the command line.
static <K,V> void	writePartitionFile(org.apache.hadoop.mapreduce.Job job, InputSampler.Sampler<K,V> sampler) Write a partition file for the given job, using the Sampler provided.

Methods inherited from class org.apache.hadoop.conf.Configured

getConf, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.conf.Configurable

getConf, setConf

Constructor Detail

InputSampler

public InputSampler(org.apache.hadoop.conf.Configuration conf)

Method Detail

writePartitionFile

public static <K,V> void writePartitionFile(org.apache.hadoop.mapreduce.Job job,
                                            InputSampler.Sampler<K,V> sampler)
                               throws java.io.IOException,
                                      java.lang.ClassNotFoundException,
                                      java.lang.InterruptedException

Write a partition file for the given job, using the Sampler provided. Queries the sampler for a sample keyset, sorts by the output key comparator, selects the keys for each rank, and writes to the destination returned from TotalOrderPartitioner.getPartitionFile(org.apache.hadoop.conf.Configuration).

Throws:

java.io.IOException

java.lang.ClassNotFoundException

java.lang.InterruptedException

run

public int run(java.lang.String[] args)
        throws java.lang.Exception

Driver for InputSampler from the command line. Configures a JobConf instance and calls writePartitionFile(org.apache.hadoop.mapreduce.Job, org.apache.hadoop.hbase.mapreduce.hadoopbackport.InputSampler.Sampler).

Specified by:

run in interface org.apache.hadoop.util.Tool

Throws:

java.lang.Exception

main

public static void main(java.lang.String[] args)
                 throws java.lang.Exception

Throws:

java.lang.Exception