org.apache.hadoop.hbase.mapreduce
Class MultiTableOutputFormat

java.lang.Object
  org.apache.hadoop.mapreduce.OutputFormat<ImmutableBytesWritable,org.apache.hadoop.io.Writable>
      org.apache.hadoop.hbase.mapreduce.MultiTableOutputFormat

public class MultiTableOutputFormat
extends org.apache.hadoop.mapreduce.OutputFormat<ImmutableBytesWritable,org.apache.hadoop.io.Writable>

Hadoop output format that writes to one or more HBase tables. The key is taken to be the table name while the output value must be either a Put or a Delete instance. All tables must already exist, and all Puts and Deletes must reference only valid column families.

Write-ahead logging (HLog) for Puts can be disabled by setting WAL_PROPERTY to WAL_OFF. Default value is WAL_ON. Note that disabling write-ahead logging is only appropriate for jobs where loss of data due to region server failure can be tolerated (for example, because it is easy to rerun a bulk import).

Nested Class Summary

protected static class

MultiTableOutputFormat.MultiTableRecordWriter Record writer for outputting to multiple HTables.

Field Summary

static boolean	WAL_OFF Property value to disable write-ahead logging
static boolean	WAL_ON Property value to use write-ahead logging
static java.lang.String	WAL_PROPERTY Set this to WAL_OFF to turn off write-ahead logging (HLog)

Constructor Summary

MultiTableOutputFormat()

Method Summary

void	checkOutputSpecs(org.apache.hadoop.mapreduce.JobContext context)
org.apache.hadoop.mapreduce.OutputCommitter	getOutputCommitter(org.apache.hadoop.mapreduce.TaskAttemptContext context)
org.apache.hadoop.mapreduce.RecordWriter<ImmutableBytesWritable,org.apache.hadoop.io.Writable>	getRecordWriter(org.apache.hadoop.mapreduce.TaskAttemptContext context)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

WAL_PROPERTY

public static final java.lang.String WAL_PROPERTY

Set this to WAL_OFF to turn off write-ahead logging (HLog)

See Also:

Constant Field Values

WAL_ON

public static final boolean WAL_ON

Property value to use write-ahead logging

See Also:

Constant Field Values

WAL_OFF

public static final boolean WAL_OFF

Property value to disable write-ahead logging

See Also:

Constant Field Values

Constructor Detail

MultiTableOutputFormat

public MultiTableOutputFormat()

Method Detail

checkOutputSpecs

public void checkOutputSpecs(org.apache.hadoop.mapreduce.JobContext context)
                      throws java.io.IOException,
                             java.lang.InterruptedException

Specified by:

checkOutputSpecs in class org.apache.hadoop.mapreduce.OutputFormat<ImmutableBytesWritable,org.apache.hadoop.io.Writable>

Throws:

java.io.IOException

java.lang.InterruptedException

getOutputCommitter

public org.apache.hadoop.mapreduce.OutputCommitter getOutputCommitter(org.apache.hadoop.mapreduce.TaskAttemptContext context)
                                                               throws java.io.IOException,
                                                                      java.lang.InterruptedException

Specified by:

getOutputCommitter in class org.apache.hadoop.mapreduce.OutputFormat<ImmutableBytesWritable,org.apache.hadoop.io.Writable>

Throws:

java.io.IOException

java.lang.InterruptedException

getRecordWriter

public org.apache.hadoop.mapreduce.RecordWriter<ImmutableBytesWritable,org.apache.hadoop.io.Writable> getRecordWriter(org.apache.hadoop.mapreduce.TaskAttemptContext context)
                                                                                                               throws java.io.IOException,
                                                                                                                      java.lang.InterruptedException

Specified by:

getRecordWriter in class org.apache.hadoop.mapreduce.OutputFormat<ImmutableBytesWritable,org.apache.hadoop.io.Writable>

Throws:

java.io.IOException

java.lang.InterruptedException