co.cask.cdap.etl.spark.plugin

Class WrappedSparkSink<IN>

java.lang.Object

co.cask.cdap.etl.api.batch.BatchConfigurable<SparkPluginContext>

co.cask.cdap.etl.api.batch.SparkSink<IN>

co.cask.cdap.etl.spark.plugin.WrappedSparkSink<IN>

Type Parameters:

IN - The type of input record to the SparkSink.

All Implemented Interfaces:

PipelineConfigurable, SubmitterLifecycle<SparkPluginContext>, Serializable

public class WrappedSparkSink<IN>
extends SparkSink<IN>

Wrapper around a SparkCompute that makes sure logging, classloading, and other pipeline capabilities are setup correctly.

See Also:

Serialized Form

Field Summary

Fields inherited from class co.cask.cdap.etl.api.batch.SparkSink

PLUGIN_TYPE

Constructor Summary

Constructors

Constructor and Description
WrappedSparkSink(SparkSink<IN> sink, Caller caller)

Method Summary

Modifier and Type	Method and Description
void	configurePipeline(PipelineConfigurer pipelineConfigurer)
void	onRunFinish(boolean succeeded, SparkPluginContext context)
void	prepareRun(SparkPluginContext context)
void	run(SparkExecutionPluginContext context, org.apache.spark.api.java.JavaRDD<IN> input)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

WrappedSparkSink

public WrappedSparkSink(SparkSink<IN> sink,
                        Caller caller)

Method Detail

configurePipeline

public void configurePipeline(PipelineConfigurer pipelineConfigurer)
                       throws IllegalArgumentException

Specified by:

configurePipeline in interface PipelineConfigurable

Overrides:

configurePipeline in class BatchConfigurable<SparkPluginContext>

Throws:

IllegalArgumentException

prepareRun

public void prepareRun(SparkPluginContext context)
                throws Exception

Specified by:

prepareRun in interface SubmitterLifecycle<SparkPluginContext>

Specified by:

prepareRun in class BatchConfigurable<SparkPluginContext>

Throws:

Exception

onRunFinish

public void onRunFinish(boolean succeeded,
                        SparkPluginContext context)

Specified by:

onRunFinish in interface SubmitterLifecycle<SparkPluginContext>

Overrides:

onRunFinish in class BatchConfigurable<SparkPluginContext>

run

public void run(SparkExecutionPluginContext context,
                org.apache.spark.api.java.JavaRDD<IN> input)
         throws Exception

Specified by:

run in class SparkSink<IN>

Throws:

Exception