co.cask.cdap.etl.spark.plugin

Class WrappedSparkCompute<IN,OUT>

java.lang.Object

co.cask.cdap.etl.api.batch.SparkCompute<IN,OUT>

co.cask.cdap.etl.spark.plugin.WrappedSparkCompute<IN,OUT>

Type Parameters:

IN - Type of input object

OUT - Type of output object

All Implemented Interfaces:

PipelineConfigurable, Serializable

public class WrappedSparkCompute<IN,OUT>
extends SparkCompute<IN,OUT>

Wrapper around a SparkCompute that makes sure logging, classloading, and other pipeline capabilities are setup correctly.

See Also:

Serialized Form

Field Summary

Fields inherited from class co.cask.cdap.etl.api.batch.SparkCompute

PLUGIN_TYPE

Constructor Summary

Constructors

Constructor and Description
WrappedSparkCompute(SparkCompute<IN,OUT> compute, Caller caller)

Method Summary

Modifier and Type	Method and Description
void	configurePipeline(PipelineConfigurer pipelineConfigurer)
void	initialize(SparkExecutionPluginContext context)
org.apache.spark.api.java.JavaRDD<OUT>	transform(SparkExecutionPluginContext context, org.apache.spark.api.java.JavaRDD<IN> input)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

WrappedSparkCompute

public WrappedSparkCompute(SparkCompute<IN,OUT> compute,
                           Caller caller)

Method Detail

configurePipeline

public void configurePipeline(PipelineConfigurer pipelineConfigurer)
                       throws IllegalArgumentException

Specified by:

configurePipeline in interface PipelineConfigurable

Overrides:

configurePipeline in class SparkCompute<IN,OUT>

Throws:

IllegalArgumentException

initialize

public void initialize(SparkExecutionPluginContext context)
                throws Exception

Overrides:

initialize in class SparkCompute<IN,OUT>

Throws:

Exception

transform

public org.apache.spark.api.java.JavaRDD<OUT> transform(SparkExecutionPluginContext context,
                                                        org.apache.spark.api.java.JavaRDD<IN> input)
                                                 throws Exception

Specified by:

transform in class SparkCompute<IN,OUT>

Throws:

Exception