co.cask.cdap.etl.spark.streaming.function

Class ComputeTransformFunction<T,U>

java.lang.Object

co.cask.cdap.etl.spark.streaming.function.ComputeTransformFunction<T,U>

Type Parameters:

T - type of object in the input rdd

U - type of object in the output rdd

All Implemented Interfaces:

Serializable, org.apache.spark.api.java.function.Function2<org.apache.spark.api.java.JavaRDD<T>,org.apache.spark.streaming.Time,org.apache.spark.api.java.JavaRDD<U>>

public class ComputeTransformFunction<T,U>
extends Object
implements org.apache.spark.api.java.function.Function2<org.apache.spark.api.java.JavaRDD<T>,org.apache.spark.streaming.Time,org.apache.spark.api.java.JavaRDD<U>>

Function used to implement a SparkCompute stage in a DStream.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
ComputeTransformFunction(JavaSparkExecutionContext sec, StageSpec stageSpec, SparkCompute<T,U> compute)

Method Summary

Modifier and Type	Method and Description
org.apache.spark.api.java.JavaRDD<U>	call(org.apache.spark.api.java.JavaRDD<T> data, org.apache.spark.streaming.Time batchTime)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ComputeTransformFunction

public ComputeTransformFunction(JavaSparkExecutionContext sec,
                                StageSpec stageSpec,
                                SparkCompute<T,U> compute)

Method Detail

call

public org.apache.spark.api.java.JavaRDD<U> call(org.apache.spark.api.java.JavaRDD<T> data,
                                                 org.apache.spark.streaming.Time batchTime)
                                          throws Exception

Specified by:

call in interface org.apache.spark.api.java.function.Function2<org.apache.spark.api.java.JavaRDD<T>,org.apache.spark.streaming.Time,org.apache.spark.api.java.JavaRDD<U>>

Throws:

Exception