co.cask.cdap.etl.spark.streaming.function

Class StreamingBatchSinkFunction<T>

java.lang.Object

co.cask.cdap.etl.spark.streaming.function.StreamingBatchSinkFunction<T>

Type Parameters:

T - type of object in the rdd

All Implemented Interfaces:

Serializable, org.apache.spark.api.java.function.Function2<org.apache.spark.api.java.JavaRDD<T>,org.apache.spark.streaming.Time,Void>

public class StreamingBatchSinkFunction<T>
extends Object
implements org.apache.spark.api.java.function.Function2<org.apache.spark.api.java.JavaRDD<T>,org.apache.spark.streaming.Time,Void>

Function used to write a batch of data to a batch sink for use with a JavaDStream. note: not using foreachRDD(VoidFunction2) method, because spark 1.3 doesn't have VoidFunction2.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
StreamingBatchSinkFunction(org.apache.spark.api.java.function.PairFlatMapFunction<T,Object,Object> sinkFunction, JavaSparkExecutionContext sec, StageSpec stageSpec)

Method Summary

Modifier and Type	Method and Description
Void	call(org.apache.spark.api.java.JavaRDD<T> data, org.apache.spark.streaming.Time batchTime)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

StreamingBatchSinkFunction

public StreamingBatchSinkFunction(org.apache.spark.api.java.function.PairFlatMapFunction<T,Object,Object> sinkFunction,
                                  JavaSparkExecutionContext sec,
                                  StageSpec stageSpec)

Method Detail

call

public Void call(org.apache.spark.api.java.JavaRDD<T> data,
                 org.apache.spark.streaming.Time batchTime)
          throws Exception

Specified by:

call in interface org.apache.spark.api.java.function.Function2<org.apache.spark.api.java.JavaRDD<T>,org.apache.spark.streaming.Time,Void>

Throws:

Exception