co.cask.cdap.etl.spark.streaming.function

Class DynamicAggregatorAggregate<GROUP_KEY,GROUP_VAL,OUT>

java.lang.Object

co.cask.cdap.etl.spark.streaming.function.DynamicAggregatorAggregate<GROUP_KEY,GROUP_VAL,OUT>

Type Parameters:

GROUP_KEY - type of group key

GROUP_VAL - type of group val

OUT - type of output object

All Implemented Interfaces:

Serializable, org.apache.spark.api.java.function.Function2<org.apache.spark.api.java.JavaPairRDD<GROUP_KEY,Iterable<GROUP_VAL>>,org.apache.spark.streaming.Time,org.apache.spark.api.java.JavaRDD<RecordInfo<Object>>>

public class DynamicAggregatorAggregate<GROUP_KEY,GROUP_VAL,OUT>
extends Object
implements org.apache.spark.api.java.function.Function2<org.apache.spark.api.java.JavaPairRDD<GROUP_KEY,Iterable<GROUP_VAL>>,org.apache.spark.streaming.Time,org.apache.spark.api.java.JavaRDD<RecordInfo<Object>>>

Serializable function that can be used to perform the aggregate part of an Aggregator. Dynamically instantiates the Aggregator plugin used to ensure that code changes are picked up and to ensure that macro substitution occurs.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
DynamicAggregatorAggregate(DynamicDriverContext dynamicDriverContext)

Method Summary

Modifier and Type	Method and Description
org.apache.spark.api.java.JavaRDD<RecordInfo<Object>>	call(org.apache.spark.api.java.JavaPairRDD<GROUP_KEY,Iterable<GROUP_VAL>> input, org.apache.spark.streaming.Time batchTime)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DynamicAggregatorAggregate

public DynamicAggregatorAggregate(DynamicDriverContext dynamicDriverContext)

Method Detail

call

public org.apache.spark.api.java.JavaRDD<RecordInfo<Object>> call(org.apache.spark.api.java.JavaPairRDD<GROUP_KEY,Iterable<GROUP_VAL>> input,
                                                                  org.apache.spark.streaming.Time batchTime)
                                                           throws Exception

Specified by:

call in interface org.apache.spark.api.java.function.Function2<org.apache.spark.api.java.JavaPairRDD<GROUP_KEY,Iterable<GROUP_VAL>>,org.apache.spark.streaming.Time,org.apache.spark.api.java.JavaRDD<RecordInfo<Object>>>

Throws:

Exception