co.cask.cdap.etl.spark.streaming.function

Class DynamicAggregatorGroupBy<GROUP_KEY,GROUP_VAL>

java.lang.Object

co.cask.cdap.etl.spark.streaming.function.DynamicAggregatorGroupBy<GROUP_KEY,GROUP_VAL>

Type Parameters:

GROUP_KEY - type of group key

GROUP_VAL - type of group val

All Implemented Interfaces:

Serializable, org.apache.spark.api.java.function.Function2<org.apache.spark.api.java.JavaRDD<GROUP_VAL>,org.apache.spark.streaming.Time,org.apache.spark.api.java.JavaPairRDD<GROUP_KEY,GROUP_VAL>>

public class DynamicAggregatorGroupBy<GROUP_KEY,GROUP_VAL>
extends Object
implements org.apache.spark.api.java.function.Function2<org.apache.spark.api.java.JavaRDD<GROUP_VAL>,org.apache.spark.streaming.Time,org.apache.spark.api.java.JavaPairRDD<GROUP_KEY,GROUP_VAL>>

Serializable function that can be used to perform the group by part of an Aggregator. Dynamically instantiates the Aggregator plugin used to ensure that code changes are picked up and to ensure that macro substitution occurs.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
DynamicAggregatorGroupBy(DynamicDriverContext dynamicDriverContext)

Method Summary

Modifier and Type	Method and Description
org.apache.spark.api.java.JavaPairRDD<GROUP_KEY,GROUP_VAL>	call(org.apache.spark.api.java.JavaRDD<GROUP_VAL> input, org.apache.spark.streaming.Time batchTime)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DynamicAggregatorGroupBy

public DynamicAggregatorGroupBy(DynamicDriverContext dynamicDriverContext)

Method Detail

call

public org.apache.spark.api.java.JavaPairRDD<GROUP_KEY,GROUP_VAL> call(org.apache.spark.api.java.JavaRDD<GROUP_VAL> input,
                                                                       org.apache.spark.streaming.Time batchTime)
                                                                throws Exception

Specified by:

call in interface org.apache.spark.api.java.function.Function2<org.apache.spark.api.java.JavaRDD<GROUP_VAL>,org.apache.spark.streaming.Time,org.apache.spark.api.java.JavaPairRDD<GROUP_KEY,GROUP_VAL>>

Throws:

Exception