co.cask.cdap.etl.spark.streaming.function

Class DynamicJoinOn<JOIN_KEY,T>

java.lang.Object

co.cask.cdap.etl.spark.streaming.function.DynamicJoinOn<JOIN_KEY,T>

Type Parameters:

JOIN_KEY - type of join key

T - type of input object

All Implemented Interfaces:

Serializable, org.apache.spark.api.java.function.Function2<org.apache.spark.api.java.JavaRDD<T>,org.apache.spark.streaming.Time,org.apache.spark.api.java.JavaPairRDD<JOIN_KEY,T>>

public class DynamicJoinOn<JOIN_KEY,T>
extends Object
implements org.apache.spark.api.java.function.Function2<org.apache.spark.api.java.JavaRDD<T>,org.apache.spark.streaming.Time,org.apache.spark.api.java.JavaPairRDD<JOIN_KEY,T>>

Serializable function that can be used to perform the joinOn part of a Joiner. Dynamically instantiates the relevant Joiner plugin to ensure that code changes are picked up and to ensure that macro substitution occurs.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
DynamicJoinOn(DynamicDriverContext dynamicDriverContext, String inputStageName)

Method Summary

Modifier and Type	Method and Description
org.apache.spark.api.java.JavaPairRDD<JOIN_KEY,T>	call(org.apache.spark.api.java.JavaRDD<T> input, org.apache.spark.streaming.Time batchTime)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DynamicJoinOn

public DynamicJoinOn(DynamicDriverContext dynamicDriverContext,
                     String inputStageName)

Method Detail

call

public org.apache.spark.api.java.JavaPairRDD<JOIN_KEY,T> call(org.apache.spark.api.java.JavaRDD<T> input,
                                                              org.apache.spark.streaming.Time batchTime)
                                                       throws Exception

Specified by:

call in interface org.apache.spark.api.java.function.Function2<org.apache.spark.api.java.JavaRDD<T>,org.apache.spark.streaming.Time,org.apache.spark.api.java.JavaPairRDD<JOIN_KEY,T>>

Throws:

Exception