co.cask.cdap.etl.spark.streaming.function

Class DynamicTransform<T>

java.lang.Object

co.cask.cdap.etl.spark.streaming.function.DynamicTransform<T>

Type Parameters:

T - type of input object

All Implemented Interfaces:

Serializable, org.apache.spark.api.java.function.Function2<org.apache.spark.api.java.JavaRDD<T>,org.apache.spark.streaming.Time,org.apache.spark.api.java.JavaRDD<RecordInfo<Object>>>

public class DynamicTransform<T>
extends Object
implements org.apache.spark.api.java.function.Function2<org.apache.spark.api.java.JavaRDD<T>,org.apache.spark.streaming.Time,org.apache.spark.api.java.JavaRDD<RecordInfo<Object>>>

Serializable function that can be used to perform a flat map on a DStream. Dynamically instantiates the Transform plugin used to perform the flat map to ensure that code changes are picked up and to ensure that macro substitution occurs.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
DynamicTransform(DynamicDriverContext dynamicDriverContext, boolean isMultiOutput)

Method Summary

Modifier and Type	Method and Description
org.apache.spark.api.java.JavaRDD<RecordInfo<Object>>	call(org.apache.spark.api.java.JavaRDD<T> input, org.apache.spark.streaming.Time batchTime)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DynamicTransform

public DynamicTransform(DynamicDriverContext dynamicDriverContext,
                        boolean isMultiOutput)

Method Detail

call

public org.apache.spark.api.java.JavaRDD<RecordInfo<Object>> call(org.apache.spark.api.java.JavaRDD<T> input,
                                                                  org.apache.spark.streaming.Time batchTime)
                                                           throws Exception

Specified by:

call in interface org.apache.spark.api.java.function.Function2<org.apache.spark.api.java.JavaRDD<T>,org.apache.spark.streaming.Time,org.apache.spark.api.java.JavaRDD<RecordInfo<Object>>>

Throws:

Exception