co.cask.cdap.etl.spark.streaming

Class DynamicDriverContext

java.lang.Object

co.cask.cdap.etl.spark.streaming.DynamicDriverContext

All Implemented Interfaces:

Externalizable, Serializable

public class DynamicDriverContext
extends Object
implements Externalizable

Serializable context that can be used to dynamically instantiate plugins from a driver context. Used for Spark Streaming programs that use checkpoints. Basically just a wrapper around the PluginFunctionContext, which is used in Spark executor closures. The reason we need this on top of the PluginFunctionContext is because JavaSparkExecutionContext can only be used in Spark driver closures and not in spark executor closurers, and we need JavaSparkExecutionContext to make sure runtime arguments and logical start time are fetched correctly.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
DynamicDriverContext()
DynamicDriverContext(StageSpec stageSpec, JavaSparkExecutionContext sec, StageStatisticsCollector collector)

Method Summary

Modifier and Type	Method and Description
PluginFunctionContext	getPluginFunctionContext()
JavaSparkExecutionContext	getSparkExecutionContext()
void	readExternal(ObjectInput in)
void	writeExternal(ObjectOutput out)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DynamicDriverContext

public DynamicDriverContext()

DynamicDriverContext

public DynamicDriverContext(StageSpec stageSpec,
                            JavaSparkExecutionContext sec,
                            StageStatisticsCollector collector)

Method Detail

writeExternal

public void writeExternal(ObjectOutput out)
                   throws IOException

Specified by:

writeExternal in interface Externalizable

Throws:

IOException

readExternal

public void readExternal(ObjectInput in)
                  throws IOException,
                         ClassNotFoundException

Specified by:

readExternal in interface Externalizable

Throws:

IOException

ClassNotFoundException

getSparkExecutionContext

public JavaSparkExecutionContext getSparkExecutionContext()

getPluginFunctionContext

public PluginFunctionContext getPluginFunctionContext()