co.cask.cdap.etl.spark

Class SparkStageStatisticsCollector

java.lang.Object

co.cask.cdap.etl.spark.SparkStageStatisticsCollector

All Implemented Interfaces:

StageStatisticsCollector, Serializable

public class SparkStageStatisticsCollector
extends Object
implements StageStatisticsCollector, Serializable

Implementation of StageStatisticsCollector for batch spark pipelines.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
SparkStageStatisticsCollector(org.apache.spark.api.java.JavaSparkContext jsc)

Method Summary

Modifier and Type	Method and Description
long	getErrorRecordCount()
long	getInputRecordCount()
long	getOutputRecordCount()
void	incrementErrorRecordCount()
void	incrementInputRecordCount()
void	incrementOutputRecordCount()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SparkStageStatisticsCollector

public SparkStageStatisticsCollector(org.apache.spark.api.java.JavaSparkContext jsc)

Method Detail

incrementInputRecordCount

public void incrementInputRecordCount()

Specified by:

incrementInputRecordCount in interface StageStatisticsCollector

incrementOutputRecordCount

public void incrementOutputRecordCount()

Specified by:

incrementOutputRecordCount in interface StageStatisticsCollector

incrementErrorRecordCount

public void incrementErrorRecordCount()

Specified by:

incrementErrorRecordCount in interface StageStatisticsCollector

getInputRecordCount

public long getInputRecordCount()

getOutputRecordCount

public long getOutputRecordCount()

getErrorRecordCount

public long getErrorRecordCount()