com.stratio.deep.rdd

Class CassandraEntityRDD<T extends IDeepType>

java.lang.Object

org.apache.spark.rdd.RDD<T>

com.stratio.deep.rdd.CassandraRDD<T>

com.stratio.deep.rdd.CassandraEntityRDD<T>

All Implemented Interfaces:

Serializable, org.apache.spark.Logging

public final class CassandraEntityRDD<T extends IDeepType>
extends CassandraRDD<T>

Stratio's implementation of an RDD reading and writing data from and to Apache Cassandra. This implementation uses Cassandra's Hadoop API.

We do not use Map as key and value objects, since ByteBuffer is not serializable.

Author:

Luca Rosellini

See Also:

Serialized Form

Field Summary

Fields inherited from class com.stratio.deep.rdd.CassandraRDD

config

Fields inherited from class org.apache.spark.rdd.RDD

org$apache$spark$rdd$RDD$$evidence$1

Constructor Summary

Constructors

Constructor and Description
CassandraEntityRDD(org.apache.spark.SparkContext sc, IDeepJobConfig<T> config) This constructor should not be called explicitly. Use DeepSparkContext instead to create an RDD.

Method Summary

Methods

Modifier and Type	Method and Description
protected T	transformElement(Pair<Map<String,ByteBuffer>,Map<String,ByteBuffer>> elem) Transform a row coming from the Cassandra's API to an element of type .

Methods inherited from class com.stratio.deep.rdd.CassandraRDD

compute, cql3SaveRDDToCassandra, getComputeCallback, getPartitions, getPreferredLocations, saveRDDToCassandra, saveRDDToCassandra

Methods inherited from class org.apache.spark.rdd.RDD

$plus$plus, aggregate, cache, cartesian, checkpoint, checkpointData_$eq, checkpointData, clearDependencies, coalesce, coalesce$default$2, collect, collect, collectPartitions, computeOrReadCheckpoint, conf, context, count, countApprox, countApprox$default$2, countApproxDistinct, countApproxDistinct$default$1, countByValue, countByValueApprox, countByValueApprox$default$2, dependencies, distinct, distinct, doCheckpoint, elementClassTag, filter, filterWith, first, firstParent, flatMap, flatMapWith, flatMapWith$default$2, fold, foreach, foreachPartition, foreachWith, generator_$eq, generator, getCheckpointFile, getDependencies, getStorageLevel, glom, groupBy, groupBy, groupBy, id, isCheckpointed, isTraceEnabled, iterator, keyBy, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logTrace, logTrace, logWarning, logWarning, map, mapPartitions, mapPartitions$default$2, mapPartitionsWithContext, mapPartitionsWithContext$default$2, mapPartitionsWithIndex, mapPartitionsWithIndex$default$2, mapPartitionsWithSplit, mapPartitionsWithSplit$default$2, mapWith, mapWith$default$2, markCheckpointed, name_$eq, name, org$apache$spark$Logging$$log__$eq, org$apache$spark$Logging$$log_, org$apache$spark$rdd$RDD$$countPartition$1, org$apache$spark$rdd$RDD$$debugString$1, org$apache$spark$rdd$RDD$$dependencies__$eq, org$apache$spark$rdd$RDD$$dependencies_, org$apache$spark$rdd$RDD$$mergeMaps$1, org$apache$spark$rdd$RDD$$partitions__$eq, org$apache$spark$rdd$RDD$$partitions_, origin, partitioner, partitions, persist, persist, pipe, pipe, pipe, pipe$default$2, pipe$default$3, pipe$default$4, preferredLocations, reduce, repartition, sample, saveAsObjectFile, saveAsTextFile, saveAsTextFile, setGenerator, setName, sparkContext, subtract, subtract, subtract, take, takeOrdered, takeSample, toArray, toDebugString, toJavaRDD, top, toString, union, unpersist, unpersist$default$1, zip, zipPartitions, zipPartitions, zipPartitions, zipPartitions, zipPartitions, zipPartitions

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

CassandraEntityRDD

public CassandraEntityRDD(org.apache.spark.SparkContext sc,
                  IDeepJobConfig<T> config)

This constructor should not be called explicitly.
Use DeepSparkContext instead to create an RDD.

Parameters:

sc -

config -

Method Detail

transformElement

protected T transformElement(Pair<Map<String,ByteBuffer>,Map<String,ByteBuffer>> elem)

Transform a row coming from the Cassandra's API to an element of type .

Specified by:

transformElement in class CassandraRDD<T extends IDeepType>

Parameters:

elem - the element to transform.

Returns:

the transformed element.