co.cask.cdap.data2.dataset2.lib.partitioned

Class SimplePartitionConsumer

java.lang.Object

co.cask.cdap.data2.dataset2.lib.partitioned.SimplePartitionConsumer

public class SimplePartitionConsumer
extends Object

A simple, consumer for Partitions of a PartitionedFileSet which maintains state in memory.

Constructor Summary

Constructors

Constructor and Description
SimplePartitionConsumer(co.cask.cdap.api.dataset.lib.PartitionedFileSet partitionedFileSet) Creates an instance of a SimplePartitionConsumer which begins consuming from the beginning.

Method Summary

Methods

Modifier and Type	Method and Description
List<co.cask.cdap.api.dataset.lib.PartitionDetail>	consumePartitions()
List<co.cask.cdap.api.dataset.lib.PartitionDetail>	consumePartitions(int limit)
List<co.cask.cdap.api.dataset.lib.PartitionDetail>	consumePartitions(int limit, co.cask.cdap.api.Predicate<co.cask.cdap.api.dataset.lib.PartitionDetail> predicate)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SimplePartitionConsumer

public SimplePartitionConsumer(co.cask.cdap.api.dataset.lib.PartitionedFileSet partitionedFileSet)

Creates an instance of a SimplePartitionConsumer which begins consuming from the beginning.

Parameters:

partitionedFileSet - the PartitionedFileSet to consume from

Method Detail

consumePartitions

public List<co.cask.cdap.api.dataset.lib.PartitionDetail> consumePartitions()

Returns:

a list of Partitions of the underlying PartitionedFileSet created since the last call to this method. This excludes partitions created in in-progress transactions including the one in which the call to this method is made.

consumePartitions

public List<co.cask.cdap.api.dataset.lib.PartitionDetail> consumePartitions(int limit)

Parameters:

limit - limit to be applied while consuming partitions

Returns:

a list of Partitions of the underlying PartitionedFileSet created since the last call to this method. This excludes partitions created in in-progress transactions including the one in which the call to this method is made.

consumePartitions

public List<co.cask.cdap.api.dataset.lib.PartitionDetail> consumePartitions(int limit,
                                                                   co.cask.cdap.api.Predicate<co.cask.cdap.api.dataset.lib.PartitionDetail> predicate)

Parameters:

limit - limit to be applied while consuming partitions

predicate - predicate to be applied while consuming partitions

Returns:

a list of Partitions of the underlying PartitionedFileSet created since the last call to this method. This excludes partitions created in in-progress transactions including the one in which the call to this method is made.