| Package | Description |
|---|---|
| co.cask.cdap.data.runtime |
This package contains implementations and guice-bindings needed for data-fabric runtime.
|
| co.cask.cdap.data.stream |
This package contains classes for management and manipulation of stream files.
|
| co.cask.cdap.data.stream.service.upload | |
| co.cask.cdap.data2.metadata.system | |
| co.cask.cdap.data2.transaction.queue.inmemory |
This package contains in memory implementation of queues.
|
| co.cask.cdap.data2.transaction.stream |
This package contains transaction stream consumer classes for maintaining stateful consumption of stream entries.
|
| co.cask.cdap.data2.transaction.stream.hbase |
This package contains class for stream consumer implementation that uses HBase as state store.
|
| co.cask.cdap.data2.transaction.stream.inmemory | |
| co.cask.cdap.data2.transaction.stream.leveldb |
This package contains class for stream consumer implementation that uses LevelDB as state store.
|
| Modifier and Type | Method and Description |
|---|---|
FileWriter<co.cask.cdap.api.flow.flowlet.StreamEvent> |
LocationStreamFileWriterFactory.create(StreamConfig config,
int generation) |
FileWriter<co.cask.cdap.api.flow.flowlet.StreamEvent> |
InMemoryStreamFileWriterFactory.create(StreamConfig config,
int generation) |
| Modifier and Type | Method and Description |
|---|---|
StreamConfig |
AbstractStreamCoordinatorClient.createStream(Id.Stream streamId,
Callable<StreamConfig> action) |
StreamConfig |
StreamCoordinatorClient.createStream(Id.Stream streamId,
Callable<StreamConfig> action)
Creates a stream by performing the given action.
|
static StreamConfig |
StreamUtils.ensureExists(StreamAdmin admin,
Id.Stream streamId) |
| Modifier and Type | Method and Description |
|---|---|
FileWriter<co.cask.cdap.api.flow.flowlet.StreamEvent> |
StreamFileWriterFactory.create(StreamConfig config,
int generation) |
static StreamFileOffset |
StreamUtils.decodeOffset(StreamConfig config,
DataInput in)
Decode a
StreamFileOffset encoded by the StreamUtils.encodeOffset(DataOutput, StreamFileOffset)
method. |
static int |
StreamUtils.getGeneration(StreamConfig config)
Finds the current generation id of a stream.
|
| Modifier and Type | Method and Description |
|---|---|
StreamConfig |
AbstractStreamCoordinatorClient.createStream(Id.Stream streamId,
Callable<StreamConfig> action) |
StreamConfig |
StreamCoordinatorClient.createStream(Id.Stream streamId,
Callable<StreamConfig> action)
Creates a stream by performing the given action.
|
| Constructor and Description |
|---|
LiveStreamFileReader(StreamConfig streamConfig,
StreamFileOffset beginOffset)
Construct a reader with
Constants.Stream.NEW_FILE_CHECK_INTERVAL
as the max interval to check for new file. |
LiveStreamFileReader(StreamConfig streamConfig,
StreamFileOffset beginOffset,
long maxFileCheckInterval)
Creates a new file reader.
|
MultiLiveStreamFileReader(StreamConfig streamConfig,
Iterable<? extends StreamFileOffset> offsets) |
| Constructor and Description |
|---|
FileContentWriterFactory(StreamConfig streamConfig,
ConcurrentStreamWriter streamWriter,
Map<String,String> headers) |
LengthBasedContentWriterFactory(StreamConfig streamConfig,
ConcurrentStreamWriter streamWriter,
Map<String,String> headers,
long bufferThreshold) |
| Constructor and Description |
|---|
StreamSystemMetadataWriter(MetadataStore metadataStore,
Id.Stream streamId,
StreamConfig config) |
| Modifier and Type | Method and Description |
|---|---|
StreamConfig |
InMemoryStreamAdmin.create(Id.Stream streamId) |
StreamConfig |
InMemoryStreamAdmin.create(Id.Stream streamId,
Properties props) |
StreamConfig |
InMemoryStreamAdmin.getConfig(Id.Stream streamId) |
| Modifier and Type | Field and Description |
|---|---|
protected StreamConfig |
StreamConsumerStateStore.streamConfig |
| Modifier and Type | Method and Description |
|---|---|
StreamConfig |
StreamConfig.Builder.build() |
StreamConfig |
FileStreamAdmin.create(Id.Stream streamId) |
StreamConfig |
StreamAdmin.create(Id.Stream streamId)
Creates stream if doesn't exist.
|
StreamConfig |
FileStreamAdmin.create(Id.Stream streamId,
Properties props) |
StreamConfig |
StreamAdmin.create(Id.Stream streamId,
Properties props)
Creates stream if doesn't exist.
|
StreamConfig |
FileStreamAdmin.getConfig(Id.Stream streamId) |
StreamConfig |
StreamAdmin.getConfig(Id.Stream streamId)
Returns the configuration of the given stream.
|
| Modifier and Type | Method and Description |
|---|---|
static StreamConfig.Builder |
StreamConfig.builder(StreamConfig config) |
StreamConsumerStateStore |
StreamConsumerStateStoreFactory.create(StreamConfig streamConfig)
Creates a
StreamConsumerStateStore for the given stream. |
protected abstract StreamConsumer |
AbstractStreamFileConsumerFactory.create(TableId tableId,
StreamConfig streamConfig,
ConsumerConfig consumerConfig,
StreamConsumerStateStore stateStore,
StreamConsumerState beginConsumerState,
FileReader<StreamEventOffset,Iterable<StreamFileOffset>> reader,
ReadFilter extraFilter)
Creates a
StreamConsumer. |
| Constructor and Description |
|---|
AbstractStreamFileConsumer(CConfiguration cConf,
StreamConfig streamConfig,
ConsumerConfig consumerConfig,
FileReader<StreamEventOffset,Iterable<StreamFileOffset>> reader,
StreamConsumerStateStore consumerStateStore,
StreamConsumerState beginConsumerState,
ReadFilter extraFilter) |
StreamConsumerStateStore(StreamConfig streamConfig) |
| Modifier and Type | Method and Description |
|---|---|
StreamConsumerStateStore |
HBaseStreamConsumerStateStoreFactory.create(StreamConfig streamConfig) |
protected StreamConsumer |
HBaseStreamFileConsumerFactory.create(TableId tableId,
StreamConfig streamConfig,
ConsumerConfig consumerConfig,
StreamConsumerStateStore stateStore,
StreamConsumerState beginConsumerState,
FileReader<StreamEventOffset,Iterable<StreamFileOffset>> reader,
ReadFilter extraFilter) |
| Constructor and Description |
|---|
HBaseStreamConsumerStateStore(StreamConfig streamConfig,
org.apache.hadoop.hbase.client.HTable hTable)
Constructor to create an instance for a given stream.
|
HBaseStreamFileConsumer(CConfiguration cConf,
StreamConfig streamConfig,
ConsumerConfig consumerConfig,
HBaseTableUtil tableUtil,
org.apache.hadoop.hbase.client.HTable hTable,
FileReader<StreamEventOffset,Iterable<StreamFileOffset>> reader,
StreamConsumerStateStore stateStore,
StreamConsumerState beginConsumerState,
ReadFilter extraFilter,
AbstractRowKeyDistributor keyDistributor)
Constructor.
|
| Modifier and Type | Method and Description |
|---|---|
StreamConsumerStateStore |
InMemoryStreamConsumerStateStoreFactory.create(StreamConfig streamConfig) |
| Constructor and Description |
|---|
InMemoryStreamConsumerStateStore(StreamConfig streamConfig,
InMemoryTable table) |
| Modifier and Type | Method and Description |
|---|---|
StreamConsumerStateStore |
LevelDBStreamConsumerStateStoreFactory.create(StreamConfig streamConfig) |
protected StreamConsumer |
LevelDBStreamFileConsumerFactory.create(TableId tableId,
StreamConfig streamConfig,
ConsumerConfig consumerConfig,
StreamConsumerStateStore stateStore,
StreamConsumerState beginConsumerState,
FileReader<StreamEventOffset,Iterable<StreamFileOffset>> reader,
ReadFilter extraFilter) |
| Constructor and Description |
|---|
LevelDBStreamConsumerStateStore(StreamConfig streamConfig,
LevelDBTableCore tableCore) |
LevelDBStreamFileConsumer(CConfiguration cConf,
StreamConfig streamConfig,
ConsumerConfig consumerConfig,
FileReader<StreamEventOffset,Iterable<StreamFileOffset>> reader,
StreamConsumerStateStore stateStore,
StreamConsumerState beginConsumerState,
ReadFilter extraFilter,
LevelDBTableCore tableCore,
Object dbLock) |
Copyright © 2016 Cask Data, Inc. Licensed under the Apache License, Version 2.0.