Package io.confluent.parallelconsumer

Interface ParallelStreamProcessor<K,V>

All Superinterfaces:

AutoCloseable, Closeable, DrainingCloseable, ParallelConsumer<K,V>

All Known Implementing Classes:

JStreamParallelEoSStreamProcessor, ParallelEoSStreamProcessor

public interface ParallelStreamProcessor<K,V>
extends ParallelConsumer<K,V>, DrainingCloseable

Parallel message consumer which also can optionally produce 0 or many ProducerRecord results to be published back to Kafka.

See Also:

• pollAndProduce(java.util.function.Function<io.confluent.parallelconsumer.PollContext<K, V>, org.apache.kafka.clients.producer.ProducerRecord<K, V>>)
• pollAndProduceMany(java.util.function.Function<io.confluent.parallelconsumer.PollContext<K, V>, java.util.List<org.apache.kafka.clients.producer.ProducerRecord<K, V>>>, java.util.function.Consumer<io.confluent.parallelconsumer.ParallelStreamProcessor.ConsumeProduceResult<K, V, K, V>>)

Nested Class Summary

Nested Classes

Modifier and Type	Interface	Description
static class	ParallelStreamProcessor.ConsumeProduceResult<K,V,KK,VV>	A simple triple structure to capture the set of coinciding data.

Nested classes/interfaces inherited from interface io.confluent.parallelconsumer.internal.DrainingCloseable

DrainingCloseable.DrainingMode

Nested classes/interfaces inherited from interface io.confluent.parallelconsumer.ParallelConsumer

ParallelConsumer.Tuple<L,R>

Field Summary

Fields inherited from interface io.confluent.parallelconsumer.internal.DrainingCloseable

DEFAULT_TIMEOUT

Method Summary

Modifier and Type	Method	Description
static <KK, VV> ParallelStreamProcessor<KK,VV>	createEosStreamProcessor(ParallelConsumerOptions<KK,VV> options)
void	poll(Consumer<PollContext<K,V>> usersVoidConsumptionFunction)	Register a function to be applied in parallel to each received message
void	pollAndProduce(Function<PollContext<K,V>,org.apache.kafka.clients.producer.ProducerRecord<K,V>> userFunction)	Register a function to be applied in parallel to each received message, which in turn returns a ProducerRecord to be sent back to the broker.
void	pollAndProduce(Function<PollContext<K,V>,org.apache.kafka.clients.producer.ProducerRecord<K,V>> userFunction, Consumer<ParallelStreamProcessor.ConsumeProduceResult<K,V,K,V>> callback)	Register a function to be applied in parallel to each received message, which in turn returns a ProducerRecord to be sent back to the broker.
void	pollAndProduceMany(Function<PollContext<K,V>,List<org.apache.kafka.clients.producer.ProducerRecord<K,V>>> userFunction)	Register a function to be applied in parallel to each received message, which in turn returns one or many ProducerRecords to be sent back to the broker.
void	pollAndProduceMany(Function<PollContext<K,V>,List<org.apache.kafka.clients.producer.ProducerRecord<K,V>>> userFunction, Consumer<ParallelStreamProcessor.ConsumeProduceResult<K,V,K,V>> callback)	Register a function to be applied in parallel to each received message, which in turn returns one or more ProducerRecords to be sent back to the broker.

Methods inherited from interface io.confluent.parallelconsumer.internal.DrainingCloseable

close, close, closeDontDrainFirst, closeDontDrainFirst, closeDrainFirst, closeDrainFirst, workRemaining

Methods inherited from interface io.confluent.parallelconsumer.ParallelConsumer

pauseIfRunning, resumeIfPaused, subscribe, subscribe, subscribe, subscribe

Method Details

createEosStreamProcessor

static <KK,
VV> ParallelStreamProcessor<KK,VV> createEosStreamProcessor(ParallelConsumerOptions<KK,VV> options)

poll

void poll(Consumer<PollContext<K,V>> usersVoidConsumptionFunction)

Register a function to be applied in parallel to each received message

Parameters:

usersVoidConsumptionFunction - the function

pollAndProduceMany

void pollAndProduceMany(Function<PollContext<K,V>,List<org.apache.kafka.clients.producer.ProducerRecord<K,V>>> userFunction,
 Consumer<ParallelStreamProcessor.ConsumeProduceResult<K,V,K,V>> callback)

Register a function to be applied in parallel to each received message, which in turn returns one or more ProducerRecords to be sent back to the broker.

Parameters:

callback - applied after the produced message is acknowledged by kafka

pollAndProduceMany

void pollAndProduceMany(Function<PollContext<K,V>,List<org.apache.kafka.clients.producer.ProducerRecord<K,V>>> userFunction)

Register a function to be applied in parallel to each received message, which in turn returns one or many ProducerRecords to be sent back to the broker.

pollAndProduce

void pollAndProduce(Function<PollContext<K,V>,org.apache.kafka.clients.producer.ProducerRecord<K,V>> userFunction)

Register a function to be applied in parallel to each received message, which in turn returns a ProducerRecord to be sent back to the broker.

pollAndProduce

void pollAndProduce(Function<PollContext<K,V>,org.apache.kafka.clients.producer.ProducerRecord<K,V>> userFunction,
 Consumer<ParallelStreamProcessor.ConsumeProduceResult<K,V,K,V>> callback)

Register a function to be applied in parallel to each received message, which in turn returns a ProducerRecord to be sent back to the broker.

Parameters:

callback - applied after the produced message is acknowledged by kafka