Clients -
kafka :
Consumer
Represent a Kafka consumer endpoint.
Constructor
__init
(ConsumerConfig config)
- config ConsumerConfig
-
Configurations related to consumer endpoint.
Remote Methods
assign | Assigns consumer to a set of topic partitions. |
close | Closes consumer connection to the external Kafka broker. |
commit | Commits current consumed offsets for consumer. |
commitOffset | Commits given offsets and partitions for the given topics, for consumer. |
connect | Connects consumer to the provided host in the consumer configs. |
getAssignment | Returns the currently assigned partitions for the consumer. |
getAvailableTopics | Returns the available list of topics for a particular consumer. |
getBeginningOffsets | Returns start offsets for given set of partitions. |
getCommittedOffset | Returns last committed offsets for the given topic partitions. |
getEndOffsets | Returns last offsets for given set of partitions. |
getPausedPartitions | Returns the partitions, which are currently paused. |
getPositionOffset | Returns the offset of the next record that will be fetched, if a records exists in that position. |
getSubscription | Returns set of topics wich are currently subscribed by the consumer. |
getTopicPartitions | Retrieve the set of partitions in which the topic belongs. |
pause | Pause consumer retrieving messages from set of partitions. |
poll | Poll the consumer for external broker for records. |
resume | Resume consumer retrieving messages from set of partitions which were paused earlier. |
seek | Seek for a given offset in a topic partition. |
seekToBeginning | Seek the beginning of the offsets for the given set of topic partitions. |
seekToEnd | Seek end of the offsets for the given set of topic partitions. |
subscribe | Subscribes the consumer to the provided set of topics. |
subscribeToPattern | Subscribes the consumer to the topics which matches to the provided pattern. |
subscribeWithPartitionRebalance | Subscribes to consumer to the provided set of topics with rebalance listening is enabled. This function can be used inside a service, to subscribe to a set of topics, while rebalancing the patition assignment of the consumers. |
unsubscribe | Unsubscribe the consumer from all the topic subscriptions. |
Methods
Fields
- consumerConfig ConsumerConfig? ()
-
Used to store configurations related to a Kafka connection.
Assigns consumer to a set of topic partitions.
Parameters
- partitions TopicPartition
-
Topic partitions to be assigned.
-
Return Type
(ConsumerError?) kafka:ConsumerError
if encounters an error, returns nil otherwise.
Closes consumer connection to the external Kafka broker.
Parameters
- duration int (default -1)
-
Timeout duration for the close operation execution.
-
Return Type
(ConsumerError?) kafka:ConsumerError
if encounters an error, returns nil otherwise.
Commits current consumed offsets for consumer.
-
Return Type
(ConsumerError?) kafka:ConsumerError
if encounters an error, returns nil otherwise.
Commits given offsets and partitions for the given topics, for consumer.
Parameters
- offsets PartitionOffset
-
Offsets to be commited.
- duration int (default -1)
-
Timeout duration for the commit operation execution.
-
Return Type
(ConsumerError?) kafka:ConsumerError
if encounters an error, returns nil otherwise.
Connects consumer to the provided host in the consumer configs.
-
Return Type
(ConsumerError?) kafka:ConsumerError
if encounters an error, returns nil otherwise.
Returns the currently assigned partitions for the consumer.
-
Return Type
(TopicPartition | ConsumerError) Array of assigned partitions for the consumer if executes successfully,
kafka:ConsumerError
otherwise.
Returns the available list of topics for a particular consumer.
Parameters
- duration int (default -1)
-
Timeout duration for the get available topics execution.
-
Return Type
(string | ConsumerError) Array of topics currently available (authorized) for the consumer to subscribe, returns
kafka:ConsumerError
if the operation fails.
getBeginningOffsets
(TopicPartition partitions, int duration)
returns PartitionOffset | ConsumerErrorReturns start offsets for given set of partitions.
Parameters
- partitions TopicPartition
-
Array of topic partitions to get the starting offsets.
- duration int (default -1)
-
Timeout duration for the get beginning offsets execution.
-
Return Type
(PartitionOffset | ConsumerError) Starting offsets for the given partitions if executes successfully, returns
kafka:ConsumerError
if the operation fails.
Returns last committed offsets for the given topic partitions.
Parameters
- partition TopicPartition
-
Topic partition in which the committed offset is returned for consumer.
- duration int (default -1)
-
Timeout duration for the get committed offset operation to execute.
-
Return Type
(PartitionOffset | ConsumerError) Committed offset for the consumer for the given partition if executes successfully, returns
kafka:ConsumerError
if the operation fails.
Returns last offsets for given set of partitions.
Parameters
- partitions TopicPartition
-
Set of partitions to get the last offsets.
- duration int (default -1)
-
Timeout duration for the get end offsets operation to execute.
-
Return Type
(PartitionOffset | ConsumerError) End offsets for the given partitions if executes successfully, returns
kafka:ConsumerError
if the operation fails.
Returns the partitions, which are currently paused.
-
Return Type
(TopicPartition | ConsumerError) Set of partitions paused from message retrieval if executes successfully, returns
kafka:ConsumerError
if the operation fails.
Returns the offset of the next record that will be fetched, if a records exists in that position.
Parameters
- partition TopicPartition
-
Topic partition in which the position is required.
- duration int (default -1)
-
Timeout duration for the get position offset operation to execute.
-
Return Type
(int | ConsumerError) Offset which will be fetched next (if a records exists in that offset), returns
kafka:ConsumerError
if the operation fails.
Returns set of topics wich are currently subscribed by the consumer.
-
Return Type
(string | ConsumerError) Array of subscribed topics for the consumer if executes successfully, returns
kafka:ConsumerError
if the operation fails.
Retrieve the set of partitions in which the topic belongs.
Parameters
- topic string
-
Given topic for partition information is needed.
- duration int (default -1)
-
Timeout duration for the get topic partitions operation to execute.
-
Return Type
(TopicPartition | ConsumerError) Array of partitions for the given topic if executes successfully, returns
kafka:ConsumerError
if the operation fails.
Pause consumer retrieving messages from set of partitions.
Parameters
- partitions TopicPartition
-
Set of partitions to pause the retrieval of messages.
-
Return Type
(ConsumerError?) kafka:ConsumerError
if encounters an error, returns nil otherwise.
Poll the consumer for external broker for records.
Parameters
- timeoutValue int
-
Polling time in milliseconds.
-
Return Type
(ConsumerRecord | ConsumerError) Array of consumer records if executes successfully, returns
kafka:ConsumerError
if the operation fails.
Resume consumer retrieving messages from set of partitions which were paused earlier.
Parameters
- partitions TopicPartition
-
Set of partitions to resume the retrieval of messages.
-
Return Type
(ConsumerError?) kafka:ConsumerError
if encounters an error, returns nil otherwise.
Seek for a given offset in a topic partition.
Parameters
- offset PartitionOffset
-
PartitionOffset to seek.
-
Return Type
(ConsumerError?) kafka:ConsumerError
if encounters an error, returns nil otherwise.
Seek the beginning of the offsets for the given set of topic partitions.
Parameters
- partitions TopicPartition
-
Set of topic partitions to seek.
-
Return Type
(ConsumerError?) kafka:ConsumerError
if encounters an error, returns nil otherwise.
Seek end of the offsets for the given set of topic partitions.
Parameters
- partitions TopicPartition
-
Set of topic partitions to seek.
-
Return Type
(ConsumerError?) kafka:ConsumerError
if encounters an error, returns nil otherwise.
Subscribes the consumer to the provided set of topics.
Parameters
- topics string
-
Array of topics to be subscribed.
-
Return Type
(ConsumerError?) kafka:ConsumerError
if encounters an error, returns nil otherwise.
Subscribes the consumer to the topics which matches to the provided pattern.
Parameters
- regex string
-
Pattern which should be matched with the topics to be subscribed.
-
Return Type
(ConsumerError?) kafka:ConsumerError
if encounters an error, returns nil otherwise.
subscribeWithPartitionRebalance
(string topics, function(Consumer, TopicPartition) returns (())
onPartitionsRevoked, function(Consumer, TopicPartition) returns (())
onPartitionsAssigned)
Subscribes to consumer to the provided set of topics with rebalance listening is enabled. This function can be used inside a service, to subscribe to a set of topics, while rebalancing the patition assignment of the consumers.
Parameters
- topics string
-
Array of topics to be subscribed.
-
onPartitionsRevoked
function(Consumer, TopicPartition) returns (())
-
Function which will be executed if partitions are revoked from this consumer.
-
onPartitionsAssigned
function(Consumer, TopicPartition) returns (())
-
Function which will be executed if partitions are assigned this consumer.
-
Return Type
(ConsumerError?) kafka:ConsumerError
if encounters an error, returns nil otherwise.
Unsubscribe the consumer from all the topic subscriptions.
-
Return Type
(ConsumerError?) kafka:ConsumerError
if encounters an error, returns nil otherwise.
Parameters
- s service
- name string (default ())
-
Return Type
(error)