Clients -
kafka :
Consumer
Represent a Kafka consumer endpoint.
Constructor
__init
(ConsumerConfig config)
- config ConsumerConfig
-
Configurations related to consumer endpoint.
Remote Methods
assign | Assigns consumer to a set of topic partitions. |
close | Closes consumer connection to the external Kafka broker. |
commit | Commits current consumed offsets for consumer. |
commitOffset | Commits given offsets and partitions for the given topics, for consumer. |
connect | Connects consumer to the provided host in the consumer configs. |
getAssignment | Returns the currently assigned partitions for the consumer. |
getAvailableTopics | Returns the available list of topics for a particular consumer. |
getBeginningOffsets | Returns start offsets for given set of partitions. |
getCommittedOffset | Returns last committed offsets for the given topic partitions. |
getEndOffsets | Returns last offsets for given set of partitions. |
getPausedPartitions | Returns the partitions, which are currently paused. |
getPositionOffset | Returns the offset of the next record that will be fetched, if a records exists in that position. |
getSubscription | Returns set of topics which are currently subscribed by the consumer. |
getTopicPartitions | Retrieve the set of partitions in which the topic belongs. |
pause | Pause consumer retrieving messages from set of partitions. |
poll | Poll the consumer for external broker for records. |
resume | Resume consumer retrieving messages from set of partitions which were paused earlier. |
seek | Seek for a given offset in a topic partition. |
seekToBeginning | Seek the beginning of the offsets for the given set of topic partitions. |
seekToEnd | Seek end of the offsets for the given set of topic partitions. |
subscribe | Subscribes the consumer to the provided set of topics. |
subscribeToPattern | Subscribes the consumer to the topics which matches to the provided pattern. |
subscribeWithPartitionRebalance | Subscribes to consumer to the provided set of topics with rebalance listening is enabled. This function can be used inside a service, to subscribe to a set of topics, while rebalancing the patition assignment of the consumers. |
unsubscribe | Unsubscribe the consumer from all the topic subscriptions. |
Methods
Fields
- consumerConfig ConsumerConfig? (default ())
-
Used to store configurations related to a Kafka connection.
Assigns consumer to a set of topic partitions.
Parameters
- partitions TopicPartition[]
-
Topic partitions to be assigned.
-
Return Type
(ConsumerError?) kafka:ConsumerError
if encounters an error, returns nil otherwise.
Closes consumer connection to the external Kafka broker.
Parameters
- duration int (default -1)
-
Timeout duration for the close operation execution.
-
Return Type
(ConsumerError?) kafka:ConsumerError
if encounters an error, returns nil otherwise.
Commits current consumed offsets for consumer.
-
Return Type
(ConsumerError?) kafka:ConsumerError
if encounters an error, returns nil otherwise.
Commits given offsets and partitions for the given topics, for consumer.
Parameters
- offsets PartitionOffset[]
-
Offsets to be commited.
- duration int (default -1)
-
Timeout duration for the commit operation execution.
-
Return Type
(ConsumerError?) kafka:ConsumerError
if encounters an error, returns nil otherwise.
Connects consumer to the provided host in the consumer configs.
-
Return Type
(ConsumerError?) kafka:ConsumerError
if encounters an error, returns nil otherwise.
Returns the currently assigned partitions for the consumer.
-
Return Type
(TopicPartition[] | ConsumerError) Array of assigned partitions for the consumer if executes successfully,
kafka:ConsumerError
otherwise.
Returns the available list of topics for a particular consumer.
Parameters
- duration int (default -1)
-
Timeout duration for the get available topics execution.
-
Return Type
(string[] | ConsumerError) Array of topics currently available (authorized) for the consumer to subscribe, returns
kafka:ConsumerError
if the operation fails.
getBeginningOffsets
(TopicPartition[] partitions, int duration)
returns PartitionOffset[] | ConsumerErrorReturns start offsets for given set of partitions.
Parameters
- partitions TopicPartition[]
-
Array of topic partitions to get the starting offsets.
- duration int (default -1)
-
Timeout duration for the get beginning offsets execution.
-
Return Type
(PartitionOffset[] | ConsumerError) Starting offsets for the given partitions if executes successfully, returns
kafka:ConsumerError
if the operation fails.
Returns last committed offsets for the given topic partitions.
Parameters
- partition TopicPartition
-
Topic partition in which the committed offset is returned for consumer.
- duration int (default -1)
-
Timeout duration for the get committed offset operation to execute.
-
Return Type
(PartitionOffset | ConsumerError) Committed offset for the consumer for the given partition if executes successfully, returns
kafka:ConsumerError
if the operation fails.
Returns last offsets for given set of partitions.
Parameters
- partitions TopicPartition[]
-
Set of partitions to get the last offsets.
- duration int (default -1)
-
Timeout duration for the get end offsets operation to execute.
-
Return Type
(PartitionOffset[] | ConsumerError) End offsets for the given partitions if executes successfully, returns
kafka:ConsumerError
if the operation fails.
Returns the partitions, which are currently paused.
-
Return Type
(TopicPartition[] | ConsumerError) Set of partitions paused from message retrieval if executes successfully, returns
kafka:ConsumerError
if the operation fails.
Returns the offset of the next record that will be fetched, if a records exists in that position.
Parameters
- partition TopicPartition
-
Topic partition in which the position is required.
- duration int (default -1)
-
Timeout duration for the get position offset operation to execute.
-
Return Type
(int | ConsumerError) Offset which will be fetched next (if a records exists in that offset), returns
kafka:ConsumerError
if the operation fails.
Returns set of topics which are currently subscribed by the consumer.
-
Return Type
(string[] | ConsumerError) Array of subscribed topics for the consumer if executes successfully, returns
kafka:ConsumerError
if the operation fails.
Retrieve the set of partitions in which the topic belongs.
Parameters
- topic string
-
Given topic for partition information is needed.
- duration int (default -1)
-
Timeout duration for the get topic partitions operation to execute.
-
Return Type
(TopicPartition[] | ConsumerError) Array of partitions for the given topic if executes successfully, returns
kafka:ConsumerError
if the operation fails.
Pause consumer retrieving messages from set of partitions.
Parameters
- partitions TopicPartition[]
-
Set of partitions to pause the retrieval of messages.
-
Return Type
(ConsumerError?) kafka:ConsumerError
if encounters an error, returns nil otherwise.
Poll the consumer for external broker for records.
Parameters
- timeoutValue int
-
Polling time in milliseconds.
-
Return Type
(ConsumerRecord[] | ConsumerError) Array of consumer records if executes successfully, returns
kafka:ConsumerError
if the operation fails.
Resume consumer retrieving messages from set of partitions which were paused earlier.
Parameters
- partitions TopicPartition[]
-
Set of partitions to resume the retrieval of messages.
-
Return Type
(ConsumerError?) kafka:ConsumerError
if encounters an error, returns nil otherwise.
Seek for a given offset in a topic partition.
Parameters
- offset PartitionOffset
-
PartitionOffset to seek.
-
Return Type
(ConsumerError?) kafka:ConsumerError
if encounters an error, returns nil otherwise.
Seek the beginning of the offsets for the given set of topic partitions.
Parameters
- partitions TopicPartition[]
-
Set of topic partitions to seek.
-
Return Type
(ConsumerError?) kafka:ConsumerError
if encounters an error, returns nil otherwise.
Seek end of the offsets for the given set of topic partitions.
Parameters
- partitions TopicPartition[]
-
Set of topic partitions to seek.
-
Return Type
(ConsumerError?) kafka:ConsumerError
if encounters an error, returns nil otherwise.
Subscribes the consumer to the provided set of topics.
Parameters
- topics string[]
-
Array of topics to be subscribed.
-
Return Type
(ConsumerError?) kafka:ConsumerError
if encounters an error, returns nil otherwise.
Subscribes the consumer to the topics which matches to the provided pattern.
Parameters
- regex string
-
Pattern which should be matched with the topics to be subscribed.
-
Return Type
(ConsumerError?) kafka:ConsumerError
if encounters an error, returns nil otherwise.
subscribeWithPartitionRebalance
(string[] topics, function(Consumer, TopicPartition[]) returns (())
onPartitionsRevoked, function(Consumer, TopicPartition[]) returns (())
onPartitionsAssigned)
Subscribes to consumer to the provided set of topics with rebalance listening is enabled. This function can be used inside a service, to subscribe to a set of topics, while rebalancing the patition assignment of the consumers.
Parameters
- topics string[]
-
Array of topics to be subscribed.
-
onPartitionsRevoked
function(Consumer, TopicPartition[]) returns (())
-
Function which will be executed if partitions are revoked from this consumer.
-
onPartitionsAssigned
function(Consumer, TopicPartition[]) returns (())
-
Function which will be executed if partitions are assigned this consumer.
-
Return Type
(ConsumerError?) kafka:ConsumerError
if encounters an error, returns nil otherwise.
Unsubscribe the consumer from all the topic subscriptions.
-
Return Type
(ConsumerError?) kafka:ConsumerError
if encounters an error, returns nil otherwise.
Parameters
- s service
- name string? (default ())
-
Return Type
(error?)