Class KafkaDocumentSource<T>
java.lang.Object
no.mnemonic.messaging.documentchannel.kafka.KafkaDocumentSource<T>
- Type Parameters:
T
- document type
- All Implemented Interfaces:
AutoCloseable
,no.mnemonic.commons.metrics.MetricAspect
,DocumentSource<T>
public class KafkaDocumentSource<T>
extends Object
implements DocumentSource<T>, no.mnemonic.commons.metrics.MetricAspect
Kafka version of a document channel source. The source is configured with a kafka cluster, topic and groupID.
Multiple sources may be configured to the same topic and groupID, which will load-balance the incoming documents
between the active sources.
Multiple sources with different groupIDs will receive individual copies of each document on the topic.
-
Nested Class Summary
Modifier and TypeClassDescriptionstatic class
static enum
static interface
-
Method Summary
Modifier and TypeMethodDescriptionstatic <T> KafkaDocumentSource.Builder<T>
builder()
void
close()
createDocumentSubscription
(DocumentChannelListener<T> listener) Fetch the current cursor of this source.no.mnemonic.commons.metrics.Metrics
void
Seek to cursorsetMaxAssignmentWait
(long maxAssignmentWait) This parameter only applies to getCursor(), and should probably never need to be changed.boolean
waitForAssignment
(Duration maxWait) Let client wait for partition assignment to complete.Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
Methods inherited from interface no.mnemonic.messaging.documentchannel.DocumentSource
poll
-
Method Details
-
seek
Seek to cursor- Parameters:
cursor
- A string representation of a cursor as returned in aKafkaDocument
. A null cursor will not change the offset of the consumer.- Throws:
KafkaInvalidSeekException
- if the cursor is invalid, or points to a timestamp which is not reachable
-
getMetrics
public no.mnemonic.commons.metrics.Metrics getMetrics() throws no.mnemonic.commons.metrics.MetricException- Specified by:
getMetrics
in interfaceno.mnemonic.commons.metrics.MetricAspect
- Throws:
no.mnemonic.commons.metrics.MetricException
-
waitForAssignment
Let client wait for partition assignment to complete. Method is reentrant, so calling this AFTER assignment is already done will immediately return true.- Parameters:
maxWait
- max duration to wait before returning false- Returns:
- true if partitions are assigned, or false if no partitions were assigned before the timeout
- Throws:
InterruptedException
-
getCursor
Fetch the current cursor of this source. The method will block up tomaxAssignmentWait
milliseconds waiting for partitions assignment.Note; this cursor is not thread safe; if there is a separate consumer thread polling, you should rather use the cursor of the KafkaDocument.
- Returns:
- the String cursor for the current position of this source
- Throws:
InterruptedException
- if interrupted while waiting for Kafka to assign partitions- See Also:
-
createDocumentSubscription
- Specified by:
createDocumentSubscription
in interfaceDocumentSource<T>
-
poll
- Specified by:
poll
in interfaceDocumentSource<T>
-
close
public void close()- Specified by:
close
in interfaceAutoCloseable
- Specified by:
close
in interfaceDocumentSource<T>
-
getKafkaConsumer
-
setMaxAssignmentWait
This parameter only applies to getCursor(), and should probably never need to be changed.- Parameters:
maxAssignmentWait
- max millis to wait for assignments.
-
builder
-