embedded-kafka-connect
API
Generated with
embedded-kafka-connect/io.github.embeddedkafka.connect/EmbeddedKafkaConnect

EmbeddedKafkaConnect

io.github.embeddedkafka.connect.EmbeddedKafkaConnect
See theEmbeddedKafkaConnect companion object
trait EmbeddedKafkaConnect extends EmbeddedKafka

Helper trait for running Kafka Connect server. Use .startConnect to start the server.

Attributes

Companion:
object
Graph
Supertypes
trait EmbeddedKafka
trait KafkaOps
trait ZooKeeperOps
trait ProducerOps[EmbeddedKafkaConfig]
trait ConsumerOps[EmbeddedKafkaConfig]
trait AdminOps[EmbeddedKafkaConfig]
class Object
trait Matchable
class Any
Known subtypes

Members list

Concise view

Value members

Inherited methods

def consumeFirstKeyedMessageFrom[K, V](topic: String, autoCommit: Boolean, timeout: Duration)(implicit config: EmbeddedKafkaConfig, keyDeserializer: Deserializer[K], valueDeserializer: Deserializer[V]): (K, V)

Consumes the first message available in a given topic, deserializing it as type (K, V).

Consumes the first message available in a given topic, deserializing it as type (K, V).

Only the message that is returned is committed if autoCommit is false. If autoCommit is true then all messages that were polled will be committed.

Attributes

autoCommit

if false, only the offset for the consumed message will be committed. if true, the offset for the last polled message will be committed instead.

config

an implicit EmbeddedKafkaConfig

keyDeserializer

an implicit Deserializer for the type K

timeout

the interval to wait for messages before throwing TimeoutException

topic

the topic to consume a message from

valueDeserializer

an implicit Deserializer for the type V

Returns:

the first message consumed from the given topic, with a type (K, V)

Inherited from:
ConsumerOps
def consumeFirstMessageFrom[V](topic: String, autoCommit: Boolean, timeout: Duration)(implicit config: EmbeddedKafkaConfig, valueDeserializer: Deserializer[V]): V

Consumes the first message available in a given topic, deserializing it as type V.

Consumes the first message available in a given topic, deserializing it as type V.

Only the message that is returned is committed if autoCommit is false. If autoCommit is true then all messages that were polled will be committed.

Attributes

autoCommit

if false, only the offset for the consumed message will be committed. if true, the offset for the last polled message will be committed instead.

config

an implicit EmbeddedKafkaConfig

timeout

the interval to wait for messages before throwing TimeoutException

topic

the topic to consume a message from

valueDeserializer

an implicit Deserializer for the type V

Returns:

the first message consumed from the given topic, with a type V

Inherited from:
ConsumerOps
def consumeFirstStringMessageFrom(topic: String, autoCommit: Boolean, timeout: Duration)(implicit config: EmbeddedKafkaConfig): String

Attributes

Inherited from:
ConsumerOps
def consumeNumberKeyedMessagesFrom[K, V](topic: String, number: Int, autoCommit: Boolean, timeout: Duration)(implicit config: EmbeddedKafkaConfig, keyDeserializer: Deserializer[K], valueDeserializer: Deserializer[V]): List[(K, V)]

Attributes

Inherited from:
ConsumerOps
def consumeNumberKeyedMessagesFromTopics[K, V](topics: Set[String], number: Int, autoCommit: Boolean, timeout: Duration, resetTimeoutOnEachMessage: Boolean)(implicit config: EmbeddedKafkaConfig, keyDeserializer: Deserializer[K], valueDeserializer: Deserializer[V]): Map[String, List[(K, V)]]

Consumes the first n messages available in given topics, deserializes them as type (K, V), and returns the n messages in a Map from topic name to List[(K, V)].

Consumes the first n messages available in given topics, deserializes them as type (K, V), and returns the n messages in a Map from topic name to List[(K, V)].

Only the messages that are returned are committed if autoCommit is false. If autoCommit is true then all messages that were polled will be committed.

Attributes

autoCommit

if false, only the offset for the consumed messages will be committed. if true, the offset for the last polled message will be committed instead.

config

an implicit EmbeddedKafkaConfig

keyDeserializer

an implicit Deserializer for the type K

number

the number of messages to consume in a batch

resetTimeoutOnEachMessage

when true, throw TimeoutException if we have a silent period (no incoming messages) for the timeout interval; when false, throw TimeoutException after the timeout interval if we haven't received all of the expected messages

timeout

the interval to wait for messages before throwing TimeoutException

topics

the topics to consume messages from

valueDeserializer

an implicit Deserializer for the type V

Returns:

the List of messages consumed from the given topics, each with a type (K, V)

Inherited from:
ConsumerOps
def consumeNumberMessagesFrom[V](topic: String, number: Int, autoCommit: Boolean, timeout: Duration)(implicit config: EmbeddedKafkaConfig, valueDeserializer: Deserializer[V]): List[V]

Attributes

Inherited from:
ConsumerOps
def consumeNumberMessagesFromTopics[V](topics: Set[String], number: Int, autoCommit: Boolean, timeout: Duration, resetTimeoutOnEachMessage: Boolean)(implicit config: EmbeddedKafkaConfig, valueDeserializer: Deserializer[V]): Map[String, List[V]]

Consumes the first n messages available in given topics, deserializes them as type V, and returns the n messages in a Map from topic name to List[V].

Consumes the first n messages available in given topics, deserializes them as type V, and returns the n messages in a Map from topic name to List[V].

Only the messages that are returned are committed if autoCommit is false. If autoCommit is true then all messages that were polled will be committed.

Attributes

autoCommit

if false, only the offset for the consumed messages will be committed. if true, the offset for the last polled message will be committed instead.

config

an implicit EmbeddedKafkaConfig

number

the number of messages to consume in a batch

resetTimeoutOnEachMessage

when true, throw TimeoutException if we have a silent period (no incoming messages) for the timeout interval; when false, throw TimeoutException after the timeout interval if we haven't received all of the expected messages

timeout

the interval to wait for messages before throwing TimeoutException

topics

the topics to consume messages from

valueDeserializer

an implicit Deserializer for the type V

Returns:

the List of messages consumed from the given topics, each with a type V

Inherited from:
ConsumerOps
def consumeNumberStringMessagesFrom(topic: String, number: Int, autoCommit: Boolean, timeout: Duration)(implicit config: EmbeddedKafkaConfig): List[String]

Attributes

Inherited from:
ConsumerOps
def createCustomTopic(topic: String, topicConfig: Map[String, String], partitions: Int, replicationFactor: Int)(implicit config: EmbeddedKafkaConfig): Try[Unit]

Creates a topic with a custom configuration.

Creates a topic with a custom configuration.

Attributes

config

an implicit EmbeddedKafkaConfig

partitions

number of partitions

replicationFactor

replication factor

topic

the topic name

topicConfig

per topic configuration Map

Inherited from:
AdminOps
def deleteTopics(topics: List[String])(implicit config: EmbeddedKafkaConfig): Try[Unit]

Either deletes or marks for deletion a list of topics.

Either deletes or marks for deletion a list of topics.

Attributes

config

an implicit EmbeddedKafkaConfig

topics

the topic names

Inherited from:
AdminOps
def publishStringMessageToKafka(topic: String, message: String)(implicit config: EmbeddedKafkaConfig): Unit

Publishes synchronously a message of type String to the running Kafka broker.

Publishes synchronously a message of type String to the running Kafka broker.

Attributes

config

an implicit EmbeddedKafkaConfig

message

the message to publish

topic

the topic to which publish the message (it will be auto-created)

Throws:
KafkaUnavailableException

if unable to connect to Kafka

Inherited from:
ProducerOps
def publishToKafka[K, T](topic: String, messages: Seq[(K, T)])(implicit config: EmbeddedKafkaConfig, keySerializer: Serializer[K], serializer: Serializer[T]): Unit

Publishes synchronously a batch of message to the running Kafka broker.

Publishes synchronously a batch of message to the running Kafka broker.

Attributes

config

an implicit EmbeddedKafkaConfig

keySerializer

an implicit Serializer for the type K

messages

the keys and messages of type (K, T) to publish

serializer

an implicit Serializer for the type T

topic

the topic to which publish the message (it will be auto-created)

Throws:
KafkaUnavailableException

if unable to connect to Kafka

Inherited from:
ProducerOps
def publishToKafka[K, T](topic: String, key: K, message: T)(implicit config: EmbeddedKafkaConfig, keySerializer: Serializer[K], serializer: Serializer[T]): Unit

Publishes synchronously a message to the running Kafka broker.

Publishes synchronously a message to the running Kafka broker.

Attributes

config

an implicit EmbeddedKafkaConfig

key

the key of type K to publish

message

the message of type T to publish

serializer

an implicit Serializer for the type T

topic

the topic to which publish the message (it will be auto-created)

Throws:
KafkaUnavailableException

if unable to connect to Kafka

Inherited from:
ProducerOps
def publishToKafka[T](producerRecord: ProducerRecord[String, T])(implicit config: EmbeddedKafkaConfig, serializer: Serializer[T]): Unit

Publishes synchronously a message to the running Kafka broker.

Publishes synchronously a message to the running Kafka broker.

Attributes

config

an implicit EmbeddedKafkaConfig

producerRecord

the producerRecord of type T to publish

serializer

an implicit Serializer for the type T

Throws:
KafkaUnavailableException

if unable to connect to Kafka

Inherited from:
ProducerOps
def publishToKafka[T](topic: String, message: T)(implicit config: EmbeddedKafkaConfig, serializer: Serializer[T]): Unit

Publishes synchronously a message to the running Kafka broker.

Publishes synchronously a message to the running Kafka broker.

Attributes

config

an implicit EmbeddedKafkaConfig

message

the message of type T to publish

serializer

an implicit Serializer for the type T

topic

the topic to which publish the message (it will be auto-created)

Throws:
KafkaUnavailableException

if unable to connect to Kafka

Inherited from:
ProducerOps
def startConnect[T](connectPort: Int, offsets: Path, extraConfig: Map[String, String])(block: => T)(implicit config: EmbeddedKafkaConfig): T

Start a Kafka Connect server and pass a block of code that can operate while the server is active.

Start a Kafka Connect server and pass a block of code that can operate while the server is active.

Attributes

block

the code block that will executed while the Kafka Connect server is active. Once the block has been executed the server will be stopped.

connectPort

the Kafka Connect port for the REST API to listen on.

extraConfig

additional Kafka Connect configuration (overwrite existing keys in default config)

offsets

the file to store offset data in.

Inherited from:
EmbeddedKafkaConnectSupport (hidden)
protected def withAdminClient[T](body: AdminClient => T)(implicit config: EmbeddedKafkaConfig): Try[T]

Creates an AdminClient, then executes the body passed as a parameter.

Creates an AdminClient, then executes the body passed as a parameter.

Attributes

body

the function to execute

config

an implicit EmbeddedKafkaConfig

Inherited from:
AdminOps
def withConsumer[K, V, T](body: KafkaConsumer[K, V] => T)(implicit config: EmbeddedKafkaConfig, keyDeserializer: Deserializer[K], valueDeserializer: Deserializer[V]): T

Loaner pattern that allows running a code block with a newly created producer. The producer's lifecycle will be automatically handled and closed at the end of the given code block.

Loaner pattern that allows running a code block with a newly created producer. The producer's lifecycle will be automatically handled and closed at the end of the given code block.

Attributes

body

the function to execute that returns T

config

an implicit EmbeddedKafkaConfig

keyDeserializer

an implicit Deserializer for the type K

valueDeserializer

an implicit Deserializer for the type V

Inherited from:
ConsumerOps
def withProducer[K, V, T](body: KafkaProducer[K, V] => T)(implicit config: EmbeddedKafkaConfig, keySerializer: Serializer[K], valueSerializer: Serializer[V]): T

Loaner pattern that allows running a code block with a newly created consumer. The consumer's lifecycle will be automatically handled and closed at the end of the given code block.

Loaner pattern that allows running a code block with a newly created consumer. The consumer's lifecycle will be automatically handled and closed at the end of the given code block.

Attributes

body

the function to execute that returns T

config

an implicit EmbeddedKafkaConfig

keySerializer

an implicit Serializer for the type K

valueSerializer

an implicit Serializer for the type V

Inherited from:
ProducerOps
def withRunningKafka[T](body: => T)(implicit config: EmbeddedKafkaConfig): T

Starts a ZooKeeper instance and a Kafka broker (and performs additional logic, if any), then executes the body passed as a parameter.

Starts a ZooKeeper instance and a Kafka broker (and performs additional logic, if any), then executes the body passed as a parameter.

Attributes

body

the function to execute

config

an implicit EmbeddedKafkaConfig

Inherited from:
EmbeddedKafkaSupport (hidden)
def withRunningKafkaOnFoundPort[T](config: EmbeddedKafkaConfig)(body: EmbeddedKafkaConfig => T): T

Starts a ZooKeeper instance and a Kafka broker (and performs additional logic, if any), then executes the body passed as a parameter. The actual ports of the servers will be detected and inserted into a copied version of the EmbeddedKafkaConfig that gets passed to body. This is useful if you set any port to 0, which will listen on an arbitrary available port.

Starts a ZooKeeper instance and a Kafka broker (and performs additional logic, if any), then executes the body passed as a parameter. The actual ports of the servers will be detected and inserted into a copied version of the EmbeddedKafkaConfig that gets passed to body. This is useful if you set any port to 0, which will listen on an arbitrary available port.

Attributes

body

the function to execute, given an EmbeddedKafkaConfig with the actual ports the servers are running on

config

the user-defined EmbeddedKafkaConfig

Inherited from:
EmbeddedKafkaSupport (hidden)

Inherited fields

protected val adminClientCloseTimeout: FiniteDuration

Attributes

Inherited from:
AdminOps
protected val autoCreateTopics: Boolean

Attributes

Inherited from:
KafkaOps
protected val brokerId: Short

Attributes

Inherited from:
KafkaOps
protected val consumerPollingTimeout: FiniteDuration

Attributes

Inherited from:
ConsumerOps
protected val logCleanerDedupeBufferSize: Int

Attributes

Inherited from:
KafkaOps
protected val producerPublishTimeout: FiniteDuration

Attributes

Inherited from:
ProducerOps
protected val topicCreationTimeout: FiniteDuration

Attributes

Inherited from:
AdminOps
protected val topicDeletionTimeout: FiniteDuration

Attributes

Inherited from:
AdminOps
val zkConnectionTimeoutMs: Int

Attributes

Inherited from:
AdminOps
val zkSessionTimeoutMs: Int

Attributes

Inherited from:
AdminOps
Generated with