kafka4m
package kafka4m
The high-level API space for kafka consumers
- Alphabetic
- By Inheritance
- kafka4m
- AnyRef
- Any
- Hide All
- Show All
- Public
- All
Value Members
-
def
adminTopic(config: Config): String
- config
the root configuration
- returns
the admin topic as per the config
- def byteArrayProducer[A](config: Config)(implicit apr: Aux[A, Key, Bytes]): RichKafkaProducer[Key, Bytes]
-
def
closeOnComplete(config: Config): Boolean
- config
the kafka4m config
- returns
true if observables should be closed when complete
-
def
consumerTopics(config: Config): Set[String]
- config
the root configuration
- returns
the consumer topic as per the config
-
def
ensureTopicBlocking(config: Config)(implicit ec: ExecutionContext): Option[String]
Kafka Streams will fail if the topic does not yet exist.
Kafka Streams will fail if the topic does not yet exist. This way we can provide a means to 'getOrCreate' a topic if that's how it's configured.
- def fireAndForget(config: Config): Boolean
-
def
kafkaConsumer(config: Config)(implicit scheduler: Scheduler): RichKafkaConsumer[String, Array[Byte]]
- config
the kafka4m config
- returns
A RichKafkaConsumer from the given kafka4m configuration
-
def
kafkaProducer[A, K, V](config: Config)(implicit apr: Aux[A, K, V]): RichKafkaProducer[K, V]
- config
the kafka4m config
- returns
a RichKafkaProducer for the given config
-
def
producerTopic(config: Config): String
- config
the root configuration
- returns
the producer topic as per the config
-
def
read(config: Config)(implicit scheduler: Scheduler): Observable[ConsumerRecord[Key, Bytes]]
- config
the kafka4m configuration which contains the 'kafka4m.consumer' values
- returns
an Observable of data coming from kafka. The offsets, etc will be controlled by the kafka4m.consumer configuration, which includes default offset strategy, etc.
-
def
write[A](config: Config)(implicit ev: Aux[A, Key, Bytes]): Consumer[A, Long]
- A
any type A which can be converted into a kafka ProducerRecord
- config
the kafka4m configuration
- returns
a consumer of the 'A' values and produce the number written
-
def
writeKeyAndBytes(config: Config = ConfigFactory.load()): Consumer[(String, Array[Byte]), Long]
- config
the kafka4m configuration
- returns
a consumer which will consume a stream of key/byte-array values into kafka and return the number written
-
def
writeText(config: Config = ConfigFactory.load()): Consumer[String, Long]
- config
the kafka4m configuration
- returns
a consumer which will consume raw text data and write it with null keys
-
object
Kafka4mApp extends ConfigApp with StrictLogging
An ETL entry point to read data into or out of kafka