Class/Object

io.gearpump.streaming.kafka.lib

KafkaSourceConfig

Related Docs: object KafkaSourceConfig | package lib

Permalink

class KafkaSourceConfig extends Serializable

this class extends kafka kafka.consumer.ConsumerConfig with specific configs for KafkaSource

Linear Supertypes
Serializable, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. KafkaSourceConfig
  2. Serializable
  3. AnyRef
  4. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new KafkaSourceConfig(consumerProps: Properties = new Properties)

    Permalink

    consumerProps

    kafka consumer config

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any
  5. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  6. def consumerConfig: ConsumerConfig

    Permalink
  7. val consumerProps: Properties

    Permalink

    kafka consumer config

  8. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  9. def equals(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  10. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  11. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any
  12. def getConsumerStartOffset: Long

    Permalink
  13. def getConsumerTopics: List[String]

    Permalink

    returns

    a list of kafka consumer topics

  14. def getFetchSleepMS: Int

    Permalink

    consumer.FetchThread will sleep for a while if no more messages or the incoming queue size is above the io.gearpump.streaming.kafka.lib.KafkaSourceConfig.FETCH_THRESHOLD this is to get sleep interval

    consumer.FetchThread will sleep for a while if no more messages or the incoming queue size is above the io.gearpump.streaming.kafka.lib.KafkaSourceConfig.FETCH_THRESHOLD this is to get sleep interval

    returns

    sleep interval in milliseconds

  15. def getFetchThreshold: Int

    Permalink

    io.gearpump.streaming.kafka.lib.consumer.FetchThread stops fetching new messages if its incoming queue size is above the threshold and starts again when the queue size is below it

    io.gearpump.streaming.kafka.lib.consumer.FetchThread stops fetching new messages if its incoming queue size is above the threshold and starts again when the queue size is below it

    returns

    fetch threshold

  16. def getGrouper: KafkaGrouper

    Permalink

    get io.gearpump.streaming.kafka.lib.grouper.KafkaGrouper instance, which defines how kafka.common.TopicAndPartitions are mapped to source tasks

  17. def hashCode(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  18. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any
  19. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  20. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef
  21. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef
  22. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef
  23. def toString(): String

    Permalink
    Definition Classes
    AnyRef → Any
  24. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  25. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  26. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  27. def withConsumerStartOffset(earliestOrLatest: Long): KafkaSourceConfig

    Permalink
  28. def withConsumerTopics(topics: String): KafkaSourceConfig

    Permalink

    set kafka consumer topics

    set kafka consumer topics

    topics

    comma-separated string

    returns

    new KafkaConfig based on this but with io.gearpump.streaming.kafka.lib.KafkaSourceConfig.CONSUMER_TOPICS set to given value

  29. def withFetchSleepMS(sleepMS: Int): KafkaSourceConfig

    Permalink

    consumer.FetchThread will sleep for a while if no more messages or the incoming queue size is above the io.gearpump.streaming.kafka.lib.KafkaSourceConfig.FETCH_THRESHOLD this is to set sleep interval

    consumer.FetchThread will sleep for a while if no more messages or the incoming queue size is above the io.gearpump.streaming.kafka.lib.KafkaSourceConfig.FETCH_THRESHOLD this is to set sleep interval

    sleepMS

    sleep interval in milliseconds

    returns

    new KafkaConfig based on this but with io.gearpump.streaming.kafka.lib.KafkaSourceConfig.FETCH_SLEEP_MS set to given value

  30. def withFetchThreshold(threshold: Int): KafkaSourceConfig

    Permalink

    consumer.FetchThread stops fetching new messages if its incoming queue size is above the threshold and starts again when the queue size is below it

    consumer.FetchThread stops fetching new messages if its incoming queue size is above the threshold and starts again when the queue size is below it

    threshold

    queue size

    returns

    new KafkaConfig based on this but with io.gearpump.streaming.kafka.lib.KafkaSourceConfig.FETCH_THRESHOLD set to give value

  31. def withGrouper(className: String): KafkaSourceConfig

    Permalink

    set io.gearpump.streaming.kafka.lib.grouper.KafkaGrouper, which defines how kafka.common.TopicAndPartitions are mapped to source tasks

    set io.gearpump.streaming.kafka.lib.grouper.KafkaGrouper, which defines how kafka.common.TopicAndPartitions are mapped to source tasks

    className

    name of the factory class

    returns

    new KafkaConfig based on this but with io.gearpump.streaming.kafka.lib.KafkaSourceConfig.GROUPER_CLASS set to given value

Inherited from Serializable

Inherited from AnyRef

Inherited from Any

Ungrouped