Package

com.twitter.finatra.kafkastreams.transformer.stores

internal

Permalink

package internal

Visibility
  1. Public
  2. All

Type Members

  1. class CachingFinatraKeyValueStoreImpl[K, V] extends FinatraKeyValueStore[K, V] with Logging

    Permalink

    A write-behind caching layer around the FinatraKeyValueStore.

    A write-behind caching layer around the FinatraKeyValueStore.

    We cache Java objects here and then periodically flush entries into RocksDB which involves serializing the objects into byte arrays. As such this cache: 1) Reduces the number of reads/writes to RocksDB 2) Reduces the number of serialization/deserialization operations which can be expensive for some classes 3) Reduces the number of publishes to the Kafka changelog topic backing this key value store

    This caching does introduce a few odd corner cases :-( 1. Items in the cache have pass-by-reference semantics but items in rocksdb have pass-by-value semantics. Modifying items after a put is a bad idea! Ideally, only immutable objects would be stored in a CachingFinatraKeyValueStore 2. Range queries currently only work against the uncached RocksDB data. This is because sorted Java maps are much less performant than their unsorted counterparts. We typically only use range queries for queryable state where it is ok to read stale data If fresher data is required for range queries, decrease your commit interval, or disable caching on your key value store

    This class is inspired by: https://github.com/apache/samza/blob/1.0.0/samza-kv/src/main/scala/org/apache/samza/storage/kv/CachedStore.scala

  2. class FinatraKeyValueStoreImpl[K, V] extends FinatraKeyValueStore[K, V] with Logging

    Permalink
  3. case class FinatraTransformerLifecycleKeyValueStore[K, V](name: String, flushListener: Option[(String, K, V) ⇒ Unit] = None) extends FinatraKeyValueStore[K, V] with Logging with Product with Serializable

    Permalink

    A FinatraKeyValueStore which allows FinatraTransformer#getKeyValueStore to retrieve a key value store in it's static constructor and then have that returned store participate in the Kafka Streams init and close lifecycle

    A FinatraKeyValueStore which allows FinatraTransformer#getKeyValueStore to retrieve a key value store in it's static constructor and then have that returned store participate in the Kafka Streams init and close lifecycle

    name

    State store name

    flushListener

    Callback that will be called every time a cached store entry is flushed into the underlying RocksDB store

  4. case class MetricsFinatraKeyValueStore[K, V](inner: FinatraKeyValueStore[K, V], statsReceiver: StatsReceiver) extends FinatraKeyValueStore[K, V] with Product with Serializable

    Permalink

    Metrics wrapper around a FinatraKeyValueStore used for recording latency of methods

  5. case class Timer[K](time: Time, metadata: TimerMetadata, key: K) extends Product with Serializable

    Permalink

    time

    Time to fire the timer

Value Members

  1. object FinatraStoresGlobalManager extends Logging

    Permalink

    This class stores a global list of all Finatra Key Value Stores for use by Finatra's queryable state functionality.

    This class stores a global list of all Finatra Key Value Stores for use by Finatra's queryable state functionality.

    Note: We maintain or own global list of state stores so we can directly retrieve the FinatraKeyValueStore responsible for the queried key instead of having to iterate through all key value stores hosted in this Kafka Streams instance.

  2. object MetricsFinatraKeyValueStore extends Serializable

    Permalink

Ungrouped