com.twitter.algebird

ExpHist

case class ExpHist(conf: Config, buckets: Vector[Bucket], total: Long, time: Timestamp) extends Product with Serializable

Exponential Histogram algorithm from http://www-cs-students.stanford.edu/~datar/papers/sicomp_streams.pdf

An Exponential Histogram is a sliding window counter that can guarantee a bounded relative error. You configure the data structure with

- epsilon, the relative error you're willing to tolerate - windowSize, the number of time ticks that you want to track

You interact with the data structure by adding (number, timestamp) pairs into the exponential histogram. querying it for an approximate counts with guess.

The approximate count is guaranteed to be within conf.epsilon relative error of the true count seen across the supplied windowSize.

Next steps:

- efficient serialization - Query EH with a shorter window than the configured window - Discussion of epsilon vs memory tradeoffs

conf

the config values for this instance.

buckets

Vector of timestamps of each (powers of 2) ticks. This is the key to the exponential histogram representation. See ExpHist.Canonical for more info.

total

total ticks tracked. total == buckets.map(_.size).sum

time

current timestamp of this instance.

Linear Supertypes
Serializable, Serializable, Product, Equals, AnyRef, Any
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. ExpHist
  2. Serializable
  3. Serializable
  4. Product
  5. Equals
  6. AnyRef
  7. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Instance Constructors

  1. new ExpHist(conf: Config, buckets: Vector[Bucket], total: Long, time: Timestamp)

    conf

    the config values for this instance.

    buckets

    Vector of timestamps of each (powers of 2) ticks. This is the key to the exponential histogram representation. See ExpHist.Canonical for more info.

    total

    total ticks tracked. total == buckets.map(_.size).sum

    time

    current timestamp of this instance.

Value Members

  1. final def !=(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  2. final def !=(arg0: Any): Boolean

    Definition Classes
    Any
  3. final def ##(): Int

    Definition Classes
    AnyRef → Any
  4. final def ==(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  5. final def ==(arg0: Any): Boolean

    Definition Classes
    Any
  6. def add(delta: Long, ts: Timestamp): ExpHist

    Increment ExpHist by delta at the supplied timestamp.

  7. def addAll(unsorted: Vector[Bucket]): ExpHist

    Efficiently add many buckets at once.

    Efficiently add many buckets at once.

    unsorted

    vector of buckets. All timestamps must be >= this.time.

    returns

    ExpHist instance with all buckets added, stepped forward to the max timestamp in unsorted.

  8. def approximateSum: Approximate[Long]

    Returns an Approximate instance encoding the bounds and the closest long to the estimated sum tracked by this instance.

  9. final def asInstanceOf[T0]: T0

    Definition Classes
    Any
  10. val buckets: Vector[Bucket]

    Vector of timestamps of each (powers of 2) ticks.

    Vector of timestamps of each (powers of 2) ticks. This is the key to the exponential histogram representation. See ExpHist.Canonical for more info.

  11. def clone(): AnyRef

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  12. val conf: Config

    the config values for this instance.

  13. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  14. def finalize(): Unit

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  15. def fold: Fold[Bucket, ExpHist]

    Returns a Fold instance that uses add to accumulate deltas into this exponential histogram instance.

  16. final def getClass(): Class[_]

    Definition Classes
    AnyRef → Any
  17. def guess: Double

    Estimate of the count seen across the last conf.

    Estimate of the count seen across the last conf.windowSize timestamps. Guaranteed to be within conf.epsilon of the true count.

  18. def inc(ts: Timestamp): ExpHist

    Increment ExpHist by 1 at the supplied timestamp.

  19. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any
  20. def lowerBoundSum: Long

    Smallest possible count seen in the last conf.

    Smallest possible count seen in the last conf.windowSize timestamps.

  21. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  22. final def notify(): Unit

    Definition Classes
    AnyRef
  23. final def notifyAll(): Unit

    Definition Classes
    AnyRef
  24. def oldestBucketSize: Long

  25. def relativeError: Double

    relative error of guess, guaranteed to be <= conf.

    relative error of guess, guaranteed to be <= conf.epsilon.

  26. def step(newTime: Timestamp): ExpHist

    Steps this instance forward to the new supplied time.

    Steps this instance forward to the new supplied time. Any buckets with a timestamp <= (newTime - conf.windowSize) will be evicted.

    newTime

    the new current time.

    returns

    ExpHist instance stepped forward to newTime.

  27. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef
  28. val time: Timestamp

    current timestamp of this instance.

  29. def toString(): String

    Definition Classes
    ExpHist → AnyRef → Any
  30. val total: Long

    total ticks tracked.

    total ticks tracked. total == buckets.map(_.size).sum

  31. def upperBoundSum: Long

    Largest possible count seen in the last conf.

    Largest possible count seen in the last conf.windowSize timestamps.

  32. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  33. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  34. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from Serializable

Inherited from Serializable

Inherited from Product

Inherited from Equals

Inherited from AnyRef

Inherited from Any

Ungrouped