Package

com.twitter

algebird

Permalink

package algebird

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. algebird
  2. AnyRef
  3. Any
  1. Hide All
  2. Show all
Visibility
  1. Public
  2. All

Type Members

  1. abstract class AbstractApplicative[M[_]] extends Applicative[M]

    Permalink

    For use from Java/minimizing code bloat in scala

  2. trait AbstractEventuallyAggregator[A, E, O, C] extends Aggregator[A, Either[E, O], C]

    Permalink
  3. abstract class AbstractField[T] extends Field[T]

    Permalink
  4. abstract class AbstractFunctor[M[_]] extends Functor[M]

    Permalink

    For use from Java/minimizing code bloat in scala

  5. abstract class AbstractGroup[T] extends Group[T]

    Permalink
  6. abstract class AbstractMonad[M[_]] extends Monad[M]

    Permalink

    For use from Java/minimizing code bloat in scala

  7. abstract class AbstractMonoid[T] extends Monoid[T]

    Permalink
  8. abstract class AbstractRing[T] extends Ring[T]

    Permalink
  9. abstract class AbstractSemigroup[T] extends Semigroup[T]

    Permalink
  10. class AdaptiveCache[K, V] extends StatefulSummer[Map[K, V]]

    Permalink

    This is a wrapper around SummingCache that attempts to grow the capacity by up to some maximum, as long as there's enough RAM.

    This is a wrapper around SummingCache that attempts to grow the capacity by up to some maximum, as long as there's enough RAM. It determines that there's enough RAM to grow by maintaining a SentinelCache which keeps caching and summing the evicted values. Once the SentinelCache has grown to the same size as the current cache, plus some margin, without running out of RAM, then this indicates that we have enough headroom to double the capacity.

  11. sealed trait AdaptiveVector[V] extends IndexedSeq[V]

    Permalink

    An IndexedSeq that automatically switches representation between dense and sparse depending on sparsity Should be an efficient representation for all sizes, and it should not be necessary to special case immutable algebras based on the sparsity of the vectors.

  12. case class AdjoinedUnit[T](ones: BigInt, get: T) extends Product with Serializable

    Permalink

    This is for the case where your Ring[T] is a Rng (i.e.

    This is for the case where your Ring[T] is a Rng (i.e. there is no unit).

    See also

    http://en.wikipedia.org/wiki/Pseudo-ring#Adjoining_an_identity_element

  13. class AdjoinedUnitRing[T] extends Ring[AdjoinedUnit[T]]

    Permalink
  14. case class AffineFunction[R](slope: R, intercept: R) extends Serializable with Product with Serializable

    Permalink

    Represents functions of the kind: f(x) = slope * x + intercept

  15. class AffineFunctionMonoid[R] extends Monoid[AffineFunction[R]]

    Permalink

    This feeds the value in on the LEFT!!! This may seem counter intuitive, but with this approach, a stream/iterator which is summed will have the same output as applying the function one at a time in order to the input.

    This feeds the value in on the LEFT!!! This may seem counter intuitive, but with this approach, a stream/iterator which is summed will have the same output as applying the function one at a time in order to the input. If we did the "lexigraphically correct" thing, which might be (f+g)(x) = f(g(x)) then we would wind up reversing the list in the sum. (f1 + f2)(x) = f2(f1(x)) so that: listOfFn.foldLeft(x) { (v, fn) => fn(v) } = (Monoid.sum(listOfFn))(x)

  16. trait Aggregator[-A, B, +C] extends Serializable

    Permalink

    This is a type that models map/reduce(map).

    This is a type that models map/reduce(map). First each item is mapped, then we reduce with a semigroup, then finally we present the results.

    Unlike Fold, Aggregator keeps it's middle aggregation type externally visible. This is because Aggregators are useful in parallel map/reduce systems where there may be some additional types needed to cross the map/reduce boundary (such a serialization and intermediate storage). If you don't care about the middle type, an _ may be used and the main utility of the instance is still preserved (e.g. def operate[T](ag: Aggregator[T, _, Int]): Int)

    Note, join is very useful to combine multiple aggregations with one pass. Also GeneratedTupleAggregator.fromN((agg1, agg2, ... aggN)) can glue these together well.

    This type is the the Fold.M from Haskell's fold package: https://hackage.haskell.org/package/folds-0.6.2/docs/Data-Fold-M.html

  17. class AggregatorApplicative[I] extends Applicative[[O]Aggregator[I, _, O]]

    Permalink

    Aggregators are Applicatives, but this hides the middle type.

    Aggregators are Applicatives, but this hides the middle type. If you need a join that does not hide the middle type use join on the trait, or GeneratedTupleAggregator.fromN

  18. final case class AndVal(get: Boolean) extends AnyVal with Product with Serializable

    Permalink
  19. trait Applicative[M[_]] extends Functor[M]

    Permalink

    Simple implementation of an Applicative type-class.

    Simple implementation of an Applicative type-class. There are many choices for the canonical second operation (join, sequence, joinWith, ap), all equivalent. For a Functor modeling concurrent computations with failure, like Future, combining results with join can save a lot of time over combining with flatMap. (Given two operations, if the second fails before the first completes, one can fail the entire computation right then. With flatMap, one would have to wait for the first operation to complete before failing it.)

    Laws Applicatives must follow: map(apply(x))(f) == apply(f(x)) join(apply(x), apply(y)) == apply((x, y)) (sequence and joinWith specialize join - they should behave appropriately)

    Annotations
    @implicitNotFound( ... )
  20. class ApplicativeField[T, M[_]] extends ApplicativeRing[T, M] with Field[M[T]]

    Permalink

    Group, Ring, and Field ARE NOT AUTOMATIC.

    Group, Ring, and Field ARE NOT AUTOMATIC. You have to check that the laws hold for your Applicative. If your M[_] is a wrapper type (Option[_], Some[_], Try[_], Future[_], etc...) this generally works.

  21. class ApplicativeGroup[T, M[_]] extends ApplicativeMonoid[T, M] with Group[M[T]]

    Permalink

    Group, Ring, and Field ARE NOT AUTOMATIC.

    Group, Ring, and Field ARE NOT AUTOMATIC. You have to check that the laws hold for your Applicative. If your M[_] is a wrapper type (Option[_], Some[_], Try[_], Future[_], etc...) this generally works.

  22. class ApplicativeMonoid[T, M[_]] extends ApplicativeSemigroup[T, M] with Monoid[M[T]]

    Permalink

    This is a Monoid, for all Applicatives.

  23. class ApplicativeOperators[A, M[_]] extends FunctorOperators[A, M]

    Permalink

    This enrichment allows us to use our Applicative instances in for expressions: if (import Applicative._) has been done

  24. class ApplicativeRing[T, M[_]] extends ApplicativeGroup[T, M] with Ring[M[T]]

    Permalink

    Group, Ring, and Field ARE NOT AUTOMATIC.

    Group, Ring, and Field ARE NOT AUTOMATIC. You have to check that the laws hold for your Applicative. If your M[_] is a wrapper type (Option[_], Some[_], Try[_], Future[_], etc...) this generally works.

  25. class ApplicativeSemigroup[T, M[_]] extends Semigroup[M[T]]

    Permalink

    This is a Semigroup, for all Applicatives.

  26. case class Approximate[N](min: N, estimate: N, max: N, probWithinBounds: Double)(implicit numeric: Numeric[N]) extends Product with Serializable

    Permalink
  27. case class ApproximateBoolean(isTrue: Boolean, withProb: Double) extends Product with Serializable

    Permalink
  28. abstract class ArrayBufferedOperation[I, O] extends Buffered[I, O]

    Permalink
  29. class ArrayGroup[T] extends ArrayMonoid[T] with Group[Array[T]]

    Permalink

    Extends pair-wise sum Array monoid into a Group negate is defined as the negation of each element of the array.

  30. class ArrayMonoid[T] extends Monoid[Array[T]]

    Permalink

    Pair-wise sum Array monoid.

    Pair-wise sum Array monoid.

    plus returns left[i] + right[i] for all array elements. The resulting array will be as long as the longest array (with its elements duplicated) zero is an empty array

  31. case class AveragedValue(count: Long, value: Double) extends Product with Serializable

    Permalink
  32. sealed abstract class BF extends Serializable

    Permalink

    Bloom Filter data structure

  33. case class BFHash(numHashes: Int, width: Int, seed: Long = 0L) extends (String) ⇒ Iterable[Int] with Product with Serializable

    Permalink
  34. case class BFInstance(hashes: BFHash, bits: BitSet, width: Int) extends BF with Product with Serializable

    Permalink
  35. case class BFItem(item: String, hashes: BFHash, width: Int) extends BF with Product with Serializable

    Permalink

    Bloom Filter with 1 value.

  36. case class BFSparse(hashes: BFHash, bits: EWAHCompressedBitmap, width: Int) extends BF with Product with Serializable

    Permalink
  37. case class BFZero(hashes: BFHash, width: Int) extends BF with Product with Serializable

    Permalink

    Empty bloom filter.

  38. case class BitSetLite(in: Array[Byte]) extends Product with Serializable

    Permalink

    A super lightweight (hopefully) version of BitSet

  39. case class BloomFilterAggregator(bfMonoid: BloomFilterMonoid) extends MonoidAggregator[String, BF, BF] with Product with Serializable

    Permalink
  40. case class BloomFilterMonoid(numHashes: Int, width: Int, seed: Int) extends Monoid[BF] with Product with Serializable

    Permalink

    Bloom Filter - a probabilistic data structure to test presence of an element.

    Bloom Filter - a probabilistic data structure to test presence of an element.

    Operations 1) insert: hash the value k times, updating the bitfield at the index equal to each hashed value 2) query: hash the value k times. If there are k collisions, then return true; otherwise false.

    http://en.wikipedia.org/wiki/Bloom_filter

  41. trait Buffered[I, O] extends Serializable

    Permalink

    Represents something that consumes I and may emit O.

    Represents something that consumes I and may emit O. Has some internal state that may be used to improve performance. Generally used to model folds or reduces (see BufferedReduce)

  42. trait BufferedReduce[V] extends Buffered[V, V]

    Permalink

    This never emits on put, you must call flush designed to be use in the stackable pattern with ArrayBufferedOperation

  43. class BufferedSumAll[V] extends ArrayBufferedOperation[V, V] with StatefulSummer[V] with BufferedReduce[V]

    Permalink
  44. final case class Bytes(array: Array[Byte]) extends Serializable with Product with Serializable

    Permalink

    A wrapper for Array[Byte] that provides sane implementations of hashCode, equals, and toString.

    A wrapper for Array[Byte] that provides sane implementations of hashCode, equals, and toString. The wrapped array of bytes is assumed to be never modified.

    Note: Unfortunately we cannot make Bytes a value class because a value class may not override the hashCode and equals methods (cf. SIP-15, criterion 4).

    Alternatives

    Instead of wrapping an Array[Byte] with this class you can also convert an Array[Byte] to a Seq[Byte] via Scala's toSeq method:

    val arrayByte: Array[Byte] = Array(1.toByte)
    val seqByte: Seq[Byte] = arrayByte.toSeq

    Like Bytes, a Seq[Byte] has sane hashCode, equals, and toString implementations.

    Performance-wise we found that a Seq[Byte] is comparable to Bytes. For example, a CMS[Seq[Byte]] was measured to be only slightly slower than CMS[Bytes] (think: single-digit percentages).

    array

    the wrapped array of bytes

    See also

    MinHasher

  45. sealed abstract class CMS[K] extends Serializable with CMSCounting[K, CMS]

    Permalink

    A Count-Min sketch data structure that allows for counting and frequency estimation of elements in a data stream.

    A Count-Min sketch data structure that allows for counting and frequency estimation of elements in a data stream.

    Tip: If you also need to track heavy hitters ("Top N" problems), take a look at TopCMS.

    Usage

    This example demonstrates how to count Long elements with CMS, i.e. K=Long.

    Note that the actual counting is always performed with a Long, regardless of your choice of K. That is, the counting table behind the scenes is backed by Long values (at least in the current implementation), and thus the returned frequency estimates are always instances of Approximate[Long].

    K

    The type used to identify the elements to be counted.

    Example:
    1. // Implicits that enabling CMS-hashing of `Long` values.
      import com.twitter.algebird.CMSHasherImplicits._
      // Creates a monoid for a CMS that can count `Long` elements.
      val cmsMonoid: CMSMonoid[Long] = {
        val eps = 0.001
        val delta = 1E-10
        val seed = 1
        CMS.monoid[Long](eps, delta, seed)
      }
      // Creates a CMS instance that has counted the element `1L`.
      val cms: CMS[Long] = cmsMonoid.create(1L)
      // Estimates the frequency of `1L`
      val estimate: Approximate[Long] = cms.frequency(1L)
  46. case class CMSAggregator[K](cmsMonoid: CMSMonoid[K]) extends MonoidAggregator[K, CMS[K], CMS[K]] with Product with Serializable

    Permalink

    An Aggregator for CMS.

    An Aggregator for CMS. Can be created using CMS.aggregator.

  47. trait CMSCounting[K, C[_]] extends AnyRef

    Permalink

    A trait for CMS implementations that can count elements in a data stream and that can answer point queries (i.e.

    A trait for CMS implementations that can count elements in a data stream and that can answer point queries (i.e. frequency estimates) for these elements.

    Known implementations: CMS, TopCMS.

    K

    The type used to identify the elements to be counted.

    C

    The type of the actual CMS that implements this trait.

  48. case class CMSHash[K](a: Int, b: Int, width: Int)(implicit evidence$16: CMSHasher[K]) extends Serializable with Product with Serializable

    Permalink
  49. trait CMSHasher[K] extends Serializable

    Permalink

    The Count-Min sketch uses d (aka depth) pair-wise independent hash functions drawn from a universal hashing family of the form:

    The Count-Min sketch uses d (aka depth) pair-wise independent hash functions drawn from a universal hashing family of the form:

    h(x) = [a * x + b (mod p)] (mod m)

    As a requirement for using CMS you must provide an implicit CMSHasher[K] for the type K of the items you want to count. Algebird ships with several such implicits for commonly used types K such as Long and BigInt:

    import com.twitter.algebird.CMSHasherImplicits._

    If your type K is not supported out of the box, you have two options: 1) You provide a "translation" function to convert items of your (unsupported) type K to a supported type such as Double, and then use the contramap function of CMSHasher to create the required CMSHasher[K] for your type (see the documentation of contramap for an example); 2) You implement a CMSHasher[K] from scratch, using the existing CMSHasher implementations as a starting point.

  50. trait CMSHeavyHitters[K] extends AnyRef

    Permalink

    A trait for CMS implementations that can track heavy hitters in a data stream.

    A trait for CMS implementations that can track heavy hitters in a data stream.

    It is up to the implementation how the semantics of tracking heavy hitters are defined. For instance, one implementation could track the "top %" heavy hitters whereas another implementation could track the "top N" heavy hitters.

    Known implementations: TopCMS.

    K

    The type used to identify the elements to be counted.

  51. case class CMSInstance[K](countsTable: CountsTable[K], totalCount: Long, params: CMSParams[K]) extends CMS[K] with Product with Serializable

    Permalink

    The general Count-Min sketch structure, used for holding any number of elements.

  52. case class CMSItem[K](item: K, totalCount: Long, params: CMSParams[K]) extends CMS[K] with Product with Serializable

    Permalink

    Used for holding a single element, to avoid repeatedly adding elements from sparse counts tables.

  53. class CMSMonoid[K] extends Monoid[CMS[K]]

    Permalink

    Monoid for adding CMS sketches.

    Monoid for adding CMS sketches.

    Usage

    eps and delta are parameters that bound the error of each query estimate. For example, errors in answering point queries (e.g., how often has element x appeared in the stream described by the sketch?) are often of the form: "with probability p >= 1 - delta, the estimate is close to the truth by some factor depending on eps."

    The type K is the type of items you want to count. You must provide an implicit CMSHasher[K] for K, and Algebird ships with several such implicits for commonly used types such as Long and BigInt:

    import com.twitter.algebird.CMSHasherImplicits._

    If your type K is not supported out of the box, you have two options: 1) You provide a "translation" function to convert items of your (unsupported) type K to a supported type such as Double, and then use the contramap function of CMSHasher to create the required CMSHasher[K] for your type (see the documentation of CMSHasher for an example); 2) You implement a CMSHasher[K] from scratch, using the existing CMSHasher implementations as a starting point.

    Note: Because Arrays in Scala/Java not have sane equals and hashCode implementations, you cannot safely use types such as Array[Byte]. Extra work is required for Arrays. For example, you may opt to convert Array[T] to a Seq[T] via toSeq, or you can provide appropriate wrapper classes. Algebird provides one such wrapper class, Bytes, to safely wrap an Array[Byte] for use with CMS.

    K

    The type used to identify the elements to be counted. For example, if you want to count the occurrence of user names, you could map each username to a unique numeric ID expressed as a Long, and then count the occurrences of those Longs with a CMS of type K=Long. Note that this mapping between the elements of your problem domain and their identifiers used for counting via CMS should be bijective. We require a CMSHasher context bound for K, see CMSHasherImplicits for available implicits that can be imported. Which type K should you pick in practice? For domains that have less than 2^64 unique elements, you'd typically use Long. For larger domains you can try BigInt, for example. Other possibilities include Spire's SafeLong and Numerical data types (https://github.com/non/spire), though Algebird does not include the required implicits for CMS-hashing (cf. CMSHasherImplicits.

  54. case class CMSParams[K](hashes: Seq[CMSHash[K]], eps: Double, delta: Double) extends Product with Serializable

    Permalink

    Configuration parameters for CMS.

    Configuration parameters for CMS.

    K

    The type used to identify the elements to be counted.

    hashes

    Pair-wise independent hashes functions. We need N=depth such functions (depth can be derived from delta).

    eps

    One-sided error bound on the error of each point query, i.e. frequency estimate.

    delta

    A bound on the probability that a query estimate does not lie within some small interval (an interval that depends on eps) around the truth.

  55. case class CMSZero[K](params: CMSParams[K]) extends CMS[K] with Product with Serializable

    Permalink

    Zero element.

    Zero element. Used for initialization.

  56. class CassandraMurmurHash extends AnyRef

    Permalink
  57. class ConstantGroup[T] extends Group[T]

    Permalink
  58. case class DecayedValue(value: Double, scaledTime: Double) extends Ordered[DecayedValue] with Product with Serializable

    Permalink
  59. case class DecayedValueMonoid(eps: Double) extends Monoid[DecayedValue] with Product with Serializable

    Permalink
  60. case class DecayedVector[C[_]](vector: C[Double], scaledTime: Double) extends Product with Serializable

    Permalink
  61. case class DenseHLL(bits: Int, v: Bytes) extends HLL with Product with Serializable

    Permalink

    These are the individual instances which the Monoid knows how to add

  62. case class DenseVector[V](iseq: Vector[V], sparseValue: V, denseCount: Int) extends AdaptiveVector[V] with Product with Serializable

    Permalink
  63. class DivOp[T] extends AnyRef

    Permalink
  64. class EitherMonoid[L, R] extends EitherSemigroup[L, R] with Monoid[Either[L, R]]

    Permalink
  65. class EitherSemigroup[L, R] extends Semigroup[Either[L, R]]

    Permalink

    Either semigroup is useful for error handling.

    Either semigroup is useful for error handling. if everything is correct, use Right (it's right, get it?), if something goes wrong, use Left. plus does the normal thing for plus(Right, Right), or plus(Left, Left), but if exactly one is Left, we return that value (to keep the error condition). Typically, the left value will be a string representing the errors.

  66. case class Empty[T]() extends Interval[T] with Product with Serializable

    Permalink
  67. trait EventuallyAggregator[A, E, O, C] extends AbstractEventuallyAggregator[A, E, O, C]

    Permalink
  68. class EventuallyGroup[E, O] extends EventuallyMonoid[E, O] with Group[Either[E, O]]

    Permalink

    See also

    EventuallySemigroup

  69. class EventuallyMonoid[E, O] extends EventuallySemigroup[E, O] with Monoid[Either[E, O]]

    Permalink

    See also

    EventuallySemigroup

  70. trait EventuallyMonoidAggregator[A, E, O, C] extends AbstractEventuallyAggregator[A, E, O, C] with MonoidAggregator[A, Either[E, O], C]

    Permalink
  71. class EventuallyRing[E, O] extends EventuallyGroup[E, O] with Ring[Either[E, O]]

    Permalink

    See also

    EventuallySemigroup

  72. class EventuallySemigroup[E, O] extends Semigroup[Either[E, O]]

    Permalink

    Classes that support algebraic structures with dynamic switching between two representations, the original type O and the eventual type E.

    Classes that support algebraic structures with dynamic switching between two representations, the original type O and the eventual type E. In the case of Semigroup, we specify - Two Semigroups eventualSemigroup and originalSemigroup - A Semigroup homomorphism convert: O => E - A conditional mustConvert: O => Boolean Then we get a Semigroup[Either[E,O]], where: Left(x) + Left(y) = Left(x+y) Left(x) + Right(y) = Left(x+convert(y)) Right(x) + Left(y) = Left(convert(x)+y) Right(x) + Right(y) = Left(convert(x+y)) if mustConvert(x+y) Right(x+y) otherwise. EventuallyMonoid, EventuallyGroup, and EventuallyRing are defined analogously, with the contract that convert respect the appropriate structure.

  73. case class ExclusiveLower[T](lower: T)(implicit ordering: Ordering[T]) extends Interval[T] with Lower[T] with Product with Serializable

    Permalink
  74. case class ExclusiveUpper[T](upper: T)(implicit ordering: Ordering[T]) extends Interval[T] with Upper[T] with Product with Serializable

    Permalink
  75. trait Field[T] extends Ring[T]

    Permalink

    Field: Ring + division.

    Field: Ring + division. It is a generalization of Ring and adds support for inversion and multiplicative identity.

    Annotations
    @implicitNotFound( ... )
  76. case class First[+T](get: T) extends Product with Serializable

    Permalink
  77. trait FlatMapPreparer[A, T] extends Preparer[A, T]

    Permalink

    A Preparer that has had one or more flatMap operations applied.

    A Preparer that has had one or more flatMap operations applied. It can only accept MonoidAggregators.

  78. sealed trait Fold[-I, +O] extends Serializable

    Permalink

    Folds are first-class representations of "Traversable.foldLeft." They have the nice property that they can be fused to work in parallel over an input sequence.

    Folds are first-class representations of "Traversable.foldLeft." They have the nice property that they can be fused to work in parallel over an input sequence.

    A Fold accumulates inputs (I) into some internal type (X), converting to a defined output type (O) when done. We use existential types to hide internal details and to allow for internal and external (X and O) types to differ for "map" and "join."

    In discussing this type we draw parallels to Function1 and related types. You can think of a fold as a function "Seq[I] => O" but in reality we do not have to materialize the input sequence at once to "run" the fold.

    The traversal of the input data structure is NOT done by Fold itself. Instead we expose some methods like "overTraversable" that know how to iterate through various sequence types and drive the fold. We also expose some internal state so library authors can fold over their own types.

    See the companion object for constructors.

  79. class FoldApplicative[I] extends Applicative[[O]Fold[I, O]]

    Permalink

    Folds are Applicatives!

  80. final class FoldState[X, -I, +O] extends Serializable

    Permalink

    A FoldState defines a left fold with a "hidden" accumulator type.

    A FoldState defines a left fold with a "hidden" accumulator type. It is exposed so library authors can run Folds over their own sequence types.

    The fold can be executed correctly according to the properties of "add" and your traversed data structure. For example, the "add" function of a monoidal fold will be associative. A FoldState is valid for only one iteration because the accumulator (seeded by "start" may be mutable.

    The three components of a fold are add: (X, I) => X - updates and returns internal state for every input I start: X - the initial state end: X => O - transforms internal state to a final result

    Folding over Seq(x, y) would produce the result end(add(add(start, x), y))

  81. class Function1Monoid[T] extends Monoid[(T) ⇒ T]

    Permalink

    Function1 monoid.

    Function1 monoid. plus means function composition, zero is the identity function

  82. trait Functor[M[_]] extends AnyRef

    Permalink

    Simple implementation of a Functor type-class.

    Simple implementation of a Functor type-class.

    Laws Functors must follow: map(m)(id) == m map(m)(f andThen g) == map(map(m)(f))(g)

    Annotations
    @implicitNotFound( ... )
  83. class FunctorOperators[A, M[_]] extends AnyRef

    Permalink

    This enrichment allows us to use our Functor instances in for expressions: if (import Functor._) has been done

  84. case class GenHLLAggregator[K](hllMonoid: HyperLogLogMonoid, hash: Hash128[K]) extends MonoidAggregator[K, HLL, HLL] with Product with Serializable

    Permalink
  85. trait GeneratedGroupImplicits extends AnyRef

    Permalink
  86. trait GeneratedMonoidImplicits extends AnyRef

    Permalink
  87. trait GeneratedRingImplicits extends AnyRef

    Permalink
  88. trait GeneratedSemigroupImplicits extends AnyRef

    Permalink
  89. trait GeneratedTupleAggregator extends AnyRef

    Permalink
  90. abstract class GenericMapMonoid[K, V, M <: Map[K, V]] extends Monoid[M] with MapOperations[K, V, M]

    Permalink
  91. trait GenericMapRing[K, V, M <: Map[K, V]] extends Ring[M] with MapOperations[K, V, M]

    Permalink

    You can think of this as a Sparse vector ring

  92. trait Group[T] extends Monoid[T]

    Permalink

    Group: this is a monoid that also has subtraction (and negation): So, you can do (a-b), or -a (which is equal to 0 - a).

    Group: this is a monoid that also has subtraction (and negation): So, you can do (a-b), or -a (which is equal to 0 - a).

    Annotations
    @implicitNotFound( ... )
  93. sealed abstract class HLL extends Serializable

    Permalink
  94. case class HLLSeries(bits: Int, rows: Vector[Map[Int, Long]]) extends Product with Serializable

    Permalink

    HLLSeries can produce a HyperLogLog counter for any window into the past, using a constant factor more space than HyperLogLog.

    HLLSeries can produce a HyperLogLog counter for any window into the past, using a constant factor more space than HyperLogLog.

    For each hash bucket, rather than keeping a single max RhoW value, it keeps every RhoW value it has seen, and the max timestamp where it saw that value. This allows it to reconstruct an HLL as it would be had it started at zero at any given point in the past, and seen the same updates this structure has seen.

    bits

    The number of bits to use

    rows

    Vector of maps of RhoW -> max timestamp where it was seen

    returns

    New HLLSeries

  95. trait Hash128[-K] extends Serializable

    Permalink

    A typeclass to represent hashing to 128 bits.

    A typeclass to represent hashing to 128 bits. Used for HLL, but possibly other applications

  96. class HashingTrickMonoid[V] extends Monoid[AdaptiveVector[V]]

    Permalink
  97. case class HeavyHitter[K](item: K, count: Long) extends Serializable with Product with Serializable

    Permalink
  98. case class HeavyHitters[K](hhs: Set[HeavyHitter[K]]) extends Serializable with Product with Serializable

    Permalink

    Containers for holding heavy hitter items and their associated counts.

  99. abstract class HeavyHittersLogic[K] extends Serializable

    Permalink

    Controls how a CMS that implements CMSHeavyHitters tracks heavy hitters.

  100. case class HyperLogLogAggregator(hllMonoid: HyperLogLogMonoid) extends MonoidAggregator[Array[Byte], HLL, HLL] with Product with Serializable

    Permalink
  101. class HyperLogLogMonoid extends Monoid[HLL]

    Permalink
  102. class HyperLogLogSeriesMonoid extends Monoid[HLLSeries]

    Permalink

    val hllSeriesMonoid = new HyperLogLogSeriesMonoid(bits)

    Example Usage

    val hllSeriesMonoid = new HyperLogLogSeriesMonoid(bits)

    val examples: Seq[Array[Byte], Long] val series = examples .map { case (bytes, timestamp) => hllSeriesMonoid.create(bytes, timestamp) } .reduce { hllSeriesMonoid.plus(_,_) }

    val estimate1 = series.since(timestamp1.toLong).toHLL.estimatedSize val estimate2 = series.since(timestamp2.toLong).toHLL.estimatedSize

  103. case class InclusiveLower[T](lower: T)(implicit ordering: Ordering[T]) extends Interval[T] with Lower[T] with Product with Serializable

    Permalink
  104. case class InclusiveUpper[T](upper: T)(implicit ordering: Ordering[T]) extends Interval[T] with Upper[T] with Product with Serializable

    Permalink
  105. class IndexedSeqGroup[T] extends IndexedSeqMonoid[T] with Group[IndexedSeq[T]]

    Permalink
  106. class IndexedSeqMonoid[T] extends IndexedSeqSemigroup[T] with Monoid[IndexedSeq[T]]

    Permalink
  107. class IndexedSeqRing[T] extends IndexedSeqGroup[T] with Ring[IndexedSeq[T]]

    Permalink
  108. class IndexedSeqSemigroup[T] extends Semigroup[IndexedSeq[T]]

    Permalink

    Note that this works similar to Semigroup[Map[Int,T]] not like Semigroup[List[T]] This does element-wise operations, like standard vector math, not concatenation, like Semigroup[String] or Semigroup[List[T]]

    Note that this works similar to Semigroup[Map[Int,T]] not like Semigroup[List[T]] This does element-wise operations, like standard vector math, not concatenation, like Semigroup[String] or Semigroup[List[T]]

    If l.size != r.size, then only sums the elements up to the index min(l.size, r.size); appends the remainder to the result.

  109. class IntegralPredecessible[T] extends Predecessible[T]

    Permalink
  110. class IntegralSuccessible[T] extends Successible[T]

    Permalink
  111. case class Intersection[L[t] <: Lower[t], U[t] <: Upper[t], T](lower: L[T], upper: U[T]) extends Interval[T] with Product with Serializable

    Permalink
  112. sealed trait Interval[T] extends Serializable

    Permalink

    Represents a single interval on a T with an Ordering

  113. class JListMonoid[T] extends Monoid[List[T]]

    Permalink

    Since Lists are mutable, this always makes a full copy.

    Since Lists are mutable, this always makes a full copy. Prefer scala immutable Lists if you use scala immutable lists, the tail of the result of plus is always the right argument

  114. class JMapMonoid[K, V] extends Monoid[Map[K, V]]

    Permalink

    Since maps are mutable, this always makes a full copy.

    Since maps are mutable, this always makes a full copy. Prefer scala immutable maps if you use scala immutable maps, this operation is much faster TODO extend this to Group, Ring

  115. case class Last[+T](get: T) extends Product with Serializable

    Permalink
  116. class ListMonoid[T] extends Monoid[List[T]]

    Permalink

    List concatenation monoid.

    List concatenation monoid. plus means concatenation, zero is empty list

  117. sealed trait Lower[T] extends Interval[T]

    Permalink
  118. trait MapAggregator[A, B, K, C] extends Aggregator[A, B, Map[K, C]]

    Permalink
  119. class MapGroup[K, V] extends MapMonoid[K, V] with Group[Map[K, V]]

    Permalink

    You can think of this as a Sparse vector group

  120. class MapMonoid[K, V] extends GenericMapMonoid[K, V, Map[K, V]]

    Permalink
  121. trait MapMonoidAggregator[A, B, K, C] extends MonoidAggregator[A, B, Map[K, C]]

    Permalink
  122. trait MapOperations[K, V, M <: Map[K, V]] extends AnyRef

    Permalink
  123. trait MapPreparer[A, T] extends Preparer[A, T]

    Permalink

    A Preparer that has had zero or more map transformations applied, but no flatMaps.

    A Preparer that has had zero or more map transformations applied, but no flatMaps. This can produce any type of Aggregator.

  124. class MapRing[K, V] extends MapGroup[K, V] with GenericMapRing[K, V, Map[K, V]]

    Permalink
  125. case class Max[+T](get: T) extends Product with Serializable

    Permalink
  126. case class MaxAggregator[T]()(implicit ord: Ordering[T]) extends Aggregator[T, T, T] with Product with Serializable

    Permalink
  127. trait Metric[-V] extends Serializable

    Permalink
    Annotations
    @implicitNotFound( ... )
  128. case class Min[+T](get: T) extends Product with Serializable

    Permalink
  129. case class MinAggregator[T]()(implicit ord: Ordering[T]) extends Aggregator[T, T, T] with Product with Serializable

    Permalink
  130. final case class MinHashSignature(bytes: Array[Byte]) extends AnyVal with Product with Serializable

    Permalink

    MinHasher as a Monoid operates on this class to avoid the too generic Array[Byte].

    MinHasher as a Monoid operates on this class to avoid the too generic Array[Byte]. The bytes are assumed to be never modified. The only reason we did not use IndexedSeq[Byte] instead of Array[Byte] is because a ByteBuffer is used internally in MinHasher and it can wrap Array[Byte].

  131. abstract class MinHasher[H] extends Monoid[MinHashSignature]

    Permalink

    Instances of MinHasher can create, combine, and compare fixed-sized signatures of arbitrarily sized sets.

    Instances of MinHasher can create, combine, and compare fixed-sized signatures of arbitrarily sized sets.

    A signature is represented by a byte array of approx maxBytes size. You can initialize a signature with a single element, usually a Long or String. You can combine any two set's signatures to produce the signature of their union. You can compare any two set's signatures to estimate their Jaccard similarity. You can use a set's signature to estimate the number of distinct values in the set. You can also use a combination of the above to estimate the size of the intersection of two sets from their signatures. The more bytes in the signature, the more accurate all of the above will be.

    You can also use these signatures to quickly find similar sets without doing n^2 comparisons. Each signature is assigned to several buckets; sets whose signatures end up in the same bucket are likely to be similar. The targetThreshold controls the desired level of similarity - the higher the threshold, the more efficiently you can find all the similar sets.

    This abstract superclass is generic with regards to the size of the hash used. Depending on the number of unique values in the domain of the sets, you may want a MinHasher16, a MinHasher32, or a new custom subclass.

    This implementation is modeled after Chapter 3 of Ullman and Rajaraman's Mining of Massive Datasets: http://infolab.stanford.edu/~ullman/mmds/ch3a.pdf

  132. class MinHasher16 extends MinHasher[Char]

    Permalink
  133. class MinHasher32 extends MinHasher[Int]

    Permalink
  134. sealed trait MinPlus[+V] extends Serializable

    Permalink
  135. class MinPlusSemiring[V] extends Ring[MinPlus[V]]

    Permalink
  136. final case class MinPlusValue[V](get: V) extends AnyVal with MinPlus[V] with Product with Serializable

    Permalink
  137. class MinusOp[T] extends AnyRef

    Permalink
  138. case class Moments(m0: Long, m1: Double, m2: Double, m3: Double, m4: Double) extends Product with Serializable

    Permalink

    A class to calculate the first five central moments over a sequence of Doubles.

    A class to calculate the first five central moments over a sequence of Doubles. Given the first five central moments, we can then calculate metrics like skewness and kurtosis.

    m{i} denotes the ith central moment.

  139. trait Monad[M[_]] extends Applicative[M]

    Permalink

    Simple implementation of a Monad type-class.

    Simple implementation of a Monad type-class. Subclasses only need to override apply and flatMap, but they should override map, join, joinWith, and sequence if there are better implementations.

    Laws Monads must follow: identities: flatMap(apply(x))(fn) == fn(x) flatMap(m)(apply _) == m associativity on flatMap (you can either flatMap f first, or f to g: flatMap(flatMap(m)(f))(g) == flatMap(m) { x => flatMap(f(x))(g) }

    Annotations
    @implicitNotFound( ... )
  140. class MonadOperators[A, M[_]] extends ApplicativeOperators[A, M]

    Permalink

    This enrichment allows us to use our Monad instances in for expressions: if (import Monad._) has been done

  141. trait Monoid[T] extends Semigroup[T]

    Permalink

    Monoid (take a deep breath, and relax about the weird name): This is a semigroup that has an additive identity (called zero), such that a+0=a, 0+a=a, for every a

    Monoid (take a deep breath, and relax about the weird name): This is a semigroup that has an additive identity (called zero), such that a+0=a, 0+a=a, for every a

    Annotations
    @implicitNotFound( ... )
  142. trait MonoidAggregator[-A, B, +C] extends Aggregator[A, B, C]

    Permalink
  143. class MonoidCombinator[A, B] extends SemigroupCombinator[A, B] with Monoid[(A, B)]

    Permalink
  144. final case class MurmurHash128(seed: Long) extends AnyVal with Product with Serializable

    Permalink
  145. class NumericRing[T] extends Ring[T]

    Permalink
  146. class OptionGroup[T] extends OptionMonoid[T] with Group[Option[T]]

    Permalink

    Some(5) - Some(3) == Some(2) Some(5) - Some(5) == None negate Some(5) == Some(-5) Note: Some(0) and None are equivalent under this Group

  147. class OptionMonoid[T] extends Monoid[Option[T]]

    Permalink

    Some(5) + Some(3) == Some(8) Some(5) + None == Some(5)

  148. final case class OrVal(get: Boolean) extends AnyVal with Product with Serializable

    Permalink
  149. class PlusOp[T] extends AnyRef

    Permalink
  150. trait Predecessible[T] extends Serializable

    Permalink

    This is a typeclass to represent things which are countable down.

    This is a typeclass to represent things which are countable down. Note that it is important that a value prev(t) is always less than t. Note that prev returns Option because this class comes with the notion that some items may reach a minimum key, which is None.

  151. sealed trait Preparer[A, T] extends Serializable

    Permalink

    Preparer is a way to build up an Aggregator through composition using a more natural API: it allows you to start with the input type and describe a series of transformations and aggregations from there, rather than starting from the aggregation and composing "outwards" in both directions.

    Preparer is a way to build up an Aggregator through composition using a more natural API: it allows you to start with the input type and describe a series of transformations and aggregations from there, rather than starting from the aggregation and composing "outwards" in both directions.

    Uses of Preparer will always start with a call to Preparer[A], and end with a call to monoidAggregate or a related method, to produce an Aggregator instance.

  152. sealed trait Priority[+P, +F] extends AnyRef

    Permalink

    Priority is a type class for prioritized implicit search.

    Priority is a type class for prioritized implicit search.

    This type class will attempt to provide an implicit instance of P (the preferred type). If that type is not available it will fallback to F (the fallback type). If neither type is available then a Priority[P, F] instance will not be available.

    This type can be useful for problems where multiple algorithms can be used, depending on the type classes available.

    taken from non/algebra until we make algebird depend on non/algebra

  153. class Product10Group[X, A, B, C, D, E, F, G, H, I, J] extends Product10Monoid[X, A, B, C, D, E, F, G, H, I, J] with Group[X]

    Permalink

    Combine 10 groups into a product group

  154. class Product10Monoid[X, A, B, C, D, E, F, G, H, I, J] extends Product10Semigroup[X, A, B, C, D, E, F, G, H, I, J] with Monoid[X]

    Permalink

    Combine 10 monoids into a product monoid

  155. class Product10Ring[X, A, B, C, D, E, F, G, H, I, J] extends Product10Group[X, A, B, C, D, E, F, G, H, I, J] with Ring[X]

    Permalink

    Combine 10 rings into a product ring

  156. class Product10Semigroup[X, A, B, C, D, E, F, G, H, I, J] extends Semigroup[X]

    Permalink

    Combine 10 semigroups into a product semigroup

  157. class Product11Group[X, A, B, C, D, E, F, G, H, I, J, K] extends Product11Monoid[X, A, B, C, D, E, F, G, H, I, J, K] with Group[X]

    Permalink

    Combine 11 groups into a product group

  158. class Product11Monoid[X, A, B, C, D, E, F, G, H, I, J, K] extends Product11Semigroup[X, A, B, C, D, E, F, G, H, I, J, K] with Monoid[X]

    Permalink

    Combine 11 monoids into a product monoid

  159. class Product11Ring[X, A, B, C, D, E, F, G, H, I, J, K] extends Product11Group[X, A, B, C, D, E, F, G, H, I, J, K] with Ring[X]

    Permalink

    Combine 11 rings into a product ring

  160. class Product11Semigroup[X, A, B, C, D, E, F, G, H, I, J, K] extends Semigroup[X]

    Permalink

    Combine 11 semigroups into a product semigroup

  161. class Product12Group[X, A, B, C, D, E, F, G, H, I, J, K, L] extends Product12Monoid[X, A, B, C, D, E, F, G, H, I, J, K, L] with Group[X]

    Permalink

    Combine 12 groups into a product group

  162. class Product12Monoid[X, A, B, C, D, E, F, G, H, I, J, K, L] extends Product12Semigroup[X, A, B, C, D, E, F, G, H, I, J, K, L] with Monoid[X]

    Permalink

    Combine 12 monoids into a product monoid

  163. class Product12Ring[X, A, B, C, D, E, F, G, H, I, J, K, L] extends Product12Group[X, A, B, C, D, E, F, G, H, I, J, K, L] with Ring[X]

    Permalink

    Combine 12 rings into a product ring

  164. class Product12Semigroup[X, A, B, C, D, E, F, G, H, I, J, K, L] extends Semigroup[X]

    Permalink

    Combine 12 semigroups into a product semigroup

  165. class Product13Group[X, A, B, C, D, E, F, G, H, I, J, K, L, M] extends Product13Monoid[X, A, B, C, D, E, F, G, H, I, J, K, L, M] with Group[X]

    Permalink

    Combine 13 groups into a product group

  166. class Product13Monoid[X, A, B, C, D, E, F, G, H, I, J, K, L, M] extends Product13Semigroup[X, A, B, C, D, E, F, G, H, I, J, K, L, M] with Monoid[X]

    Permalink

    Combine 13 monoids into a product monoid

  167. class Product13Ring[X, A, B, C, D, E, F, G, H, I, J, K, L, M] extends Product13Group[X, A, B, C, D, E, F, G, H, I, J, K, L, M] with Ring[X]

    Permalink

    Combine 13 rings into a product ring

  168. class Product13Semigroup[X, A, B, C, D, E, F, G, H, I, J, K, L, M] extends Semigroup[X]

    Permalink

    Combine 13 semigroups into a product semigroup

  169. class Product14Group[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N] extends Product14Monoid[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N] with Group[X]

    Permalink

    Combine 14 groups into a product group

  170. class Product14Monoid[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N] extends Product14Semigroup[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N] with Monoid[X]

    Permalink

    Combine 14 monoids into a product monoid

  171. class Product14Ring[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N] extends Product14Group[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N] with Ring[X]

    Permalink

    Combine 14 rings into a product ring

  172. class Product14Semigroup[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N] extends Semigroup[X]

    Permalink

    Combine 14 semigroups into a product semigroup

  173. class Product15Group[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O] extends Product15Monoid[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O] with Group[X]

    Permalink

    Combine 15 groups into a product group

  174. class Product15Monoid[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O] extends Product15Semigroup[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O] with Monoid[X]

    Permalink

    Combine 15 monoids into a product monoid

  175. class Product15Ring[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O] extends Product15Group[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O] with Ring[X]

    Permalink

    Combine 15 rings into a product ring

  176. class Product15Semigroup[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O] extends Semigroup[X]

    Permalink

    Combine 15 semigroups into a product semigroup

  177. class Product16Group[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P] extends Product16Monoid[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P] with Group[X]

    Permalink

    Combine 16 groups into a product group

  178. class Product16Monoid[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P] extends Product16Semigroup[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P] with Monoid[X]

    Permalink

    Combine 16 monoids into a product monoid

  179. class Product16Ring[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P] extends Product16Group[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P] with Ring[X]

    Permalink

    Combine 16 rings into a product ring

  180. class Product16Semigroup[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P] extends Semigroup[X]

    Permalink

    Combine 16 semigroups into a product semigroup

  181. class Product17Group[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q] extends Product17Monoid[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q] with Group[X]

    Permalink

    Combine 17 groups into a product group

  182. class Product17Monoid[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q] extends Product17Semigroup[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q] with Monoid[X]

    Permalink

    Combine 17 monoids into a product monoid

  183. class Product17Ring[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q] extends Product17Group[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q] with Ring[X]

    Permalink

    Combine 17 rings into a product ring

  184. class Product17Semigroup[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q] extends Semigroup[X]

    Permalink

    Combine 17 semigroups into a product semigroup

  185. class Product18Group[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R] extends Product18Monoid[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R] with Group[X]

    Permalink

    Combine 18 groups into a product group

  186. class Product18Monoid[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R] extends Product18Semigroup[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R] with Monoid[X]

    Permalink

    Combine 18 monoids into a product monoid

  187. class Product18Ring[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R] extends Product18Group[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R] with Ring[X]

    Permalink

    Combine 18 rings into a product ring

  188. class Product18Semigroup[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R] extends Semigroup[X]

    Permalink

    Combine 18 semigroups into a product semigroup

  189. class Product19Group[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S] extends Product19Monoid[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S] with Group[X]

    Permalink

    Combine 19 groups into a product group

  190. class Product19Monoid[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S] extends Product19Semigroup[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S] with Monoid[X]

    Permalink

    Combine 19 monoids into a product monoid

  191. class Product19Ring[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S] extends Product19Group[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S] with Ring[X]

    Permalink

    Combine 19 rings into a product ring

  192. class Product19Semigroup[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S] extends Semigroup[X]

    Permalink

    Combine 19 semigroups into a product semigroup

  193. class Product20Group[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T] extends Product20Monoid[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T] with Group[X]

    Permalink

    Combine 20 groups into a product group

  194. class Product20Monoid[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T] extends Product20Semigroup[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T] with Monoid[X]

    Permalink

    Combine 20 monoids into a product monoid

  195. class Product20Ring[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T] extends Product20Group[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T] with Ring[X]

    Permalink

    Combine 20 rings into a product ring

  196. class Product20Semigroup[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T] extends Semigroup[X]

    Permalink

    Combine 20 semigroups into a product semigroup

  197. class Product21Group[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U] extends Product21Monoid[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U] with Group[X]

    Permalink

    Combine 21 groups into a product group

  198. class Product21Monoid[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U] extends Product21Semigroup[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U] with Monoid[X]

    Permalink

    Combine 21 monoids into a product monoid

  199. class Product21Ring[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U] extends Product21Group[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U] with Ring[X]

    Permalink

    Combine 21 rings into a product ring

  200. class Product21Semigroup[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U] extends Semigroup[X]

    Permalink

    Combine 21 semigroups into a product semigroup

  201. class Product22Group[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U, V] extends Product22Monoid[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U, V] with Group[X]

    Permalink

    Combine 22 groups into a product group

  202. class Product22Monoid[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U, V] extends Product22Semigroup[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U, V] with Monoid[X]

    Permalink

    Combine 22 monoids into a product monoid

  203. class Product22Ring[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U, V] extends Product22Group[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U, V] with Ring[X]

    Permalink

    Combine 22 rings into a product ring

  204. class Product22Semigroup[X, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U, V] extends Semigroup[X]

    Permalink

    Combine 22 semigroups into a product semigroup

  205. class Product2Group[X, A, B] extends Product2Monoid[X, A, B] with Group[X]

    Permalink

    Combine 2 groups into a product group

  206. class Product2Monoid[X, A, B] extends Product2Semigroup[X, A, B] with Monoid[X]

    Permalink

    Combine 2 monoids into a product monoid

  207. class Product2Ring[X, A, B] extends Product2Group[X, A, B] with Ring[X]

    Permalink

    Combine 2 rings into a product ring

  208. class Product2Semigroup[X, A, B] extends Semigroup[X]

    Permalink

    Combine 2 semigroups into a product semigroup

  209. class Product3Group[X, A, B, C] extends Product3Monoid[X, A, B, C] with Group[X]

    Permalink

    Combine 3 groups into a product group

  210. class Product3Monoid[X, A, B, C] extends Product3Semigroup[X, A, B, C] with Monoid[X]

    Permalink

    Combine 3 monoids into a product monoid

  211. class Product3Ring[X, A, B, C] extends Product3Group[X, A, B, C] with Ring[X]

    Permalink

    Combine 3 rings into a product ring

  212. class Product3Semigroup[X, A, B, C] extends Semigroup[X]

    Permalink

    Combine 3 semigroups into a product semigroup

  213. class Product4Group[X, A, B, C, D] extends Product4Monoid[X, A, B, C, D] with Group[X]

    Permalink

    Combine 4 groups into a product group

  214. class Product4Monoid[X, A, B, C, D] extends Product4Semigroup[X, A, B, C, D] with Monoid[X]

    Permalink

    Combine 4 monoids into a product monoid

  215. class Product4Ring[X, A, B, C, D] extends Product4Group[X, A, B, C, D] with Ring[X]

    Permalink

    Combine 4 rings into a product ring

  216. class Product4Semigroup[X, A, B, C, D] extends Semigroup[X]

    Permalink

    Combine 4 semigroups into a product semigroup

  217. class Product5Group[X, A, B, C, D, E] extends Product5Monoid[X, A, B, C, D, E] with Group[X]

    Permalink

    Combine 5 groups into a product group

  218. class Product5Monoid[X, A, B, C, D, E] extends Product5Semigroup[X, A, B, C, D, E] with Monoid[X]

    Permalink

    Combine 5 monoids into a product monoid

  219. class Product5Ring[X, A, B, C, D, E] extends Product5Group[X, A, B, C, D, E] with Ring[X]

    Permalink

    Combine 5 rings into a product ring

  220. class Product5Semigroup[X, A, B, C, D, E] extends Semigroup[X]

    Permalink

    Combine 5 semigroups into a product semigroup

  221. class Product6Group[X, A, B, C, D, E, F] extends Product6Monoid[X, A, B, C, D, E, F] with Group[X]

    Permalink

    Combine 6 groups into a product group

  222. class Product6Monoid[X, A, B, C, D, E, F] extends Product6Semigroup[X, A, B, C, D, E, F] with Monoid[X]

    Permalink

    Combine 6 monoids into a product monoid

  223. class Product6Ring[X, A, B, C, D, E, F] extends Product6Group[X, A, B, C, D, E, F] with Ring[X]

    Permalink

    Combine 6 rings into a product ring

  224. class Product6Semigroup[X, A, B, C, D, E, F] extends Semigroup[X]

    Permalink

    Combine 6 semigroups into a product semigroup

  225. class Product7Group[X, A, B, C, D, E, F, G] extends Product7Monoid[X, A, B, C, D, E, F, G] with Group[X]

    Permalink

    Combine 7 groups into a product group

  226. class Product7Monoid[X, A, B, C, D, E, F, G] extends Product7Semigroup[X, A, B, C, D, E, F, G] with Monoid[X]

    Permalink

    Combine 7 monoids into a product monoid

  227. class Product7Ring[X, A, B, C, D, E, F, G] extends Product7Group[X, A, B, C, D, E, F, G] with Ring[X]

    Permalink

    Combine 7 rings into a product ring

  228. class Product7Semigroup[X, A, B, C, D, E, F, G] extends Semigroup[X]

    Permalink

    Combine 7 semigroups into a product semigroup

  229. class Product8Group[X, A, B, C, D, E, F, G, H] extends Product8Monoid[X, A, B, C, D, E, F, G, H] with Group[X]

    Permalink

    Combine 8 groups into a product group

  230. class Product8Monoid[X, A, B, C, D, E, F, G, H] extends Product8Semigroup[X, A, B, C, D, E, F, G, H] with Monoid[X]

    Permalink

    Combine 8 monoids into a product monoid

  231. class Product8Ring[X, A, B, C, D, E, F, G, H] extends Product8Group[X, A, B, C, D, E, F, G, H] with Ring[X]

    Permalink

    Combine 8 rings into a product ring

  232. class Product8Semigroup[X, A, B, C, D, E, F, G, H] extends Semigroup[X]

    Permalink

    Combine 8 semigroups into a product semigroup

  233. class Product9Group[X, A, B, C, D, E, F, G, H, I] extends Product9Monoid[X, A, B, C, D, E, F, G, H, I] with Group[X]

    Permalink

    Combine 9 groups into a product group

  234. class Product9Monoid[X, A, B, C, D, E, F, G, H, I] extends Product9Semigroup[X, A, B, C, D, E, F, G, H, I] with Monoid[X]

    Permalink

    Combine 9 monoids into a product monoid

  235. class Product9Ring[X, A, B, C, D, E, F, G, H, I] extends Product9Group[X, A, B, C, D, E, F, G, H, I] with Ring[X]

    Permalink

    Combine 9 rings into a product ring

  236. class Product9Semigroup[X, A, B, C, D, E, F, G, H, I] extends Semigroup[X]

    Permalink

    Combine 9 semigroups into a product semigroup

  237. trait ProductGroups extends AnyRef

    Permalink
  238. trait ProductMonoids extends AnyRef

    Permalink
  239. trait ProductRings extends AnyRef

    Permalink
  240. trait ProductSemigroups extends AnyRef

    Permalink
  241. class PureOp[A] extends AnyRef

    Permalink
  242. case class QTree[A](offset: Long, level: Int, count: Long, sum: A, lowerChild: Option[QTree[A]], upperChild: Option[QTree[A]]) extends Product with Serializable

    Permalink
  243. case class QTreeAggregator[T](percentile: Double, k: Int = QTreeAggregator.DefaultK)(implicit num: Numeric[T]) extends Aggregator[T, QTree[Unit], Intersection[InclusiveLower, InclusiveUpper, Double]] with QTreeAggregatorLike[T] with Product with Serializable

    Permalink

    QTree aggregator is an aggregator that can be used to find the approximate percentile bounds.

    QTree aggregator is an aggregator that can be used to find the approximate percentile bounds. The items that are iterated over to produce this approximation cannot be negative. Returns an Intersection which represents the bounded approximation.

  244. trait QTreeAggregatorLike[T] extends AnyRef

    Permalink
  245. case class QTreeAggregatorLowerBound[T](percentile: Double, k: Int = QTreeAggregator.DefaultK)(implicit num: Numeric[T]) extends Aggregator[T, QTree[Unit], Double] with QTreeAggregatorLike[T] with Product with Serializable

    Permalink

    QTreeAggregatorLowerBound is an aggregator that is used to find an appoximate percentile.

    QTreeAggregatorLowerBound is an aggregator that is used to find an appoximate percentile. This is similar to a QTreeAggregator, but is a convenience because instead of returning an Intersection, it instead returns the lower bound of the percentile. Like a QTreeAggregator, the items that are iterated over to produce this approximation cannot be negative.

  246. class QTreeSemigroup[A] extends Semigroup[QTree[A]]

    Permalink
  247. sealed trait ResetState[+A] extends AnyRef

    Permalink

    Used to represent cases where we need to periodically reset a + b = a + b |a + b = |(a + b) a + |b = |b |a + |b = |b

  248. class ResetStateMonoid[A] extends Monoid[ResetState[A]]

    Permalink
  249. case class ResetValue[+A](get: A) extends ResetState[A] with Product with Serializable

    Permalink
  250. class RichCBitSet extends AnyRef

    Permalink
  251. class RichTraversable[T] extends AnyRef

    Permalink
  252. sealed abstract class RightFolded[+In, +Out] extends AnyRef

    Permalink
  253. sealed abstract class RightFolded2[+In, +Out, +Acc] extends AnyRef

    Permalink
  254. class RightFolded2Monoid[In, Out, Acc] extends Monoid[RightFolded2[In, Out, Acc]]

    Permalink
  255. case class RightFoldedToFold[+In](in: List[In]) extends RightFolded[In, Nothing] with Product with Serializable

    Permalink
  256. case class RightFoldedToFold2[+In](in: List[In]) extends RightFolded2[In, Nothing, Nothing] with Product with Serializable

    Permalink
  257. case class RightFoldedValue[+Out](v: Out) extends RightFolded[Nothing, Out] with Product with Serializable

    Permalink
  258. case class RightFoldedValue2[+In, +Out, +Acc](v: Out, acc: Acc, rvals: List[In]) extends RightFolded2[In, Out, Acc] with Product with Serializable

    Permalink
  259. trait Ring[T] extends Group[T]

    Permalink

    Ring: Group + multiplication (see: http://en.wikipedia.org/wiki/Ring_%28mathematics%29) and the three elements it defines:

    Ring: Group + multiplication (see: http://en.wikipedia.org/wiki/Ring_%28mathematics%29) and the three elements it defines:

    • additive identity aka zero
    • addition
    • multiplication

    Note, if you have distributive property, additive inverses, and multiplicative identity you can prove you have a commutative group under the ring:

    1. (a + 1)*(b + 1) = a(b + 1) + (b + 1) 2. = ab + a + b + 1 3. or: 4. 5. = (a + 1)b + (a + 1) 6. = ab + b + a + 1 7. 8. So: ab + a + b + 1 == ab + b + a + 1 9. using the fact that -(ab) and -1 exist, we get: 10. a + b == b + a
    Annotations
    @implicitNotFound( ... )
  260. trait RingAggregator[-A, B, +C] extends MonoidAggregator[A, B, C]

    Permalink
  261. sealed abstract class SGD[+Pos] extends AnyRef

    Permalink
  262. class SGDMonoid[Pos] extends Monoid[SGD[Pos]]

    Permalink

    Basically a specific implementation of the RightFoldedMonoid gradient is the gradient of the function to be minimized To use this, you need to insert an initial weight SGDWeights before you start adding SGDPos objects.

    Basically a specific implementation of the RightFoldedMonoid gradient is the gradient of the function to be minimized To use this, you need to insert an initial weight SGDWeights before you start adding SGDPos objects. Otherwise you will just be doing list concatenation.

  263. case class SGDPos[+Pos](pos: List[Pos]) extends SGD[Pos] with Product with Serializable

    Permalink
  264. case class SGDWeights(count: Long, weights: IndexedSeq[Double]) extends SGD[Nothing] with Product with Serializable

    Permalink
  265. case class SSMany[T](capacity: Int, counters: Map[T, (Long, Long)], buckets: SortedMap[Long, Set[T]]) extends SpaceSaver[T] with Product with Serializable

    Permalink
  266. case class SSOne[T](capacity: Int, item: T) extends SpaceSaver[T] with Product with Serializable

    Permalink
  267. class ScMapGroup[K, V] extends ScMapMonoid[K, V] with Group[Map[K, V]]

    Permalink
  268. class ScMapMonoid[K, V] extends GenericMapMonoid[K, V, Map[K, V]]

    Permalink
  269. class ScMapRing[K, V] extends ScMapGroup[K, V] with GenericMapRing[K, V, Map[K, V]]

    Permalink
  270. trait Semigroup[T] extends Serializable

    Permalink

    Semigroup: This is a class with a plus method that is associative: a+(b+c) = (a+b)+c

    Semigroup: This is a class with a plus method that is associative: a+(b+c) = (a+b)+c

    Annotations
    @implicitNotFound( ... )
  271. class SemigroupCombinator[A, B] extends Semigroup[(A, B)]

    Permalink

    This is a combinator on semigroups, after you do the plus, you transform B with a fold function This will not be valid for all fold functions.

    This is a combinator on semigroups, after you do the plus, you transform B with a fold function This will not be valid for all fold functions. You need to prove that it is still associative.

    Clearly only values of (a,b) are valid if fold(a,b) == b, so keep that in mind.

    I have not yet found a sufficient condition on (A,B) => B that makes it correct Clearly a (trivial) constant function {(l,r) => r} works. Also, if B is List[T], and (l:A,r:List[T]) = r.sortBy(fn(l)) this works as well (due to the associativity on A, and the fact that the list never loses data).

    For approximate lists (like top-K applications) this might work (or be close enough to associative that for approximation algorithms it is fine), and in fact, that is the main motivation of this code: Produce some ordering in A, and use it to do sorted-topK on the list in B.

    Seems like an open topic here.... you are obliged to think on your own about this.

  272. class SentinelCache[K, V] extends AnyRef

    Permalink

    This is a summing cache whose goal is to grow until we run out of memory, at which point it clears itself and stops growing.

    This is a summing cache whose goal is to grow until we run out of memory, at which point it clears itself and stops growing. Note that we can lose the values in this cache at any point; we don't put anything here we care about.

  273. class SeqMonoid[T] extends Monoid[Seq[T]]

    Permalink
  274. class SetMonoid[T] extends Monoid[Set[T]]

    Permalink

    Set union monoid.

    Set union monoid. plus means union, zero is empty set

  275. case class SetSizeAggregator[A](hllBits: Int, maxSetSize: Int = 10)(implicit toBytes: (A) ⇒ Array[Byte]) extends SetSizeAggregatorBase[A] with Product with Serializable

    Permalink
  276. abstract class SetSizeAggregatorBase[A] extends EventuallyMonoidAggregator[A, HLL, Set[A], Long]

    Permalink

    convert is not not implemented here

  277. case class SetSizeHashAggregator[A](hllBits: Int, maxSetSize: Int = 10)(implicit hash: Hash128[A]) extends SetSizeAggregatorBase[A] with Product with Serializable

    Permalink

    Use a Hash128 when converting to HLL, rather than an implicit conversion to Array[Byte] Unifying with SetSizeAggregator would be nice, but since they only differ in an implicit parameter, scala seems to be giving me errors.

  278. case class SetValue[+A](get: A) extends ResetState[A] with Product with Serializable

    Permalink
  279. case class SketchMap[K, V](valuesTable: AdaptiveMatrix[V], heavyHitterKeys: List[K], totalValue: V) extends Serializable with Product with Serializable

    Permalink
  280. case class SketchMapAggregator[K, V](params: SketchMapParams[K], skmMonoid: SketchMapMonoid[K, V])(implicit valueOrdering: Ordering[V], valueMonoid: Monoid[V]) extends MonoidAggregator[(K, V), SketchMap[K, V], SketchMap[K, V]] with Product with Serializable

    Permalink

    An Aggregator for the SketchMap.

    An Aggregator for the SketchMap. Can be created using SketchMap.aggregator

  281. case class SketchMapHash[K](hasher: CMSHash[Long], seed: Int)(implicit serialization: (K) ⇒ Array[Byte]) extends Product with Serializable

    Permalink

    Hashes an arbitrary key type to one that the Sketch Map can use.

  282. class SketchMapMonoid[K, V] extends Monoid[SketchMap[K, V]]

    Permalink

    Responsible for creating instances of SketchMap.

  283. case class SketchMapParams[K](seed: Int, width: Int, depth: Int, heavyHittersCount: Int)(implicit serialization: (K) ⇒ Array[Byte]) extends Product with Serializable

    Permalink

    Convenience class for holding constant parameters of a Sketch Map.

  284. sealed abstract class SpaceSaver[T] extends AnyRef

    Permalink

    Data structure used in the Space-Saving Algorithm to find the approximate most frequent and top-k elements.

    Data structure used in the Space-Saving Algorithm to find the approximate most frequent and top-k elements. The algorithm is described in "Efficient Computation of Frequent and Top-k Elements in Data Streams". See here: www.cs.ucsb.edu/research/tech_reports/reports/2005-23.pdf In the paper the data structure is called StreamSummary but we chose to call it SpaceSaver instead. Note that the adaptation to hadoop and parallelization were not described in the article and have not been proven to be mathematically correct or preserve the guarantees or benefits of the algorithm.

  285. class SpaceSaverSemigroup[T] extends Semigroup[SpaceSaver[T]]

    Permalink
  286. case class SparseHLL(bits: Int, maxRhow: Map[Int, Max[Byte]]) extends HLL with Product with Serializable

    Permalink
  287. case class SparseVector[V](map: Map[Int, V], sparseValue: V, size: Int) extends AdaptiveVector[V] with Product with Serializable

    Permalink
  288. trait StatefulSummer[V] extends Buffered[V, V]

    Permalink

    A Stateful summer is something that is potentially more efficient (a buffer, a cache, etc...) that has the same result as a sum: Law 1: Semigroup.sumOption(items) == (Monoid.plus(items.map { stateful.put(_) }.filter { _.isDefined }, stateful.flush) && stateful.isFlushed) Law 2: isFlushed == flush.isEmpty

  289. trait Successible[T] extends AnyRef

    Permalink

    This is a typeclass to represent things which increase.

    This is a typeclass to represent things which increase. Note that it is important that a value after being incremented is always larger than it was before. Note that next returns Option because this class comes with the notion of the "greatest" key, which is None. Ints, for example, will cycle if next(java.lang.Integer.MAX_VALUE) is called, therefore we need a notion of what happens when we hit the bounds at which our ordering is violating. This is also useful for closed sets which have a fixed progression.

  290. class SumAll[V] extends StatefulSummer[V]

    Permalink

    Sum the entire iterator one item at a time.

    Sum the entire iterator one item at a time. Only emits on flush you should probably prefer BufferedSumAll

  291. class SummingCache[K, V] extends StatefulSummer[Map[K, V]]

    Permalink

    A Stateful Summer on Map[K,V] that keeps a cache of recent keys

  292. class SummingIterator[V] extends Serializable with Iterator[V]

    Permalink
  293. class SummingQueue[V] extends StatefulSummer[V]

    Permalink
  294. class SummingWithHitsCache[K, V] extends SummingCache[K, V]

    Permalink

    A SummingCache that also tracks the number of key hits

  295. class TimesOp[T] extends AnyRef

    Permalink
  296. sealed abstract class TopCMS[K] extends Serializable with CMSCounting[K, TopCMS] with CMSHeavyHitters[K]

    Permalink

    A Count-Min sketch data structure that allows for (a) counting and frequency estimation of elements in a data stream and (b) tracking the heavy hitters among these elements.

    A Count-Min sketch data structure that allows for (a) counting and frequency estimation of elements in a data stream and (b) tracking the heavy hitters among these elements.

    The logic of how heavy hitters are computed is pluggable, see HeavyHittersLogic.

    Tip: If you do not need to track heavy hitters, take a look at CMS, which is more efficient in this case.

    Usage

    This example demonstrates how to count Long elements with TopCMS, i.e. K=Long.

    Note that the actual counting is always performed with a Long, regardless of your choice of K. That is, the counting table behind the scenes is backed by Long values (at least in the current implementation), and thus the returned frequency estimates are always instances of Approximate[Long].

    K

    The type used to identify the elements to be counted.

    Example:
    1. // Implicits that enabling CMS-hashing of `Long` values.
      import com.twitter.algebird.CMSHasherImplicits._
      // Creates a monoid for a CMS that can count `Long` elements.
      val topPctCMSMonoid: TopPctCMSMonoid[Long] = {
        val eps = 0.001
        val delta = 1E-10
        val seed = 1
        val heavyHittersPct = 0.1
        TopPctCMS.monoid[Long](eps, delta, seed, heavyHittersPct)
      }
      // Creates a TopCMS instance that has counted the element `1L`.
      val topCMS: TopCMS[Long] = topPctCMSMonoid.create(1L)
      // Estimates the frequency of `1L`
      val estimate: Approximate[Long] = topCMS.frequency(1L)
      // What are the heavy hitters so far?
      val heavyHitters: Set[Long] = topCMS.heavyHitters
  297. case class TopCMSInstance[K](cms: CMS[K], hhs: HeavyHitters[K], params: TopCMSParams[K]) extends TopCMS[K] with Product with Serializable

    Permalink
  298. case class TopCMSItem[K](item: K, cms: CMS[K], params: TopCMSParams[K]) extends TopCMS[K] with Product with Serializable

    Permalink

    Used for holding a single element, to avoid repeatedly adding elements from sparse counts tables.

  299. case class TopCMSParams[K](logic: HeavyHittersLogic[K]) extends Product with Serializable

    Permalink
  300. case class TopCMSZero[K](cms: CMS[K], params: TopCMSParams[K]) extends TopCMS[K] with Product with Serializable

    Permalink

    Zero element.

    Zero element. Used for initialization.

  301. case class TopK[N](size: Int, items: List[N], max: Option[N]) extends Product with Serializable

    Permalink
  302. class TopKMonoid[T] extends Monoid[TopK[T]]

    Permalink

    A top-k monoid that is much faster than SortedListTake equivalent to: (left ++ right).sorted.take(k) but doesn't do a total sort If you can handle the mutability, mutable.PriorityQueueMonoid is even faster.

    A top-k monoid that is much faster than SortedListTake equivalent to: (left ++ right).sorted.take(k) but doesn't do a total sort If you can handle the mutability, mutable.PriorityQueueMonoid is even faster.

    NOTE!!!! This assumes the inputs are already sorted! resorting each time kills speed

  303. case class TopNCMSAggregator[K](cmsMonoid: TopNCMSMonoid[K]) extends MonoidAggregator[K, TopCMS[K], TopCMS[K]] with Product with Serializable

    Permalink

    An Aggregator for TopNCMS.

    An Aggregator for TopNCMS. Can be created using TopNCMS.aggregator.

  304. class TopNCMSMonoid[K] extends Monoid[TopCMS[K]]

    Permalink

    Monoid for top-N based TopCMS sketches.

    Monoid for top-N based TopCMS sketches. Use with care! (see warning below)

    Warning: Adding top-N CMS instances (++) is an unsafe operation

    Top-N computations are not associative. The effect is that a top-N CMS has an ordering bias (with regard to heavy hitters) when merging CMS instances (e.g. via ++). This means merging heavy hitters across CMS instances may lead to incorrect, biased results: the outcome is biased by the order in which CMS instances / heavy hitters are being merged, with the rule of thumb being that the earlier a set of heavy hitters is being merged, the more likely is the end result biased towards these heavy hitters.

    The warning above only applies when adding CMS instances (think: cms1 ++ cms2). In comparison, heavy hitters are correctly computed when:

    • a top-N CMS instance is created from a single data stream, i.e. Seq[K]
    • items are added/counted individually, i.e. cms + item or cms + (item, count).

    See the discussion in Algebird issue 353 for further details.

    Alternatives

    The following, alternative data structures may be better picks than a top-N based CMS given the warning above:

    • TopPctCMS: Has safe merge semantics for its instances including heavy hitters.
    • SpaceSaver: Has the same ordering bias than a top-N CMS, but at least it provides bounds on the bias.

    Usage

    The type K is the type of items you want to count. You must provide an implicit CMSHasher[K] for K, and Algebird ships with several such implicits for commonly used types such as Long and BigInt:

    import com.twitter.algebird.CMSHasherImplicits._

    If your type K is not supported out of the box, you have two options: 1) You provide a "translation" function to convert items of your (unsupported) type K to a supported type such as Double, and then use the contramap function of CMSHasher to create the required CMSHasher[K] for your type (see the documentation of CMSHasher for an example); 2) You implement a CMSHasher[K] from scratch, using the existing CMSHasher implementations as a starting point.

    Note: Because Arrays in Scala/Java not have sane equals and hashCode implementations, you cannot safely use types such as Array[Byte]. Extra work is required for Arrays. For example, you may opt to convert Array[T] to a Seq[T] via toSeq, or you can provide appropriate wrapper classes. Algebird provides one such wrapper class, Bytes, to safely wrap an Array[Byte] for use with CMS.

    K

    The type used to identify the elements to be counted. For example, if you want to count the occurrence of user names, you could map each username to a unique numeric ID expressed as a Long, and then count the occurrences of those Longs with a CMS of type K=Long. Note that this mapping between the elements of your problem domain and their identifiers used for counting via CMS should be bijective. We require a CMSHasher context bound for K, see CMSHasherImplicits for available implicits that can be imported. Which type K should you pick in practice? For domains that have less than 2^64 unique elements, you'd typically use Long. For larger domains you can try BigInt, for example.

  305. case class TopNLogic[K](heavyHittersN: Int) extends HeavyHittersLogic[K] with Product with Serializable

    Permalink

    Tracks the top N heavy hitters, where N is defined by heavyHittersN.

    Tracks the top N heavy hitters, where N is defined by heavyHittersN.

    Warning: top-N computations are not associative. The effect is that a top-N CMS has an ordering bias (with regard to heavy hitters) when merging instances. This means merging heavy hitters across CMS instances may lead to incorrect, biased results: the outcome is biased by the order in which CMS instances / heavy hitters are being merged, with the rule of thumb being that the earlier a set of heavy hitters is being merged, the more likely is the end result biased towards these heavy hitters.

    See also

    Discussion in Algebird issue 353

  306. case class TopPctCMSAggregator[K](cmsMonoid: TopPctCMSMonoid[K]) extends MonoidAggregator[K, TopCMS[K], TopCMS[K]] with Product with Serializable

    Permalink

    An Aggregator for TopPctCMS.

    An Aggregator for TopPctCMS. Can be created using TopPctCMS.aggregator.

  307. class TopPctCMSMonoid[K] extends Monoid[TopCMS[K]]

    Permalink

    Monoid for Top-% based TopCMS sketches.

    Monoid for Top-% based TopCMS sketches.

    Usage

    The type K is the type of items you want to count. You must provide an implicit CMSHasher[K] for K, and Algebird ships with several such implicits for commonly used types such as Long and BigInt:

    import com.twitter.algebird.CMSHasherImplicits._

    If your type K is not supported out of the box, you have two options: 1) You provide a "translation" function to convert items of your (unsupported) type K to a supported type such as Double, and then use the contramap function of CMSHasher to create the required CMSHasher[K] for your type (see the documentation of CMSHasher for an example); 2) You implement a CMSHasher[K] from scratch, using the existing CMSHasher implementations as a starting point.

    Note: Because Arrays in Scala/Java not have sane equals and hashCode implementations, you cannot safely use types such as Array[Byte]. Extra work is required for Arrays. For example, you may opt to convert Array[T] to a Seq[T] via toSeq, or you can provide appropriate wrapper classes. Algebird provides one such wrapper class, Bytes, to safely wrap an Array[Byte] for use with CMS.

    K

    The type used to identify the elements to be counted. For example, if you want to count the occurrence of user names, you could map each username to a unique numeric ID expressed as a Long, and then count the occurrences of those Longs with a CMS of type K=Long. Note that this mapping between the elements of your problem domain and their identifiers used for counting via CMS should be bijective. We require a CMSHasher context bound for K, see CMSHasherImplicits for available implicits that can be imported. Which type K should you pick in practice? For domains that have less than 2^64 unique elements, you'd typically use Long. For larger domains you can try BigInt, for example.

  308. case class TopPctLogic[K](heavyHittersPct: Double) extends HeavyHittersLogic[K] with Product with Serializable

    Permalink

    Finds all heavy hitters, i.e., elements in the stream that appear at least (heavyHittersPct * totalCount) times.

    Finds all heavy hitters, i.e., elements in the stream that appear at least (heavyHittersPct * totalCount) times.

    Every item that appears at least (heavyHittersPct * totalCount) times is output, and with probability p >= 1 - delta, no item whose count is less than (heavyHittersPct - eps) * totalCount is output.

    This also means that this parameter is an upper bound on the number of heavy hitters that will be tracked: the set of heavy hitters contains at most 1 / heavyHittersPct elements. For example, if heavyHittersPct=0.01 (or 0.25), then at most 1 / 0.01 = 100 items (or 1 / 0.25 = 4 items) will be tracked/returned as heavy hitters. This parameter can thus control the memory footprint required for tracking heavy hitters.

  309. class Tuple10Group[A, B, C, D, E, F, G, H, I, J] extends Group[(A, B, C, D, E, F, G, H, I, J)]

    Permalink

    Combine 10 groups into a product group

  310. class Tuple10Monoid[A, B, C, D, E, F, G, H, I, J] extends Monoid[(A, B, C, D, E, F, G, H, I, J)]

    Permalink

    Combine 10 monoids into a product monoid

  311. class Tuple10Ring[A, B, C, D, E, F, G, H, I, J] extends Ring[(A, B, C, D, E, F, G, H, I, J)]

    Permalink

    Combine 10 rings into a product ring

  312. class Tuple10Semigroup[A, B, C, D, E, F, G, H, I, J] extends Semigroup[(A, B, C, D, E, F, G, H, I, J)]

    Permalink

    Combine 10 semigroups into a product semigroup

  313. class Tuple11Group[A, B, C, D, E, F, G, H, I, J, K] extends Group[(A, B, C, D, E, F, G, H, I, J, K)]

    Permalink

    Combine 11 groups into a product group

  314. class Tuple11Monoid[A, B, C, D, E, F, G, H, I, J, K] extends Monoid[(A, B, C, D, E, F, G, H, I, J, K)]

    Permalink

    Combine 11 monoids into a product monoid

  315. class Tuple11Ring[A, B, C, D, E, F, G, H, I, J, K] extends Ring[(A, B, C, D, E, F, G, H, I, J, K)]

    Permalink

    Combine 11 rings into a product ring

  316. class Tuple11Semigroup[A, B, C, D, E, F, G, H, I, J, K] extends Semigroup[(A, B, C, D, E, F, G, H, I, J, K)]

    Permalink

    Combine 11 semigroups into a product semigroup

  317. class Tuple12Group[A, B, C, D, E, F, G, H, I, J, K, L] extends Group[(A, B, C, D, E, F, G, H, I, J, K, L)]

    Permalink

    Combine 12 groups into a product group

  318. class Tuple12Monoid[A, B, C, D, E, F, G, H, I, J, K, L] extends Monoid[(A, B, C, D, E, F, G, H, I, J, K, L)]

    Permalink

    Combine 12 monoids into a product monoid

  319. class Tuple12Ring[A, B, C, D, E, F, G, H, I, J, K, L] extends Ring[(A, B, C, D, E, F, G, H, I, J, K, L)]

    Permalink

    Combine 12 rings into a product ring

  320. class Tuple12Semigroup[A, B, C, D, E, F, G, H, I, J, K, L] extends Semigroup[(A, B, C, D, E, F, G, H, I, J, K, L)]

    Permalink

    Combine 12 semigroups into a product semigroup

  321. class Tuple13Group[A, B, C, D, E, F, G, H, I, J, K, L, M] extends Group[(A, B, C, D, E, F, G, H, I, J, K, L, M)]

    Permalink

    Combine 13 groups into a product group

  322. class Tuple13Monoid[A, B, C, D, E, F, G, H, I, J, K, L, M] extends Monoid[(A, B, C, D, E, F, G, H, I, J, K, L, M)]

    Permalink

    Combine 13 monoids into a product monoid

  323. class Tuple13Ring[A, B, C, D, E, F, G, H, I, J, K, L, M] extends Ring[(A, B, C, D, E, F, G, H, I, J, K, L, M)]

    Permalink

    Combine 13 rings into a product ring

  324. class Tuple13Semigroup[A, B, C, D, E, F, G, H, I, J, K, L, M] extends Semigroup[(A, B, C, D, E, F, G, H, I, J, K, L, M)]

    Permalink

    Combine 13 semigroups into a product semigroup

  325. class Tuple14Group[A, B, C, D, E, F, G, H, I, J, K, L, M, N] extends Group[(A, B, C, D, E, F, G, H, I, J, K, L, M, N)]

    Permalink

    Combine 14 groups into a product group

  326. class Tuple14Monoid[A, B, C, D, E, F, G, H, I, J, K, L, M, N] extends Monoid[(A, B, C, D, E, F, G, H, I, J, K, L, M, N)]

    Permalink

    Combine 14 monoids into a product monoid

  327. class Tuple14Ring[A, B, C, D, E, F, G, H, I, J, K, L, M, N] extends Ring[(A, B, C, D, E, F, G, H, I, J, K, L, M, N)]

    Permalink

    Combine 14 rings into a product ring

  328. class Tuple14Semigroup[A, B, C, D, E, F, G, H, I, J, K, L, M, N] extends Semigroup[(A, B, C, D, E, F, G, H, I, J, K, L, M, N)]

    Permalink

    Combine 14 semigroups into a product semigroup

  329. class Tuple15Group[A, B, C, D, E, F, G, H, I, J, K, L, M, N, O] extends Group[(A, B, C, D, E, F, G, H, I, J, K, L, M, N, O)]

    Permalink

    Combine 15 groups into a product group

  330. class Tuple15Monoid[A, B, C, D, E, F, G, H, I, J, K, L, M, N, O] extends Monoid[(A, B, C, D, E, F, G, H, I, J, K, L, M, N, O)]

    Permalink

    Combine 15 monoids into a product monoid

  331. class Tuple15Ring[A, B, C, D, E, F, G, H, I, J, K, L, M, N, O] extends Ring[(A, B, C, D, E, F, G, H, I, J, K, L, M, N, O)]

    Permalink

    Combine 15 rings into a product ring

  332. class Tuple15Semigroup[A, B, C, D, E, F, G, H, I, J, K, L, M, N, O] extends Semigroup[(A, B, C, D, E, F, G, H, I, J, K, L, M, N, O)]

    Permalink

    Combine 15 semigroups into a product semigroup

  333. class Tuple16Group[A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P] extends Group[(A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P)]

    Permalink

    Combine 16 groups into a product group

  334. class Tuple16Monoid[A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P] extends Monoid[(A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P)]

    Permalink

    Combine 16 monoids into a product monoid

  335. class Tuple16Ring[A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P] extends Ring[(A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P)]

    Permalink

    Combine 16 rings into a product ring

  336. class Tuple16Semigroup[A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P] extends Semigroup[(A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P)]

    Permalink

    Combine 16 semigroups into a product semigroup

  337. class Tuple17Group[A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q] extends Group[(A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q)]

    Permalink

    Combine 17 groups into a product group

  338. class Tuple17Monoid[A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q] extends Monoid[(A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q)]

    Permalink

    Combine 17 monoids into a product monoid

  339. class Tuple17Ring[A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q] extends Ring[(A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q)]

    Permalink

    Combine 17 rings into a product ring

  340. class Tuple17Semigroup[A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q] extends Semigroup[(A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q)]

    Permalink

    Combine 17 semigroups into a product semigroup

  341. class Tuple18Group[A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R] extends Group[(A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R)]

    Permalink

    Combine 18 groups into a product group

  342. class Tuple18Monoid[A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R] extends Monoid[(A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R)]

    Permalink

    Combine 18 monoids into a product monoid

  343. class Tuple18Ring[A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R] extends Ring[(A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R)]

    Permalink

    Combine 18 rings into a product ring

  344. class Tuple18Semigroup[A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R] extends Semigroup[(A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R)]

    Permalink

    Combine 18 semigroups into a product semigroup

  345. class Tuple19Group[A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S] extends Group[(A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S)]

    Permalink

    Combine 19 groups into a product group

  346. class Tuple19Monoid[A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S] extends Monoid[(A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S)]

    Permalink

    Combine 19 monoids into a product monoid

  347. class Tuple19Ring[A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S] extends Ring[(A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S)]

    Permalink

    Combine 19 rings into a product ring

  348. class Tuple19Semigroup[A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S] extends Semigroup[(A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S)]

    Permalink

    Combine 19 semigroups into a product semigroup

  349. class Tuple20Group[A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T] extends Group[(A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T)]

    Permalink

    Combine 20 groups into a product group

  350. class Tuple20Monoid[A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T] extends Monoid[(A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T)]

    Permalink

    Combine 20 monoids into a product monoid

  351. class Tuple20Ring[A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T] extends Ring[(A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T)]

    Permalink

    Combine 20 rings into a product ring

  352. class Tuple20Semigroup[A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T] extends Semigroup[(A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T)]

    Permalink

    Combine 20 semigroups into a product semigroup

  353. class Tuple21Group[A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U] extends Group[(A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U)]

    Permalink

    Combine 21 groups into a product group

  354. class Tuple21Monoid[A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U] extends Monoid[(A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U)]

    Permalink

    Combine 21 monoids into a product monoid

  355. class Tuple21Ring[A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U] extends Ring[(A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U)]

    Permalink

    Combine 21 rings into a product ring

  356. class Tuple21Semigroup[A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U] extends Semigroup[(A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U)]

    Permalink

    Combine 21 semigroups into a product semigroup

  357. class Tuple22Group[A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U, V] extends Group[(A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U, V)]

    Permalink

    Combine 22 groups into a product group

  358. class Tuple22Monoid[A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U, V] extends Monoid[(A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U, V)]

    Permalink

    Combine 22 monoids into a product monoid

  359. class Tuple22Ring[A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U, V] extends Ring[(A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U, V)]

    Permalink

    Combine 22 rings into a product ring

  360. class Tuple22Semigroup[A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U, V] extends Semigroup[(A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U, V)]

    Permalink

    Combine 22 semigroups into a product semigroup

  361. class Tuple2Group[A, B] extends Group[(A, B)]

    Permalink

    Combine 2 groups into a product group

  362. class Tuple2Monoid[A, B] extends Monoid[(A, B)]

    Permalink

    Combine 2 monoids into a product monoid

  363. class Tuple2Ring[A, B] extends Ring[(A, B)]

    Permalink

    Combine 2 rings into a product ring

  364. class Tuple2Semigroup[A, B] extends Semigroup[(A, B)]

    Permalink

    Combine 2 semigroups into a product semigroup

  365. class Tuple3Group[A, B, C] extends Group[(A, B, C)]

    Permalink

    Combine 3 groups into a product group

  366. class Tuple3Monoid[A, B, C] extends Monoid[(A, B, C)]

    Permalink

    Combine 3 monoids into a product monoid

  367. class Tuple3Ring[A, B, C] extends Ring[(A, B, C)]

    Permalink

    Combine 3 rings into a product ring

  368. class Tuple3Semigroup[A, B, C] extends Semigroup[(A, B, C)]

    Permalink

    Combine 3 semigroups into a product semigroup

  369. class Tuple4Group[A, B, C, D] extends Group[(A, B, C, D)]

    Permalink

    Combine 4 groups into a product group

  370. class Tuple4Monoid[A, B, C, D] extends Monoid[(A, B, C, D)]

    Permalink

    Combine 4 monoids into a product monoid

  371. class Tuple4Ring[A, B, C, D] extends Ring[(A, B, C, D)]

    Permalink

    Combine 4 rings into a product ring

  372. class Tuple4Semigroup[A, B, C, D] extends Semigroup[(A, B, C, D)]

    Permalink

    Combine 4 semigroups into a product semigroup

  373. class Tuple5Group[A, B, C, D, E] extends Group[(A, B, C, D, E)]

    Permalink

    Combine 5 groups into a product group

  374. class Tuple5Monoid[A, B, C, D, E] extends Monoid[(A, B, C, D, E)]

    Permalink

    Combine 5 monoids into a product monoid

  375. class Tuple5Ring[A, B, C, D, E] extends Ring[(A, B, C, D, E)]

    Permalink

    Combine 5 rings into a product ring

  376. class Tuple5Semigroup[A, B, C, D, E] extends Semigroup[(A, B, C, D, E)]

    Permalink

    Combine 5 semigroups into a product semigroup

  377. class Tuple6Group[A, B, C, D, E, F] extends Group[(A, B, C, D, E, F)]

    Permalink

    Combine 6 groups into a product group

  378. class Tuple6Monoid[A, B, C, D, E, F] extends Monoid[(A, B, C, D, E, F)]

    Permalink

    Combine 6 monoids into a product monoid

  379. class Tuple6Ring[A, B, C, D, E, F] extends Ring[(A, B, C, D, E, F)]

    Permalink

    Combine 6 rings into a product ring

  380. class Tuple6Semigroup[A, B, C, D, E, F] extends Semigroup[(A, B, C, D, E, F)]

    Permalink

    Combine 6 semigroups into a product semigroup

  381. class Tuple7Group[A, B, C, D, E, F, G] extends Group[(A, B, C, D, E, F, G)]

    Permalink

    Combine 7 groups into a product group

  382. class Tuple7Monoid[A, B, C, D, E, F, G] extends Monoid[(A, B, C, D, E, F, G)]

    Permalink

    Combine 7 monoids into a product monoid

  383. class Tuple7Ring[A, B, C, D, E, F, G] extends Ring[(A, B, C, D, E, F, G)]

    Permalink

    Combine 7 rings into a product ring

  384. class Tuple7Semigroup[A, B, C, D, E, F, G] extends Semigroup[(A, B, C, D, E, F, G)]

    Permalink

    Combine 7 semigroups into a product semigroup

  385. class Tuple8Group[A, B, C, D, E, F, G, H] extends Group[(A, B, C, D, E, F, G, H)]

    Permalink

    Combine 8 groups into a product group

  386. class Tuple8Monoid[A, B, C, D, E, F, G, H] extends Monoid[(A, B, C, D, E, F, G, H)]

    Permalink

    Combine 8 monoids into a product monoid

  387. class Tuple8Ring[A, B, C, D, E, F, G, H] extends Ring[(A, B, C, D, E, F, G, H)]

    Permalink

    Combine 8 rings into a product ring

  388. class Tuple8Semigroup[A, B, C, D, E, F, G, H] extends Semigroup[(A, B, C, D, E, F, G, H)]

    Permalink

    Combine 8 semigroups into a product semigroup

  389. class Tuple9Group[A, B, C, D, E, F, G, H, I] extends Group[(A, B, C, D, E, F, G, H, I)]

    Permalink

    Combine 9 groups into a product group

  390. class Tuple9Monoid[A, B, C, D, E, F, G, H, I] extends Monoid[(A, B, C, D, E, F, G, H, I)]

    Permalink

    Combine 9 monoids into a product monoid

  391. class Tuple9Ring[A, B, C, D, E, F, G, H, I] extends Ring[(A, B, C, D, E, F, G, H, I)]

    Permalink

    Combine 9 rings into a product ring

  392. class Tuple9Semigroup[A, B, C, D, E, F, G, H, I] extends Semigroup[(A, B, C, D, E, F, G, H, I)]

    Permalink

    Combine 9 semigroups into a product semigroup

  393. case class Universe[T]() extends Interval[T] with Product with Serializable

    Permalink
  394. sealed trait Upper[T] extends Interval[T]

    Permalink
  395. trait VectorSpace[F, C[_]] extends Serializable

    Permalink
    Annotations
    @implicitNotFound( ... )

Value Members

  1. object AdaptiveVector

    Permalink

    Some functions to create or convert AdaptiveVectors

  2. object AdjoinedUnit extends Serializable

    Permalink
  3. object AffineFunction extends Serializable

    Permalink
  4. object Aggregator extends Serializable

    Permalink

    Aggregators compose well.

    Aggregators compose well.

    To create a parallel aggregator that operates on a single input in parallel, use: GeneratedTupleAggregator.from2((agg1, agg2))

  5. object AndVal extends Serializable

    Permalink
  6. object AndValMonoid extends Monoid[AndVal]

    Permalink

    Boolean AND monoid.

    Boolean AND monoid. plus means logical AND, zero is true.

  7. object Applicative

    Permalink

    Follows the type-class pattern for the Applicative trait

  8. object Approximate extends Serializable

    Permalink
  9. object ApproximateBoolean extends Serializable

    Permalink
  10. object AveragedGroup extends Group[AveragedValue]

    Permalink
  11. object AveragedValue extends Serializable

    Permalink
  12. object Averager extends MonoidAggregator[Double, AveragedValue, Double]

    Permalink
  13. object BFInstance extends Serializable

    Permalink
  14. object BigIntRing extends NumericRing[BigInt]

    Permalink
  15. object BloomFilter

    Permalink
  16. object BloomFilterAggregator extends Serializable

    Permalink
  17. object BooleanField extends Field[Boolean]

    Permalink
  18. object Bytes extends Serializable

    Permalink
  19. object CMS extends Serializable

    Permalink
  20. object CMSFunctions

    Permalink

    Helper functions to generate or to translate between various CMS parameters (cf.

    Helper functions to generate or to translate between various CMS parameters (cf. CMSParams).

  21. object CMSHasherImplicits

    Permalink

    Implicits that enable CMS-hashing for common data types such as Long and BigInt.

  22. object CMSInstance extends Serializable

    Permalink
  23. object DecayedValue extends Serializable

    Permalink
  24. object DecayedVector extends Serializable

    Permalink

    Represents a container class together with time.

    Represents a container class together with time. Its monoid consists of exponentially scaling the older value and summing with the newer one.

  25. object DoubleField extends Field[Double]

    Permalink
  26. object Field extends Serializable

    Permalink
  27. object First extends Serializable

    Permalink
  28. object FlatMapPreparer extends Serializable

    Permalink
  29. object FloatField extends Field[Float]

    Permalink
  30. object Fold extends Serializable

    Permalink

    Methods to create and run Folds.

    Methods to create and run Folds.

    The Folds defined here are immutable and serializable, which we expect by default. It is important that you as a user indicate mutability or non-serializability when defining new Folds. Additionally, it is recommended that "end" functions not mutate the accumulator in order to support scans (producing a stream of intermediate outputs by calling "end" at each step).

  31. object Functor

    Permalink

    Follows the type-class pattern for the Functor trait

  32. object GeneratedTupleAggregator extends GeneratedTupleAggregator

    Permalink
  33. object Group extends GeneratedGroupImplicits with ProductGroups with Serializable

    Permalink
  34. object Hash128 extends Serializable

    Permalink

    This gives default hashes using Murmur128 with a seed of 12345678 (for no good reason, but it should not be changed lest we break serialized HLLs)

  35. object HeavyHitters extends Serializable

    Permalink
  36. object HyperLogLog

    Permalink

    Implementation of the HyperLogLog approximate counting as a Monoid

  37. object HyperLogLogAggregator extends Serializable

    Permalink

    This object makes it easier to create Aggregator instances that use HLL

  38. object IntRing extends Ring[Int]

    Permalink
  39. object Interval extends Serializable

    Permalink
  40. object JBoolField extends Field[Boolean]

    Permalink
  41. object JDoubleField extends Field[Double]

    Permalink
  42. object JFloatField extends Field[Float]

    Permalink
  43. object JIntRing extends Ring[Integer]

    Permalink
  44. object JLongRing extends Ring[Long]

    Permalink
  45. object JShortRing extends Ring[Short]

    Permalink
  46. object Last extends Serializable

    Permalink
  47. object LongRing extends Ring[Long]

    Permalink
  48. object MapAggregator extends Serializable

    Permalink
  49. object MapAlgebra

    Permalink
  50. object MapPreparer extends Serializable

    Permalink
  51. object Max extends Serializable

    Permalink
  52. object Metric extends Serializable

    Permalink

    A Metric[V] m is a function (V, V) => Double that satisfies the following properties:

    A Metric[V] m is a function (V, V) => Double that satisfies the following properties:

    1. m(v1, v2) >= 0 2. m(v1, v2) == 0 iff v1 == v2 3. m(v1, v2) == m(v2, v1) 4. m(v1, v3) <= m(v1, v2) + m(v2, v3)

    If you implement this trait, make sure that you follow these rules.

  53. object Min extends Serializable

    Permalink
  54. object MinHasher extends Serializable

    Permalink
  55. object MinPlus extends Serializable

    Permalink
  56. object MinPlusZero extends MinPlus[Nothing] with Product with Serializable

    Permalink
  57. object Moments extends Serializable

    Permalink
  58. object MomentsAggregator extends MonoidAggregator[Double, Moments, Moments]

    Permalink
  59. object MomentsGroup extends Group[Moments]

    Permalink

    A monoid to perform moment calculations.

  60. object Monad

    Permalink

    Follows the type-class pattern for the Monad trait

  61. object Monoid extends GeneratedMonoidImplicits with ProductMonoids with Serializable

    Permalink
  62. object MultiAggregator

    Permalink
  63. object NullGroup extends ConstantGroup[Null]

    Permalink
  64. object Operators

    Permalink
  65. object OrVal extends Serializable

    Permalink
  66. object OrValMonoid extends Monoid[OrVal]

    Permalink

    Boolean OR monoid.

    Boolean OR monoid. plus means logical OR, zero is false.

  67. object Predecessible extends Serializable

    Permalink
  68. object Preparer extends Serializable

    Permalink
  69. object Priority extends FindPreferred

    Permalink
  70. object QTree extends Serializable

    Permalink

    A QTree provides an approximate Map[Double,A:Monoid] suitable for range queries, quantile queries, and combinations of these (for example, if you use a numeric A, you can derive the inter-quartile mean).

    A QTree provides an approximate Map[Double,A:Monoid] suitable for range queries, quantile queries, and combinations of these (for example, if you use a numeric A, you can derive the inter-quartile mean).

    It is loosely related to the Q-Digest data structure from http://www.cs.virginia.edu/~son/cs851/papers/ucsb.sensys04.pdf, but using an immutable tree structure, and carrying a generalized sum (of type A) at each node instead of just a count.

    The basic idea is to keep a binary tree, where the root represents the entire range of the input keys, and each child node represents either the lower or upper half of its parent's range. Ranges are constrained to be dyadic intervals (https://en.wikipedia.org/wiki/Interval_(mathematics)#Dyadic_intervals) for ease of merging.

    To keep the size bounded, the total count carried by any sub-tree must be at least 1/(2^k) of the total count at the root. Any sub-trees that do not meet this criteria have their children pruned and become leaves. (It's important that they not be pruned away entirely, but that we keep a fringe of low-count leaves that can gain weight over time and ultimately split again when warranted).

    Quantile and range queries both give hard upper and lower bounds; the true result will be somewhere in the range given.

    Keys must be >= 0.

  71. object QTreeAggregator extends Serializable

    Permalink
  72. object ResetState

    Permalink
  73. object RichCBitSet

    Permalink
  74. object RightFolded

    Permalink

    This is an associative, but not commutative monoid Also, you must start on the right, with a value, and all subsequent RightFolded must be RightFoldedToFold objects or zero

    This is an associative, but not commutative monoid Also, you must start on the right, with a value, and all subsequent RightFolded must be RightFoldedToFold objects or zero

    If you add two Folded values together, you always get the one on the left, so this forms a kind of reset of the fold.

  75. object RightFolded2

    Permalink

    This monoid takes a list of values of type In or Out, and folds to the right all the Ins into Out values, leaving you with a list of Out values, then finally, maps those outs onto Acc, where there is a group, and adds all the Accs up.

    This monoid takes a list of values of type In or Out, and folds to the right all the Ins into Out values, leaving you with a list of Out values, then finally, maps those outs onto Acc, where there is a group, and adds all the Accs up. So, if you have a list: I I I O I O O I O I O the monoid is equivalent to the computation:

    map(fold(List(I,I,I),O)) + map(fold(List(I),O)) + map(fold(List(),O)) + map(fold(List(I),O)) + map(fold(List(I),O))

    This models a version of the map/reduce paradigm, where the fold happens on the mappers for each group on Ins, and then they are mapped to Accs, sent to a single reducer and all the Accs are added up.

  76. object RightFoldedZero extends RightFolded[Nothing, Nothing] with Product with Serializable

    Permalink
  77. object RightFoldedZero2 extends RightFolded2[Nothing, Nothing, Nothing] with Product with Serializable

    Permalink
  78. object Ring extends GeneratedRingImplicits with ProductRings with Serializable

    Permalink
  79. object SGD

    Permalink
  80. object SGDPos extends Serializable

    Permalink
  81. object SGDWeights extends Serializable

    Permalink
  82. object SGDZero extends SGD[Nothing] with Product with Serializable

    Permalink
  83. object SSMany extends Serializable

    Permalink
  84. object Semigroup extends GeneratedSemigroupImplicits with ProductSemigroups with Serializable

    Permalink
  85. object ShortRing extends Ring[Short]

    Permalink
  86. object SketchMap extends Serializable

    Permalink

    Data structure representing an approximation of Map[K, V], where V has an implicit ordering and monoid.

    Data structure representing an approximation of Map[K, V], where V has an implicit ordering and monoid. This is a more generic version of CountMinSketch.

    Values are stored in valuesTable, a 2D vector containing aggregated sums of values inserted to the Sketch Map.

    The data structure stores top non-zero values, called Heavy Hitters. The values are sorted by an implicit reverse ordering for the value, and the number of heavy hitters stored is based on the heavyHittersCount set in params.

    Use SketchMapMonoid to create instances of this class.

  87. object SketchMapParams extends Serializable

    Permalink
  88. object SpaceSaver

    Permalink
  89. object StringMonoid extends Monoid[String]

    Permalink
  90. object Successible

    Permalink
  91. object SummingCache extends Serializable

    Permalink
  92. object SummingIterator extends Serializable

    Permalink

    Creates an Iterator that emits partial sums of an input Iterator[V].

    Creates an Iterator that emits partial sums of an input Iterator[V]. Generally this is useful to change from processing individiual V's to possibly blocks of V @see SummingQueue or a cache of recent Keys in a V=Map[K,W] case: @see SummingCache

  93. object SummingQueue extends Serializable

    Permalink
  94. object SummingWithHitsCache extends Serializable

    Permalink
  95. object TopCMSInstance extends Serializable

    Permalink
  96. object TopKMonoid extends Serializable

    Permalink
  97. object TopNCMS

    Permalink
  98. object TopPctCMS

    Permalink
  99. object UnitGroup extends ConstantGroup[Unit]

    Permalink
  100. object VectorSpace extends Serializable

    Permalink

    This class represents a vector space.

    This class represents a vector space. For the required properties see:

    http://en.wikipedia.org/wiki/Vector_space#Definition

  101. package javaapi

    Permalink
  102. package legacy

    Permalink
  103. package macros

    Permalink
  104. package matrix

    Permalink
  105. package monad

    Permalink
  106. package mutable

    Permalink
  107. package statistics

    Permalink

Inherited from AnyRef

Inherited from Any

Ungrouped