BigSampler

Value Members

1. final def !=(arg0: Any): Boolean

   

   Definition Classes

   AnyRef → Any

2. final def ##(): Int

   

   Definition Classes

   AnyRef → Any

3. final def ==(arg0: Any): Boolean

   

   Definition Classes

   AnyRef → Any

4. final def asInstanceOf[T0]: T0

   

   Definition Classes

   Any

5. def clone(): AnyRef

   

   Attributes

   protected[java.lang]

   Definition Classes

   AnyRef

   Annotations

   @throws( ... )

6. val command: String

   

   Definition Classes

   BigSampler → Command

7. final def eq(arg0: AnyRef): Boolean

   

   Definition Classes

   AnyRef

8. def equals(arg0: Any): Boolean

   

   Definition Classes

   AnyRef → Any

9. def finalize(): Unit

   

   Attributes

   protected[java.lang]

   Definition Classes

   AnyRef

   Annotations

   @throws( classOf[java.lang.Throwable] )

10. final def getClass(): Class[_]

    

    Definition Classes

    AnyRef → Any

11. def hashCode(): Int

    

    Definition Classes

    AnyRef → Any

12. final def isInstanceOf[T0]: Boolean

    

    Definition Classes

    Any

13. final def ne(arg0: AnyRef): Boolean

    

    Definition Classes

    AnyRef

14. final def notify(): Unit

    

    Definition Classes

    AnyRef

15. final def notifyAll(): Unit

    

    Definition Classes

    AnyRef

16. def run(argv: Array[String]): Unit

    

17. def sample[T, U](s: SCollection[T], fraction: Double, fields: Seq[String], seed: Option[Int], distribution: Option[SampleDistribution], distributionFields: Seq[String], precision: Precision, hashFn: (T, String, Hasher) ⇒ Hasher, keyFn: (T) ⇒ U, maxKeySize: Int = 1e6.toInt, byteEncoding: ByteEncoding = RawEncoding)(implicit arg0: ClassTag[T], arg1: Coder[T], arg2: ClassTag[U], arg3: Coder[U]): SCollection[T]

    

    Sample wrapper function that manages sampling pipeline based on determinimism, precision, and data type.

    Sample wrapper function that manages sampling pipeline based on determinimism, precision, and data type. Can be used to build sampling for data types not supported out of the box.

    T

    Record Type

    U

    Key Type, usually we use Set[String]

    s

    The input SCollection to be sampled

    fraction

    The sample rate

    fields

    Fields to construct hash over for determinism

    seed

    Seed used to salt the deterministic hash

    distribution

    Desired output sample distribution

    distributionFields

    Fields to construct distribution over (strata = set of unique fields)

    precision

    Approximate or Exact precision

    hashFn

    Function to construct a hash given a record, field, and hasher

    keyFn

    Function to extract a value that's safe to serialize and key on, given a record

    maxKeySize

    Maximum allowed size per key (can be tweaked for very large data sets)

    byteEncoding

    Determines how bytes are encoded prior to hashing.

    returns

    SCollection containing Sample population

18. def singleInput(argv: Array[String]): ClosedTap[_]

    

19. final def synchronized[T0](arg0: ⇒ T0): T0

    

    Definition Classes

    AnyRef

20. def toString(): String

    

    Definition Classes

    AnyRef → Any

21. final def wait(): Unit

    

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

22. final def wait(arg0: Long, arg1: Int): Unit

    

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

23. final def wait(arg0: Long): Unit

    

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )