org.qcri.rheem.api

SampleDataQuantaBuilder

class SampleDataQuantaBuilder[T] extends BasicDataQuantaBuilder[SampleDataQuantaBuilder[T], T]

DataQuantaBuilder implementation for org.qcri.rheem.basic.operators.SampleOperators.

Linear Supertypes
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. SampleDataQuantaBuilder
  2. BasicDataQuantaBuilder
  3. DataQuantaBuilder
  4. Logging
  5. AnyRef
  6. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Instance Constructors

  1. new SampleDataQuantaBuilder(inputDataQuanta: DataQuantaBuilder[_, T], sampleSizeFunction: IntUnaryOperator)(implicit javaPlanBuilder: JavaPlanBuilder)

    inputDataQuanta

    DataQuantaBuilder for the input DataQuanta

    sampleSizeFunction

    the absolute size of the sample as a function of the current iteration number

Value Members

  1. final def !=(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  2. final def !=(arg0: Any): Boolean

    Definition Classes
    Any
  3. final def ##(): Int

    Definition Classes
    AnyRef → Any
  4. final def ==(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  5. final def ==(arg0: Any): Boolean

    Definition Classes
    Any
  6. def asEdges[T <: EdgeDataQuantaBuilder[T]]: EdgeDataQuantaBuilder[T]

    Enriches the set of operations to Edge-based ones.

    Enriches the set of operations to Edge-based ones. This instances must deal with data quanta of type Edge, though. Because of Java's type erasure, we need to leave it up to you whether this operation is applicable.

    returns

    a EdgeDataQuantaBuilder

    Definition Classes
    DataQuantaBuilder
  7. final def asInstanceOf[T0]: T0

    Definition Classes
    Any
  8. def asRecords[T <: RecordDataQuantaBuilder[T]]: RecordDataQuantaBuilder[T]

    Enriches the set of operations to Record-based ones.

    Enriches the set of operations to Record-based ones. This instances must deal with data quanta of type Record, though. Because of Java's type erasure, we need to leave it up to you whether this operation is applicable.

    returns

    a RecordDataQuantaBuilder

    Definition Classes
    DataQuantaBuilder
  9. def build: DataQuanta[T]

    Create the DataQuanta built by this instance.

    Create the DataQuanta built by this instance. Note the configuration being done in dataQuanta().

    returns

    the created and partially configured DataQuanta

    Attributes
    protected
    Definition Classes
    SampleDataQuantaBuilderBasicDataQuantaBuilder
  10. def cartesian[ThatOut](that: DataQuantaBuilder[_, ThatOut]): CartesianDataQuantaBuilder[T, ThatOut]

    Feed the built DataQuanta of this and the given instance into a org.qcri.rheem.basic.operators.CartesianOperator.

    Feed the built DataQuanta of this and the given instance into a org.qcri.rheem.basic.operators.CartesianOperator.

    returns

    a CartesianDataQuantaBuilder

    Definition Classes
    DataQuantaBuilder
  11. implicit def classTag: ClassTag[T]

    Provide a ClassTag for the constructed DataQuanta.

    Provide a ClassTag for the constructed DataQuanta.

    returns

    the ClassTag

    Attributes
    protected[org.qcri.rheem.api]
    Definition Classes
    BasicDataQuantaBuilderDataQuantaBuilder
  12. def clone(): AnyRef

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  13. def coGroup[ThatOut, Key](thisKeyUdf: SerializableFunction[T, Key], that: DataQuantaBuilder[_, ThatOut], thatKeyUdf: SerializableFunction[ThatOut, Key]): CoGroupDataQuantaBuilder[T, ThatOut, Key]

    Feed the built DataQuanta of this and the given instance into a org.qcri.rheem.basic.operators.CoGroupOperator.

    Feed the built DataQuanta of this and the given instance into a org.qcri.rheem.basic.operators.CoGroupOperator.

    thisKeyUdf

    the key extraction UDF for this instance

    that

    the other DataQuantaBuilder to join with

    thatKeyUdf

    the key extraction UDF for that instance

    returns

    a CoGroupDataQuantaBuilder

    Definition Classes
    DataQuantaBuilder
  14. def collect(): Collection[T]

    Feed the built DataQuanta into a LocalCallbackSink that collects all data quanta locally.

    Feed the built DataQuanta into a LocalCallbackSink that collects all data quanta locally. This triggers execution of the constructed RheemPlan.

    returns

    the collected data quanta

    Definition Classes
    DataQuantaBuilder
  15. def count: CountDataQuantaBuilder[T]

    Feed the built DataQuanta into a org.qcri.rheem.basic.operators.CountOperator.

    Feed the built DataQuanta into a org.qcri.rheem.basic.operators.CountOperator.

    returns

    a CountDataQuantaBuilder representing the org.qcri.rheem.basic.operators.CountOperator's output

    Definition Classes
    DataQuantaBuilder
  16. def customOperator[T](operator: Operator): CustomOperatorDataQuantaBuilder[T]

    Feed the built DataQuanta into a custom Operator with a single org.qcri.rheem.core.plan.rheemplan.InputSlot and a single OutputSlot.

    Feed the built DataQuanta into a custom Operator with a single org.qcri.rheem.core.plan.rheemplan.InputSlot and a single OutputSlot.

    T

    the type of the output DataQuanta

    operator

    the custom Operator

    returns

    a CustomOperatorDataQuantaBuilder

    Definition Classes
    DataQuantaBuilder
  17. def dataQuanta(): DataQuanta[T]

    Get or create the DataQuanta built by this instance.

    Get or create the DataQuanta built by this instance.

    returns

    the DataQuanta

    Attributes
    protected[org.qcri.rheem.api]
    Definition Classes
    BasicDataQuantaBuilderDataQuantaBuilder
  18. def distinct: DistinctDataQuantaBuilder[T]

    Feed the built DataQuanta into a org.qcri.rheem.basic.operators.DistinctOperator.

    Feed the built DataQuanta into a org.qcri.rheem.basic.operators.DistinctOperator.

    returns

    a DistinctDataQuantaBuilder representing the org.qcri.rheem.basic.operators.DistinctOperator's output

    Definition Classes
    DataQuantaBuilder
  19. def doWhile[Conv](conditionUdf: SerializablePredicate[Collection[Conv]], bodyBuilder: Function[DataQuantaBuilder[_, T], Tuple[DataQuantaBuilder[_, T], DataQuantaBuilder[_, Conv]]]): DoWhileDataQuantaBuilder[T, Conv]

    Feed the built DataQuanta into a org.qcri.rheem.basic.operators.DoWhileOperator.

    Feed the built DataQuanta into a org.qcri.rheem.basic.operators.DoWhileOperator.

    returns

    a DoWhileDataQuantaBuilder

    Definition Classes
    DataQuantaBuilder
  20. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  21. def equals(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any
  22. def filter(udf: SerializablePredicate[T]): FilterDataQuantaBuilder[T]

    Feed the built DataQuanta into a org.qcri.rheem.basic.operators.FilterOperator.

    Feed the built DataQuanta into a org.qcri.rheem.basic.operators.FilterOperator.

    udf

    filter UDF

    returns

    a FilterDataQuantaBuilder

    Definition Classes
    DataQuantaBuilder
  23. def finalize(): Unit

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  24. def flatMap[NewOut](udf: SerializableFunction[T, Iterable[NewOut]]): FlatMapDataQuantaBuilder[T, NewOut]

    Feed the built DataQuanta into a org.qcri.rheem.basic.operators.FlatMapOperator.

    Feed the built DataQuanta into a org.qcri.rheem.basic.operators.FlatMapOperator.

    udf

    the UDF for the org.qcri.rheem.basic.operators.FlatMapOperator

    returns

    a FlatMapDataQuantaBuilder

    Definition Classes
    DataQuantaBuilder
  25. def forEach(f: Consumer[T]): Unit

    Feed the built DataQuanta into a JavaFunction that runs locally.

    Feed the built DataQuanta into a JavaFunction that runs locally. This triggers execution of the constructed RheemPlan.

    f

    the JavaFunction

    returns

    the collected data quanta

    Definition Classes
    DataQuantaBuilder
  26. final def getClass(): Class[_]

    Definition Classes
    AnyRef → Any
  27. def getOutputTypeTrap: TypeTrap

    Retrieve an intialization value for outputTypeTrap.

    Retrieve an intialization value for outputTypeTrap.

    returns

    the TypeTrap

    Definition Classes
    SampleDataQuantaBuilderBasicDataQuantaBuilder
  28. def group(): GlobalGroupDataQuantaBuilder[T]

    Feed the built DataQuanta into a org.qcri.rheem.basic.operators.GlobalMaterializedGroupOperator.

    Feed the built DataQuanta into a org.qcri.rheem.basic.operators.GlobalMaterializedGroupOperator.

    returns

    a GlobalGroupDataQuantaBuilder

    Definition Classes
    DataQuantaBuilder
  29. def groupByKey[Key](keyUdf: SerializableFunction[T, Key]): GroupByDataQuantaBuilder[Key, T]

    Feed the built DataQuanta into a org.qcri.rheem.basic.operators.MaterializedGroupByOperator.

    Feed the built DataQuanta into a org.qcri.rheem.basic.operators.MaterializedGroupByOperator.

    keyUdf

    the key UDF for the org.qcri.rheem.basic.operators.MaterializedGroupByOperator

    returns

    a GroupByDataQuantaBuilder

    Definition Classes
    DataQuantaBuilder
  30. def hashCode(): Int

    Definition Classes
    AnyRef → Any
  31. def intersect(that: DataQuantaBuilder[_, T]): IntersectDataQuantaBuilder[T]

    Feed the built DataQuanta of this and the given instance into a org.qcri.rheem.basic.operators.IntersectOperator.

    Feed the built DataQuanta of this and the given instance into a org.qcri.rheem.basic.operators.IntersectOperator.

    that

    the other DataQuantaBuilder to intersect with

    returns

    an IntersectDataQuantaBuilder

    Definition Classes
    DataQuantaBuilder
  32. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any
  33. def join[ThatOut, Key](thisKeyUdf: SerializableFunction[T, Key], that: DataQuantaBuilder[_, ThatOut], thatKeyUdf: SerializableFunction[ThatOut, Key]): JoinDataQuantaBuilder[T, ThatOut, Key]

    Feed the built DataQuanta of this and the given instance into a org.qcri.rheem.basic.operators.JoinOperator.

    Feed the built DataQuanta of this and the given instance into a org.qcri.rheem.basic.operators.JoinOperator.

    thisKeyUdf

    the key extraction UDF for this instance

    that

    the other DataQuantaBuilder to join with

    thatKeyUdf

    the key extraction UDF for that instance

    returns

    a JoinDataQuantaBuilder

    Definition Classes
    DataQuantaBuilder
  34. def keyBy[Key](keyExtractor: SerializableFunction[T, Key]): KeyedDataQuantaBuilder[T, Key]

    Annotates a key to this instance.

    Annotates a key to this instance.

    keyExtractor

    extracts the key from the data quanta

    returns

    a KeyedDataQuantaBuilder

    Definition Classes
    DataQuantaBuilder
  35. def logger(): Logger

    Definition Classes
    Logging
  36. def map[NewOut](udf: SerializableFunction[T, NewOut]): MapDataQuantaBuilder[T, NewOut]

    Feed the built DataQuanta into a MapOperator.

    Feed the built DataQuanta into a MapOperator.

    udf

    the UDF for the MapOperator

    returns

    a MapDataQuantaBuilder

    Definition Classes
    DataQuantaBuilder
  37. def mapPartitions[NewOut](udf: SerializableFunction[Iterable[T], Iterable[NewOut]]): MapPartitionsDataQuantaBuilder[T, NewOut]

    Feed the built DataQuanta into a org.qcri.rheem.basic.operators.MapPartitionsOperator.

    Feed the built DataQuanta into a org.qcri.rheem.basic.operators.MapPartitionsOperator.

    udf

    the UDF for the org.qcri.rheem.basic.operators.MapPartitionsOperator

    returns

    a MapPartitionsDataQuantaBuilder

    Definition Classes
    DataQuantaBuilder
  38. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  39. final def notify(): Unit

    Definition Classes
    AnyRef
  40. final def notifyAll(): Unit

    Definition Classes
    AnyRef
  41. val outputTypeTrap: TypeTrap

    The type of the DataQuanta to be built.

    The type of the DataQuanta to be built.

    Attributes
    protected[org.qcri.rheem.api]
    Definition Classes
    BasicDataQuantaBuilderDataQuantaBuilder
  42. def project[NewOut](fieldNames: Array[String]): ProjectionDataQuantaBuilder[T, Nothing]

    Feed the built DataQuanta into a MapOperator with a org.qcri.rheem.basic.function.ProjectionDescriptor.

    Feed the built DataQuanta into a MapOperator with a org.qcri.rheem.basic.function.ProjectionDescriptor.

    fieldNames

    field names for the org.qcri.rheem.basic.function.ProjectionDescriptor

    returns

    a MapDataQuantaBuilder

    Definition Classes
    DataQuantaBuilder
  43. def reduce(udf: SerializableBinaryOperator[T]): GlobalReduceDataQuantaBuilder[T]

    Feed the built DataQuanta into a GlobalReduceOperator.

    Feed the built DataQuanta into a GlobalReduceOperator.

    udf

    the UDF for the GlobalReduceOperator

    returns

    a GlobalReduceDataQuantaBuilder

    Definition Classes
    DataQuantaBuilder
  44. def reduceByKey[Key](keyUdf: SerializableFunction[T, Key], udf: SerializableBinaryOperator[T]): ReduceByDataQuantaBuilder[Key, T]

    Feed the built DataQuanta into a org.qcri.rheem.basic.operators.ReduceByOperator.

    Feed the built DataQuanta into a org.qcri.rheem.basic.operators.ReduceByOperator.

    keyUdf

    the key UDF for the org.qcri.rheem.basic.operators.ReduceByOperator

    udf

    the UDF for the org.qcri.rheem.basic.operators.ReduceByOperator

    returns

    a ReduceByDataQuantaBuilder

    Definition Classes
    DataQuantaBuilder
  45. def repeat(numRepetitions: Int, bodyBuilder: Function[DataQuantaBuilder[_, T], DataQuantaBuilder[_, T]]): RepeatDataQuantaBuilder[T]

    Feed the built DataQuanta into a org.qcri.rheem.basic.operators.RepeatOperator.

    Feed the built DataQuanta into a org.qcri.rheem.basic.operators.RepeatOperator.

    returns

    a DoWhileDataQuantaBuilder

    Definition Classes
    DataQuantaBuilder
  46. def sample(sampleSizeFunction: IntUnaryOperator): SampleDataQuantaBuilder[T]

    Feed the built DataQuanta into a org.qcri.rheem.basic.operators.SampleOperator.

    Feed the built DataQuanta into a org.qcri.rheem.basic.operators.SampleOperator.

    sampleSizeFunction

    the absolute size of the sample as a function of the current iteration number

    returns

    a SampleDataQuantaBuilder

    Definition Classes
    DataQuantaBuilder
  47. def sample(sampleSize: Int): SampleDataQuantaBuilder[T]

    Feed the built DataQuanta into a org.qcri.rheem.basic.operators.SampleOperator.

    Feed the built DataQuanta into a org.qcri.rheem.basic.operators.SampleOperator.

    sampleSize

    the absolute size of the sample

    returns

    a SampleDataQuantaBuilder

    Definition Classes
    DataQuantaBuilder
  48. def sort[Key](keyUdf: SerializableFunction[T, Key]): SortDataQuantaBuilder[T, Key]

    Feed the built DataQuanta of this and the given instance into a org.qcri.rheem.basic.operators.SortOperator.

    Feed the built DataQuanta of this and the given instance into a org.qcri.rheem.basic.operators.SortOperator.

    keyUdf

    the key extraction UDF for this instance

    returns

    a SortDataQuantaBuilder

    Definition Classes
    DataQuantaBuilder
  49. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef
  50. def toString(): String

    Definition Classes
    AnyRef → Any
  51. def union(that: DataQuantaBuilder[_, T]): UnionDataQuantaBuilder[T]

    Feed the built DataQuanta of this and the given instance into a org.qcri.rheem.basic.operators.UnionAllOperator.

    Feed the built DataQuanta of this and the given instance into a org.qcri.rheem.basic.operators.UnionAllOperator.

    that

    the other DataQuantaBuilder to union with

    returns

    a UnionDataQuantaBuilder

    Definition Classes
    DataQuantaBuilder
  52. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  53. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  54. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  55. def withBroadcast[Sender <: DataQuantaBuilder[_, _]](sender: Sender, broadcastName: String): SampleDataQuantaBuilder[T]

    Register a broadcast with the DataQuanta to be built

    Register a broadcast with the DataQuanta to be built

    sender

    a DataQuantaBuilder constructing the broadcasted DataQuanta

    broadcastName

    the name of the broadcast

    returns

    this instance

    Definition Classes
    BasicDataQuantaBuilderDataQuantaBuilder
  56. def withCardinalityEstimator(cardinalityEstimator: CardinalityEstimator): SampleDataQuantaBuilder[T]

    Set a CardinalityEstimator for the currently built DataQuanta.

    Set a CardinalityEstimator for the currently built DataQuanta.

    cardinalityEstimator

    the CardinalityEstimator

    returns

    this instance

    Definition Classes
    BasicDataQuantaBuilderDataQuantaBuilder
  57. def withDatasetSize(datasetSize: Long): SampleDataQuantaBuilder[T]

    Set the size of the dataset that should be sampled.

    Set the size of the dataset that should be sampled.

    datasetSize

    the size of the dataset

    returns

    this instance

  58. def withExperiment(experiment: Experiment): SampleDataQuantaBuilder[T]

    Set an Experiment for the currently built org.qcri.rheem.core.api.Job.

    Set an Experiment for the currently built org.qcri.rheem.core.api.Job.

    experiment

    the Experiment

    returns

    this instance

    Definition Classes
    BasicDataQuantaBuilderDataQuantaBuilder
  59. def withName(name: String): SampleDataQuantaBuilder[T]

    Set a name for the DataQuanta and its associated org.qcri.rheem.core.plan.rheemplan.Operators.

    Set a name for the DataQuanta and its associated org.qcri.rheem.core.plan.rheemplan.Operators.

    name

    the name

    returns

    this instance

    Definition Classes
    BasicDataQuantaBuilderDataQuantaBuilder
  60. def withOutputClass(cls: Class[T]): SampleDataQuantaBuilder[T]

    Explicitly set an output Class for the currently built DataQuanta.

    Explicitly set an output Class for the currently built DataQuanta. Note that it is not always necessary to set it and that it can be inferred in some situations.

    cls

    the output Class

    returns

    this instance

    Definition Classes
    BasicDataQuantaBuilderDataQuantaBuilder
  61. def withOutputType(outputType: DataSetType[T]): SampleDataQuantaBuilder[T]

    Explicitly set an output DataSetType for the currently built DataQuanta.

    Explicitly set an output DataSetType for the currently built DataQuanta. Note that it is not always necessary to set it and that it can be inferred in some situations.

    outputType

    the output DataSetType

    returns

    this instance

    Definition Classes
    BasicDataQuantaBuilderDataQuantaBuilder
  62. def withSampleMethod(sampleMethod: Methods): SampleDataQuantaBuilder[T]

    Set the sample method to be used.

    Set the sample method to be used.

    sampleMethod

    the sample method

    returns

    this instance

  63. def withSeed(seed: Long): SampleDataQuantaBuilder[T]

    Set the sample method to be used.

    Set the sample method to be used.

    seed
    returns

    this instance

  64. def withTargetPlatform(platform: Platform): SampleDataQuantaBuilder[T]

    Add a target Platform on which the currently built DataQuanta should be calculated.

    Add a target Platform on which the currently built DataQuanta should be calculated. Can be invoked multiple times to set multiple possilbe target Platforms or not at all to impose no restrictions.

    platform

    the CardinalityEstimator

    returns

    this instance

    Definition Classes
    BasicDataQuantaBuilderDataQuantaBuilder
  65. def withUdfJar(path: String): SampleDataQuantaBuilder[T]

    Register a JAR file with the currently built org.qcri.rheem.core.api.Job.

    Register a JAR file with the currently built org.qcri.rheem.core.api.Job.

    path

    the path of the JAR file

    returns

    this instance

    Definition Classes
    BasicDataQuantaBuilderDataQuantaBuilder
  66. def withUdfJarOf(cls: Class[_]): SampleDataQuantaBuilder[T]

    Register the JAR file containing the given Class with the currently built org.qcri.rheem.core.api.Job.

    Register the JAR file containing the given Class with the currently built org.qcri.rheem.core.api.Job.

    cls

    the Class

    returns

    this instance

    Definition Classes
    BasicDataQuantaBuilderDataQuantaBuilder
  67. def writeTextFile(url: String, formatterUdf: SerializableFunction[T, String], jobName: String, udfLoadProfileEstimator: LoadProfileEstimator): Unit

    Feed the built DataQuanta into a org.qcri.rheem.basic.operators.TextFileSink.

    Feed the built DataQuanta into a org.qcri.rheem.basic.operators.TextFileSink. This triggers execution of the constructed RheemPlan.

    url

    the URL of the file to be written

    returns

    the collected data quanta

    Definition Classes
    DataQuantaBuilder
  68. def writeTextFile(url: String, formatterUdf: SerializableFunction[T, String], jobName: String): Unit

    Feed the built DataQuanta into a org.qcri.rheem.basic.operators.TextFileSink.

    Feed the built DataQuanta into a org.qcri.rheem.basic.operators.TextFileSink. This triggers execution of the constructed RheemPlan.

    url

    the URL of the file to be written

    jobName

    optional name for the RheemPlan

    returns

    the collected data quanta

    Definition Classes
    DataQuantaBuilder
  69. def zipWithId: ZipWithIdDataQuantaBuilder[T]

    Feed the built DataQuanta into a org.qcri.rheem.basic.operators.ZipWithIdOperator.

    Feed the built DataQuanta into a org.qcri.rheem.basic.operators.ZipWithIdOperator.

    returns

    a ZipWithIdDataQuantaBuilder representing the org.qcri.rheem.basic.operators.ZipWithIdOperator's output

    Definition Classes
    DataQuantaBuilder

Inherited from Logging

Inherited from AnyRef

Inherited from Any

Ungrouped