Package

quasar

qscript

Permalink

package qscript

Here we no longer care about provenance. Backends can’t do anything with it, so we simply represent joins and crosses directly. This also means that we don’t need to model certain things – project_d is just a data-level function, nest_d & swap_d only modify provenance and so are irrelevant here, and autojoin_d has been replaced with a lower-level join operation that doesn’t include the cross portion.

Linear Supertypes
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. qscript
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Type Members

  1. final case class AbsMerge[T[_[_]], A, Q[_[_[_]]]](src: A, left: Q[T], right: Q[T]) extends Product with Serializable

    Permalink
  2. sealed abstract class Binary[T[_[_]], A] extends MapFunc[T, A]

    Permalink
  3. final case class BucketField[T[_[_]], A](src: A, value: FreeMap[T], name: FreeMap[T]) extends QScriptBucket[T, A] with Product with Serializable

    Permalink
  4. final case class BucketIndex[T[_[_]], A](src: A, value: FreeMap[T], index: FreeMap[T]) extends QScriptBucket[T, A] with Product with Serializable

    Permalink
  5. trait Bucketable[F[_]] extends Serializable

    Permalink
  6. sealed abstract class DeadEnd extends AnyRef

    Permalink
  7. final case class Drop[T[_[_]], A](src: A, from: FreeQS[T], count: FreeQS[T]) extends QScriptCore[T, A] with Product with Serializable

    Permalink
  8. trait ElideBuckets[F[_]] extends Serializable

    Permalink
  9. trait ElideBucketsInstances extends ElideBucketsInstances0

    Permalink
  10. trait ElideBucketsInstances0 extends AnyRef

    Permalink
  11. final case class EquiJoin[T[_[_]], A](src: A, lBranch: FreeQS[T], rBranch: FreeQS[T], lKey: FreeMap[T], rKey: FreeMap[T], f: JoinType, combine: JoinFunc[T]) extends Product with Serializable

    Permalink
  12. type EquiQScript[T[_[_]], A] = Coproduct[[β]EquiJoin[T, β], [β]Coproduct[Read, [β]Coproduct[[β]QScriptCore[T, β], [β]Coproduct[[β]Const[DeadEnd, β], [β]SourcedPathable[T, β], β], β], β], A]

    Permalink

    A variant with a simpler join type.

    A variant with a simpler join type. A backend can choose to operate on this structure by applying the equiJoinsOnly transformation. Backends without true join support will likely find it easier to work with this than to handle full ThetaJoins.

  13. final case class Filter[T[_[_]], A](src: A, f: FreeMap[T]) extends QScriptCore[T, A] with Product with Serializable

    Permalink

    Eliminates some values from a dataset, based on the result of FilterFunc.

  14. type FreeMap[T[_[_]]] = Free[[β]MapFunc[T, β], Unit]

    Permalink
  15. type FreeQS[T[_[_]]] = Free[[β]Coproduct[[β]QScriptBucket[T, β], [β]Coproduct[[β]ThetaJoin[T, β], [β]Coproduct[[β]QScriptCore[T, β], [β]Coproduct[[β]Const[DeadEnd, β], [β]SourcedPathable[T, β], β], β], β], β], Unit]

    Permalink
  16. type FreeUnit[F[_]] = Free[F, Unit]

    Permalink
  17. final case class GroupBy[T[_[_]], A](src: A, values: FreeMap[T], bucket: FreeMap[T]) extends QScriptBucket[T, A] with Product with Serializable

    Permalink
  18. trait Helpers[T[_[_]]] extends AnyRef

    Permalink
  19. type JoinFunc[T[_[_]]] = Free[[β]MapFunc[T, β], JoinSide]

    Permalink
  20. sealed trait JoinSide extends AnyRef

    Permalink
  21. sealed abstract class JoinType extends AnyRef

    Permalink
  22. final case class LeftShift[T[_[_]], A](src: A, struct: FreeMap[T], repair: JoinFunc[T]) extends SourcedPathable[T, A] with Product with Serializable

    Permalink

    Flattens nested structure, converting each value into a data set, which are then unioned.

    Flattens nested structure, converting each value into a data set, which are then unioned.

    struct is an expression that evaluates to an array or object, which is then “exploded” into multiple values. repair is applied across the new set, integrating the exploded values into the original set.

  23. final case class LeftShiftBucket[T[_[_]], A](src: A, struct: FreeMap[T], repair: JoinFunc[T], bucketShift: FreeMap[T]) extends QScriptBucket[T, A] with Product with Serializable

    Permalink
  24. final case class Map[T[_[_]], A](src: A, f: FreeMap[T]) extends SourcedPathable[T, A] with Product with Serializable

    Permalink

    A data-level transformation.

  25. sealed abstract class MapFunc[T[_[_]], A] extends AnyRef

    Permalink
  26. type Merge[T[_[_]], A] = AbsMerge[T, A, FreeMap]

    Permalink
  27. type MergeJoin[T[_[_]], A] = AbsMerge[T, A, FreeQS]

    Permalink
  28. trait Mergeable[A] extends Serializable

    Permalink
  29. trait Normalizable[F[_]] extends Serializable

    Permalink
  30. trait NormalizableInstances extends NormalizableInstances0

    Permalink
  31. trait NormalizableInstances0 extends AnyRef

    Permalink
  32. final case class Nullary[T[_[_]], A](ejson: T[EJson]) extends MapFunc[T, A] with Product with Serializable

    Permalink
  33. class Optimize[T[_[_]]] extends Helpers[T]

    Permalink
  34. type Pathable[T[_[_]], A] = Coproduct[[β]Const[DeadEnd, β], [β]SourcedPathable[T, β], A]

    Permalink
  35. type QScript[T[_[_]], A] = Coproduct[[β]ThetaJoin[T, β], [β]Coproduct[Read, [β]Coproduct[[β]QScriptCore[T, β], [β]Coproduct[[β]Const[DeadEnd, β], [β]SourcedPathable[T, β], β], β], β], A]

    Permalink

    This is the primary form seen by a backend.

    This is the primary form seen by a backend. It contains reads of files.

  36. sealed abstract class QScriptBucket[T[_[_]], A] extends AnyRef

    Permalink
  37. type QScriptCommon[T[_[_]], A] = Coproduct[Read, [β]Coproduct[[β]QScriptCore[T, β], [β]Coproduct[[β]Const[DeadEnd, β], [β]SourcedPathable[T, β], β], β], A]

    Permalink

    These nodes exist in all QScript structures that a backend sees.

  38. sealed abstract class QScriptCore[T[_[_]], A] extends AnyRef

    Permalink
  39. type QScriptInternal[T[_[_]], A] = Coproduct[[β]QScriptBucket[T, β], [β]Coproduct[[β]ThetaJoin[T, β], [β]Coproduct[[β]QScriptCore[T, β], [β]Coproduct[[β]Const[DeadEnd, β], [β]SourcedPathable[T, β], β], β], β], A]

    Permalink
  40. type QScriptPrim[T[_[_]], A] = Coproduct[[β]QScriptCore[T, β], [β]Coproduct[[β]Const[DeadEnd, β], [β]SourcedPathable[T, β], β], A]

    Permalink

    These are the operations included in all forms of QScript.

  41. type QScriptPure[T[_[_]], A] = Coproduct[[β]ThetaJoin[T, β], [β]Coproduct[[β]QScriptCore[T, β], [β]Coproduct[[β]Const[DeadEnd, β], [β]SourcedPathable[T, β], β], β], A]

    Permalink

    This is the target of the core compiler.

    This is the target of the core compiler. Normalization is applied to this structure, and it contains no Read or EquiJoin.

  42. final case class Read[A](src: A, path: AbsFile[Sandboxed]) extends Product with Serializable

    Permalink

    A backend-resolved Root, which is now a path.

  43. final case class Reduce[T[_[_]], A, N <: Nat](src: A, bucket: FreeMap[T], reducers: Sized[List[ReduceFunc[FreeMap[T]]], Succ[N]], repair: Free[[β]MapFunc[T, β], Fin[Succ[N]]]) extends QScriptCore[T, A] with Product with Serializable

    Permalink

    Performs a reduction over a dataset, with the dataset partitioned by the result of the MapFunc.

    Performs a reduction over a dataset, with the dataset partitioned by the result of the MapFunc. So, rather than many-to-one, this is many-to-fewer.

    bucket partitions the values into buckets based on the result of the expression, reducers applies the provided reduction to each expression, and repair finally turns those reduced expressions into a final value.

  44. sealed trait ReduceFunc[A] extends AnyRef

    Permalink
  45. final case class Sort[T[_[_]], A](src: A, bucket: FreeMap[T], order: List[(FreeMap[T], SortDir)]) extends QScriptCore[T, A] with Product with Serializable

    Permalink

    Sorts values within a bucket.

    Sorts values within a bucket. This could be represented with LeftShift(Map(_.sort, Reduce(_ :: _, ???)) but backends tend to provide sort directly, so this avoids backends having to recognize the pattern. We could provide an algebra (Sort :+: QScript)#λ => QScript so that a backend without a native sort could eliminate this node.

  46. sealed trait SortDir extends AnyRef

    Permalink
  47. sealed abstract class SourcedPathable[T[_[_]], A] extends AnyRef

    Permalink
  48. final case class SquashBucket[T[_[_]], A](src: A) extends QScriptBucket[T, A] with Product with Serializable

    Permalink
  49. final case class Take[T[_[_]], A](src: A, from: FreeQS[T], count: FreeQS[T]) extends QScriptCore[T, A] with Product with Serializable

    Permalink
  50. sealed abstract class Ternary[T[_[_]], A] extends MapFunc[T, A]

    Permalink
  51. final case class ThetaJoin[T[_[_]], A](src: A, lBranch: FreeQS[T], rBranch: FreeQS[T], on: JoinFunc[T], f: JoinType, combine: JoinFunc[T]) extends Product with Serializable

    Permalink

    Applies a function across two datasets, in the cases where the JoinFunc evaluates to true.

    Applies a function across two datasets, in the cases where the JoinFunc evaluates to true. The branches represent the divergent operations applied to some common src. Each branch references the src exactly once. (Since no constructor has more than one recursive component, it’s guaranteed that neither side references the src _more_ than once.)

    This case represents a full θJoin, but we could have an algebra that rewites it as Filter(_, EquiJoin(...)) to simplify behavior for the backend.

  52. class Transform[T[_[_]]] extends Helpers[T]

    Permalink
  53. sealed abstract class Unary[T[_[_]], A] extends MapFunc[T, A]

    Permalink
  54. final case class Union[T[_[_]], A](src: A, lBranch: FreeQS[T], rBranch: FreeQS[T]) extends SourcedPathable[T, A] with Product with Serializable

    Permalink

    Creates a new dataset, |a|+|b|, containing all of the entries from each of the input sets, without any indication of which set they came from

    Creates a new dataset, |a|+|b|, containing all of the entries from each of the input sets, without any indication of which set they came from

    This could be handled as another join type, the anti-join (T[EJson] \/ T[EJson] => T[EJson], specifically as _.merge), with the condition being κ(true),

Value Members

  1. object BucketField extends Serializable

    Permalink
  2. object BucketIndex extends Serializable

    Permalink
  3. object Bucketable extends Serializable

    Permalink
  4. val CommonEJson: :<:[Common, EJson]

    Permalink
  5. object DeadEnd

    Permalink
  6. def DeadEndInternal[T[_[_]]]: :<:[[β]Const[DeadEnd, β], [β]Coproduct[[β]QScriptBucket[T, β], [β]Coproduct[[β]ThetaJoin[T, β], [β]Coproduct[[β]QScriptCore[T, β], [β]Coproduct[[β]Const[DeadEnd, β], [β]SourcedPathable[T, β], β], β], β], β]]

    Permalink
  7. object Drop extends Serializable

    Permalink
  8. object ElideBuckets extends ElideBucketsInstances with Serializable

    Permalink
  9. object Empty extends DeadEnd with Product with Serializable

    Permalink
  10. object EquiJoin extends Serializable

    Permalink
  11. val ExtEJson: :<:[Extension, EJson]

    Permalink
  12. object Filter extends Serializable

    Permalink
  13. object FullOuter extends JoinType with Product with Serializable

    Permalink
  14. object GroupBy extends Serializable

    Permalink
  15. object Inner extends JoinType with Product with Serializable

    Permalink
  16. object JoinSide

    Permalink
  17. object JoinType

    Permalink
  18. object LeftOuter extends JoinType with Product with Serializable

    Permalink
  19. object LeftShift extends Serializable

    Permalink
  20. object LeftShiftBucket extends Serializable

    Permalink
  21. object LeftSide extends JoinSide with Product with Serializable

    Permalink
  22. object Map extends Serializable

    Permalink
  23. object MapFunc

    Permalink
  24. object MapFuncs

    Permalink
  25. object Mergeable extends Serializable

    Permalink
  26. object Normalizable extends NormalizableInstances with Serializable

    Permalink
  27. object QScriptBucket

    Permalink
  28. def QScriptBucketInternal[T[_[_]]]: :<:[[β]QScriptBucket[T, β], [β]Coproduct[[β]QScriptBucket[T, β], [β]Coproduct[[β]ThetaJoin[T, β], [β]Coproduct[[β]QScriptCore[T, β], [β]Coproduct[[β]Const[DeadEnd, β], [β]SourcedPathable[T, β], β], β], β], β]]

    Permalink
  29. object QScriptCore

    Permalink
  30. def QScriptCoreInternal[T[_[_]]]: :<:[[β]QScriptCore[T, β], [β]Coproduct[[β]QScriptBucket[T, β], [β]Coproduct[[β]ThetaJoin[T, β], [β]Coproduct[[β]QScriptCore[T, β], [β]Coproduct[[β]Const[DeadEnd, β], [β]SourcedPathable[T, β], β], β], β], β]]

    Permalink
  31. object Read extends Serializable

    Permalink
  32. object Reduce extends Serializable

    Permalink
  33. object ReduceFunc

    Permalink
  34. object ReduceFuncs

    Permalink
  35. object RightOuter extends JoinType with Product with Serializable

    Permalink
  36. object RightSide extends JoinSide with Product with Serializable

    Permalink
  37. object Root extends DeadEnd with Product with Serializable

    Permalink

    The top level of a filesystem.

    The top level of a filesystem. During compilation this represents /, but in the structure a backend sees, it represents the mount point.

  38. object Sort extends Serializable

    Permalink
  39. object SortDir

    Permalink
  40. object SourcedPathable

    Permalink
  41. def SourcedPathableInternal[T[_[_]]]: :<:[[β]SourcedPathable[T, β], [β]Coproduct[[β]QScriptBucket[T, β], [β]Coproduct[[β]ThetaJoin[T, β], [β]Coproduct[[β]QScriptCore[T, β], [β]Coproduct[[β]Const[DeadEnd, β], [β]SourcedPathable[T, β], β], β], β], β]]

    Permalink
  42. object SquashBucket extends Serializable

    Permalink
  43. object Take extends Serializable

    Permalink
  44. object ThetaJoin extends Serializable

    Permalink
  45. def ThetaJoinInternal[T[_[_]]]: :<:[[β]ThetaJoin[T, β], [β]Coproduct[[β]QScriptBucket[T, β], [β]Coproduct[[β]ThetaJoin[T, β], [β]Coproduct[[β]QScriptCore[T, β], [β]Coproduct[[β]Const[DeadEnd, β], [β]SourcedPathable[T, β], β], β], β], β]]

    Permalink
  46. object Union extends Serializable

    Permalink
  47. def UnitF[T[_[_]]]: Free[[β]MapFunc[T, β], Unit]

    Permalink
  48. implicit def constBucketable[T[_[_]]](implicit arg0: Corecursive[T]): Aux[T, [β]Const[DeadEnd, β]]

    Permalink
  49. def rebase[M[_], A](in: M[A], field: M[A])(implicit arg0: Bind[M]): M[A]

    Permalink

Inherited from AnyRef

Inherited from Any

MRA

Ungrouped