shark.execution

UnaryOperator

abstract class UnaryOperator[T <: HiveDesc] extends Operator[T]

A base operator class that has at most one parent. Operators implementations should override the following methods:

preprocessRdd: Called on the master. Can be used to transform the RDD before passing it to processPartition. For example, the operator can use this function to sort the input.

processPartition: Called on each slave on the output of preprocessRdd. This can be used to transform rows into their desired format.

postprocessRdd: Called on the master to transform the output of processPartition before sending it downstream.

Linear Supertypes
Operator[T], Serializable, Serializable, LogHelper, Logging, AnyRef, Any
Known Subclasses
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. UnaryOperator
  2. Operator
  3. Serializable
  4. Serializable
  5. LogHelper
  6. Logging
  7. AnyRef
  8. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Instance Constructors

  1. new UnaryOperator()

Abstract Value Members

  1. abstract def processPartition(split: Int, iter: Iterator[_]): Iterator[_]

    Process a partition.

    Process a partition. Called on slaves.

    Definition Classes
    UnaryOperatorOperator

Concrete Value Members

  1. final def !=(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  2. final def !=(arg0: Any): Boolean

    Definition Classes
    Any
  3. final def ##(): Int

    Definition Classes
    AnyRef → Any
  4. final def ==(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  5. final def ==(arg0: Any): Boolean

    Definition Classes
    Any
  6. def addChild(child: Operator[_ <: HiveDesc]): Unit

    Definition Classes
    Operator
  7. def addParent(parent: Operator[_ <: HiveDesc]): Unit

    Definition Classes
    Operator
  8. final def asInstanceOf[T0]: T0

    Definition Classes
    Any
  9. def childOperators: ArrayBuffer[Operator[_ <: HiveDesc]]

    Definition Classes
    Operator
  10. def clone(): AnyRef

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  11. def desc: T

    Definition Classes
    Operator
  12. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  13. def equals(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any
  14. def errStream(): PrintStream

    Definition Classes
    LogHelper
  15. def execute(): RDD[_]

    Execute the operator.

    Execute the operator. This should recursively execute parent operators.

    Definition Classes
    UnaryOperatorOperator
  16. def executeParents(): Seq[(Int, RDD[_])]

    Attributes
    protected
    Definition Classes
    Operator
  17. def finalize(): Unit

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  18. final def getClass(): Class[_]

    Definition Classes
    AnyRef → Any
  19. def getTag: Int

    Return the join tag.

    Return the join tag. This is usually just 0. ReduceSink might set it to something else.

    Definition Classes
    Operator
  20. def hashCode(): Int

    Definition Classes
    AnyRef → Any
  21. def hconf: HiveConf

    Definition Classes
    Operator
  22. def initEvaluators(evals: Array[ExprNodeEvaluator], start: Int, length: Int, rowInspector: ObjectInspector): Array[ObjectInspector]

    Initialize an array of ExprNodeEvaluator from start, for specified length and return the result ObjectInspectors.

    Initialize an array of ExprNodeEvaluator from start, for specified length and return the result ObjectInspectors.

    Attributes
    protected
    Definition Classes
    Operator
  23. def initEvaluators(evals: Array[ExprNodeEvaluator], rowInspector: ObjectInspector): Array[ObjectInspector]

    Initialize an array of ExprNodeEvaluator and return the result ObjectInspectors.

    Initialize an array of ExprNodeEvaluator and return the result ObjectInspectors.

    Attributes
    protected
    Definition Classes
    Operator
  24. def initEvaluatorsAndReturnStruct(evals: Array[ExprNodeEvaluator], outputColName: List[String], rowInspector: ObjectInspector): StructObjectInspector

    Initialize an array of ExprNodeEvaluator and put the return values into a StructObjectInspector with integer field names.

    Initialize an array of ExprNodeEvaluator and put the return values into a StructObjectInspector with integer field names.

    Attributes
    protected
    Definition Classes
    Operator
  25. def initEvaluatorsAndReturnStruct(evals: Array[ExprNodeEvaluator], fieldObjectInspectors: Array[ObjectInspector], distinctColIndices: List[List[Integer]], outputColNames: List[String], length: Int, rowInspector: ObjectInspector): StructObjectInspector

    Copy from the org.

    Copy from the org.apache.hadoop.hive.ql.exec.ReduceSinkOperator Initializes array of ExprNodeEvaluator. Adds Union field for distinct column indices for group by. Puts the return values into a StructObjectInspector with output column names.

    If distinctColIndices is empty, the object inspector is same as Operator#initEvaluatorsAndReturnStruct(ExprNodeEvaluator[], List, ObjectInspector)

    Attributes
    protected
    Definition Classes
    Operator
  26. def initEvaluatorsAndReturnStruct(evals: Array[ExprNodeEvaluator], distinctColIndices: List[List[Integer]], outputColNames: List[String], length: Int, rowInspector: ObjectInspector): StructObjectInspector

    Copy from the org.

    Copy from the org.apache.hadoop.hive.ql.exec.ReduceSinkOperator Initializes array of ExprNodeEvaluator. Adds Union field for distinct column indices for group by. Puts the return values into a StructObjectInspector with output column names.

    If distinctColIndices is empty, the object inspector is same as Operator#initEvaluatorsAndReturnStruct(ExprNodeEvaluator[], List, ObjectInspector)

    Attributes
    protected
    Definition Classes
    Operator
  27. def initializeMasterOnAll(): Unit

    Recursively calls initializeOnMaster() for the entire query plan.

    Recursively calls initializeOnMaster() for the entire query plan. Parent operators are called before children.

    Definition Classes
    Operator
  28. def initializeOnMaster(): Unit

    Initialize the operator on master node.

    Initialize the operator on master node. This can have dependency on other nodes. When an operator's initializeOnMaster() is invoked, all its parents' initializeOnMaster() have been invoked.

    Definition Classes
    Operator
  29. def initializeOnSlave(): Unit

    Initialize the operator on slave nodes.

    Initialize the operator on slave nodes. This method should have no dependency on parents or children. Everything that is not used in this method should be marked @transient.

    Definition Classes
    Operator
  30. def inputObjectInspectors(): Seq[ObjectInspector]

    Attributes
    protected
    Definition Classes
    Operator
  31. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any
  32. def isTraceEnabled(): Boolean

    Attributes
    protected
    Definition Classes
    Logging
  33. def log: Logger

    Attributes
    protected
    Definition Classes
    Logging
  34. def logDebug(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging
  35. def logDebug(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging
  36. def logError(msg: String, exception: Throwable): Unit

    Definition Classes
    LogHelper
  37. def logError(msg: String, detail: String): Unit

    Definition Classes
    LogHelper
  38. def logError(msg: ⇒ String): Unit

    Definition Classes
    LogHelper → Logging
  39. def logError(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging
  40. def logInfo(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging
  41. def logInfo(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging
  42. def logTrace(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging
  43. def logTrace(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging
  44. def logWarning(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging
  45. def logWarning(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging
  46. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  47. final def notify(): Unit

    Definition Classes
    AnyRef
  48. final def notifyAll(): Unit

    Definition Classes
    AnyRef
  49. def objectInspector: ObjectInspector

  50. var objectInspectors: Seq[ObjectInspector]

    Definition Classes
    Operator
  51. def outStream(): PrintStream

    Definition Classes
    LogHelper
  52. def outputObjectInspector(): ObjectInspector

    Definition Classes
    Operator
  53. def parentOperator: Operator[HiveDesc]

  54. def parentOperators: ArrayBuffer[Operator[_ <: HiveDesc]]

    Definition Classes
    Operator
  55. def parentOperatorsAsJavaList: List[Operator[_ <: HiveDesc]]

    Return the parent operators as a Java List.

    Return the parent operators as a Java List. This is for interoperability with Java. We use this in explain's Java code.

    Definition Classes
    Operator
  56. def postprocessRdd(rdd: RDD[_]): RDD[_]

    Called on master.

  57. def preprocessRdd(rdd: RDD[_]): RDD[_]

    Called on master.

  58. def returnTerminalOperators(): Seq[Operator[_ <: HiveDesc]]

    Definition Classes
    Operator
  59. def returnTopOperators(): Seq[Operator[_]]

    Definition Classes
    Operator
  60. def setDesc[B >: T](d: B): Unit

    Definition Classes
    Operator
  61. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef
  62. def toString(): String

    Definition Classes
    AnyRef → Any
  63. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  64. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  65. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from Operator[T]

Inherited from Serializable

Inherited from Serializable

Inherited from LogHelper

Inherited from Logging

Inherited from AnyRef

Inherited from Any

Ungrouped