com.twitter.scalding

Config

trait Config extends Serializable

This is a wrapper class on top of Map[String, String]

Linear Supertypes
Serializable, Serializable, AnyRef, Any
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. Config
  2. Serializable
  3. Serializable
  4. AnyRef
  5. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Abstract Value Members

  1. abstract def toMap: Map[String, String]

Concrete Value Members

  1. final def !=(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  2. final def !=(arg0: Any): Boolean

    Definition Classes
    Any

  3. final def ##(): Int

    Definition Classes
    AnyRef → Any

  4. def +(kv: (String, String)): Config

  5. def ++(that: Config): Config

  6. def -(k: String): Config

  7. final def ==(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  8. final def ==(arg0: Any): Boolean

    Definition Classes
    Any

  9. def addCascadingClassSerializationTokens(clazzes: Set[Class[_]]): Config

    The serialization of your data will be smaller if any classes passed between tasks in your job are listed here.

    The serialization of your data will be smaller if any classes passed between tasks in your job are listed here. Without this, strings are used to write the types IN EACH RECORD, which compression probably takes care of, but compression acts AFTER the data is serialized into buffers and spilling has been triggered.

  10. def addFlowListener(flowListenerProvider: (Mode, Config) ⇒ FlowListener): Config

    configure flow listeneres for observability

  11. def addFlowStepListener(flowListenerProvider: (Mode, Config) ⇒ FlowStepListener): Config

  12. def addFlowStepStrategy(flowStrategyProvider: (Mode, Config) ⇒ FlowStepStrategy[JobConf]): Config

  13. def addReducerEstimator(clsName: String): Config

    Prepend an estimator so it will be tried first.

    Prepend an estimator so it will be tried first. If it returns None, the previously-set estimators will be tried in order.

  14. def addReducerEstimator[T](cls: Class[T]): Config

    Prepend an estimator so it will be tried first.

    Prepend an estimator so it will be tried first. If it returns None, the previously-set estimators will be tried in order.

  15. def addUniqueId(u: UniqueID): Config

  16. final def asInstanceOf[T0]: T0

    Definition Classes
    Any

  17. def clearFlowStepStrategies: Config

  18. def clone(): AnyRef

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  19. def ensureUniqueId: (UniqueID, Config)

    Allocate a new UniqueID if there is not one present

  20. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  21. def equals(that: Any): Boolean

    Definition Classes
    Config → AnyRef → Any

  22. def finalize(): Unit

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )

  23. def get(key: String): Option[String]

  24. def getArgs: Args

  25. def getCascadingAppJar: Option[Try[Class[_]]]

    Returns None if not set, otherwise reflection is used to create the Class.

    Returns None if not set, otherwise reflection is used to create the Class.forName

  26. def getCascadingAppName: Option[String]

    This is a name that if present is passed to flow.

    This is a name that if present is passed to flow.setName, which should appear in the job tracker.

  27. def getCascadingSerializationTokens: Map[Int, String]

  28. final def getClass(): Class[_]

    Definition Classes
    AnyRef → Any

  29. def getFlowListeners: List[Try[(Mode, Config) ⇒ FlowListener]]

  30. def getFlowStepListeners: List[Try[(Mode, Config) ⇒ FlowStepListener]]

  31. def getFlowStepStrategies: List[Try[(Mode, Config) ⇒ FlowStepStrategy[JobConf]]]

  32. def getHashJoinAutoForceRight: Boolean

  33. def getKryo: Option[KryoInstantiator]

  34. def getKryoRegisteredClasses: Set[Class[_]]

    This function gets the set of classes that have been registered to Kryo.

    This function gets the set of classes that have been registered to Kryo. They may or may not be used in this job, but Cascading might want to be made aware that these classes exist

  35. def getNumReducers: Option[Int]

    Get the number of reducers (this is the parameter Hadoop will use)

  36. def getRequireOrderedSerialization: Boolean

  37. def getScaldingExecutionId: Option[String]

  38. def getScaldingVersion: Option[String]

  39. def getSubmittedTimestamp: Option[RichDate]

  40. def getUniqueIds: Set[UniqueID]

  41. def hashCode(): Int

    Definition Classes
    Config → AnyRef → Any

  42. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any

  43. def maybeSetSubmittedTimestamp(date: RichDate = RichDate.now): (Option[RichDate], Config)

  44. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  45. final def notify(): Unit

    Definition Classes
    AnyRef

  46. final def notifyAll(): Unit

    Definition Classes
    AnyRef

  47. def setArgs(args: Args): Config

  48. def setCascadingAppId(id: String): Config

  49. def setCascadingAppJar(clazz: Class[_]): Config

    Non-fat-jar use cases require this, BUT using it with fat jars can cause problems.

    Non-fat-jar use cases require this, BUT using it with fat jars can cause problems. It is not set by default, but if you have problems you might need to set the Job class here Consider also setting this same class here: setScaldingFlowClass

  50. def setCascadingAppName(name: String): Config

  51. def setDefaultComparator(clazz: Class[_ <: Comparator[_]]): Config

  52. def setHRavenHistoryUserName: Config

    Set username from System.

    Set username from System.used for querying hRaven.

  53. def setHashJoinAutoForceRight(b: Boolean): Config

  54. def setListSpillThreshold(count: Int): Config

  55. def setMapSideAggregationThreshold(count: Int): Config

  56. def setMapSpillThreshold(count: Int): Config

  57. def setNumReducers(n: Int): Config

  58. def setReducerEstimators(clsList: String): Config

    Set the entire list of reducer estimators (overriding the existing list)

  59. def setRequireOrderedSerialization(b: Boolean): Config

    Set this configuration option to require all grouping/cogrouping to use OrderedSerialization

  60. def setScaldingExecutionId(id: String): Config

    Set an ID to be shared across this usage of run for Execution

  61. def setScaldingFlowClass(clazz: Class[_]): Config

  62. def setScaldingVersion: Config

  63. def setSerialization(kryo: Either[(Class[_ <: KryoInstantiator], KryoInstantiator), Class[_ <: KryoInstantiator]], userHadoop: Seq[Class[_ <: Serialization[_]]] = Nil): Config

  64. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef

  65. def toString(): String

    Definition Classes
    AnyRef → Any

  66. def update[R](k: String)(fn: (Option[String]) ⇒ (Option[String], R)): (R, Config)

  67. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  68. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  69. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from Serializable

Inherited from Serializable

Inherited from AnyRef

Inherited from Any

Ungrouped