com.twitter

scalding

package scalding

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. scalding
  2. AnyRef
  3. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Type Members

  1. type Grouped[K, +V] = scalding.typed.Grouped[K, V]

  2. case class JsonLine(p: String, fields: Fields = Fields.ALL, sinkMode: SinkMode = SinkMode.REPLACE, transformInTest: Boolean = false, failOnEmptyLines: Boolean = true) extends FixedPathSource with TextLineScheme with Product with Serializable

    This Source writes out the TupleEntry as a simple JSON object, using the field names as keys and the string representation of the values.

    This Source writes out the TupleEntry as a simple JSON object, using the field names as keys and the string representation of the values.

    TODO: it would be nice to have a way to add read/write transformations to pipes that doesn't require extending the sources and overriding methods.

    failOnEmptyLines

    When set to false, it just skips empty lines instead of failing the jobs. Defaults to true for backwards compatibility.

  3. type KeyedList[K, +V] = scalding.typed.KeyedList[K, V]

  4. class TypedJson[T <: AnyRef] extends FixedPathSource with TextSourceScheme with SingleMappable[T] with TypedSink[T]

  5. case class TypedJsonLzo[T <: AnyRef](p: String)(implicit evidence$3: Manifest[T]) extends TypedJson[T] with Product with Serializable

  6. type TypedPipe[+T] = scalding.typed.TypedPipe[T]

  7. type TypedSink[-T] = scalding.typed.TypedSink[T]

  8. type TypedSource[+T] = scalding.typed.TypedSource[T]

  9. type ValuePipe[+T] = scalding.typed.ValuePipe[T]

Value Members

  1. object JsonLine extends AbstractFunction5[String, Fields, SinkMode, Boolean, Boolean, JsonLine] with Serializable with Serializable

    TODO: at the next binary incompatible version remove the AbstractFunction2/scala.Serializable jank which was added to get mima to not report binary errors

  2. val TDsl: scalding.typed.TDsl.type

  3. object TypedJson extends Serializable

    This type uses the structural type of a case class, but not it's name, to describe the Json using json4s.

    This type uses the structural type of a case class, but not it's name, to describe the Json using json4s. This is intended to be used for intermediate output from a REPL session. The intended use is to save adhoc data between sessions. The fully qualified class name of classes defined in a REPL is not stable between REPL sessions.

    We believe using a fixed schema, such as thrift or Avro is a much safer way to do long term productionized data pipelines to minimize risks of incompatible changes to schema that render old data unreadable.

  4. val TypedPipe: scalding.typed.TypedPipe.type

  5. val scaldingVersion: String

Inherited from AnyRef

Inherited from Any

Ungrouped