p

planet7

tabular

package tabular

Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. tabular
  2. PrettyPrinters
  3. DataSinkImplicits
  4. DataSourceImplicits
  5. MappingBuilders
  6. AnyRef
  7. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Type Members

  1. class BufferedDataSource extends TabularDataSource
  2. class ColumnDoesNotExistException extends RuntimeException
  3. case class Csv (header: Row, dataRows: Iterator[Row]) extends Iterable[Row] with Product with Serializable

    Csv encapsulates a tabular data structure, as found in a CSV file or spreadsheet.

    Csv encapsulates a tabular data structure, as found in a CSV file or spreadsheet. It allows client code to:

    • read CSV data from a tabular data-source (String, File, InputStream, or anything else)
    • Change the column structure (add, remove, rename and reorder columns)
    • Map data values according to a function (e.g. a lookup, a data-conversion, a default value)
    • Apply fail-fast or reporting-only validations to each row
    • Export the results

    This class may be useful for:

    • ETL of externally-generated data
    • Pre-formatting data before Diffing it during a reconciliation. @see Diff

    Csv is designed to be lazy. The datasource is traversed ONCE, when accessed via the Csv.iterator method. All other operations are declarative (restructuring, data transformations, validations, etc). Thus, the time cost of traversing the data and applying these functions is only paid once, when the client code materialises the Csv.

    TODO - CAS - 07/08/2014 - A Y-shaped pipeline (spits out two CSVs) TODO - CAS - 07/08/2014 - Aggregator 1 - merge columns using a Row => Row (e.g. (firstName, surname) -> s"$firstName $surname") TODO - CAS - 07/08/2014 - Aggregator 2 - merge rows - provide a predicate for row grouping/inclusion/exclusion

  4. class CsvPrinter extends AnyRef
  5. trait DataSinkImplicits extends AnyRef
  6. trait DataSourceImplicits extends AnyRef
  7. implicit class AppendableDataSource extends AnyRef
    Definition Classes
    DataSourceImplicits
  8. case class DefaultParser (delimiter: Char) extends Parser with Product with Serializable
  9. trait Differentiator [U] extends AnyRef
  10. class LineReader extends Iterator[Row]
  11. trait MappingBuilders extends AnyRef
  12. case class Conv [T](f: (String) ⇒ T) extends Product with Serializable
    Definition Classes
    MappingBuilders
    Annotations
    @implicitNotFound( ... )
  13. class NoDataInSourceException extends RuntimeException
  14. trait Parser extends AnyRef
  15. trait PrettyPrinters extends AnyRef
  16. class RegexTwoPassParser extends Parser

    Simple parser with common defaults:

    Simple parser with common defaults:

    • Quoted strings are preserved, even with embedded delimiters, e.g.: foo, "one, monkey, two", bar ===> Array("foo", "one, monkey, two", "bar")
    • spaces around elements are ignored (unless quoted) foo , bar , " monkey one ", baz ===> Array("foo", "bar", " monkey one ", "baz")
  17. case class Row (data: Array[String], validationFailures: Seq[String] = Nil) extends Iterable[String] with Product with Serializable
  18. case class RowDiffer (header: Row, fieldComps: (String, Comparator[String])*) extends Differentiator[Row] with Product with Serializable
  19. class RowPrinter extends AnyRef
  20. class ScannerDataSource extends TabularDataSource
  21. trait TabularDataSource extends Closeable

    A way of providing data to an instance of the Csv class.

    A way of providing data to an instance of the Csv class. Most needs should be met by BufferedDataSource, which has a number of implicit adapters declared in the planet7 package.

  22. class ValidationFailedException extends RuntimeException

Value Members

  1. def by[K](f: (String) ⇒ K)(implicit arg0: Ordering[K]): Ordering[String]

    Used in sort(input, "Age" -> by(_.toInt))

  2. def combine(datasources: TabularDataSource*): TabularDataSource
    Definition Classes
    DataSourceImplicits
  3. def defaultTo(other: String): (String) ⇒ String
  4. def experimentalFromMemoryMappedFile(f: File): TabularDataSource
    Definition Classes
    DataSourceImplicits
  5. def experimentalFromScanner(f: File): TabularDataSource
    Definition Classes
    DataSourceImplicits
  6. def experimentalFromWholeFile(f: File): TabularDataSource
    Definition Classes
    DataSourceImplicits
  7. def export(csv: Csv, parser: Parser = Parsers.basic): String
  8. implicit def fromColumnStructure(s: Seq[(String, String)]): Array[String]

    Converts Seq("foo" -> "foo", "bar" -> "bar") to Seq("foo", "bar"), to make building header rows easier

  9. def fromFile(f: File, parser: Parser): TabularDataSource
    Definition Classes
    DataSourceImplicits
  10. implicit def fromFile(f: File): TabularDataSource
    Definition Classes
    DataSourceImplicits
  11. def fromInputStream(is: InputStream, parser: Parser): TabularDataSource
    Definition Classes
    DataSourceImplicits
  12. implicit def fromInputStream(is: InputStream): TabularDataSource
    Definition Classes
    DataSourceImplicits
  13. implicit def fromIterable(it: Iterable[String], parser: Parser): TabularDataSource
    Definition Classes
    DataSourceImplicits
  14. implicit def fromIterable(it: Iterable[String]): TabularDataSource
    Definition Classes
    DataSourceImplicits
  15. def fromString(s: String, parser: Parser): TabularDataSource
    Definition Classes
    DataSourceImplicits
  16. implicit def fromString(s: String): TabularDataSource
    Definition Classes
    DataSourceImplicits
  17. def given[T1, T2, T3](colMod1: String, colMod2: String, colMod3: String): Nothing
    Definition Classes
    MappingBuilders
  18. def given[T1, T2](col1: String, col2: String)(pf: PartialFunction[(T1, T2), String])(implicit t1Conv: Conv[T1], t2Conv: Conv[T2]): (String) ⇒ (Row) ⇒ (Row) ⇒ Row
    Definition Classes
    MappingBuilders
  19. def given[T1](col1: String): Nothing
    Definition Classes
    MappingBuilders
  20. def ignore(columnNames: String*): (Array[String]) ⇒ Array[String]
  21. def showDiffs(left: Row, right: Row): String
    Definition Classes
    PrettyPrinters
  22. def sort(csv: Csv, differ: RowDiffer): Csv
  23. def sort(csv: Csv, fieldComps: (String, Comparator[String])*): Csv
  24. implicit val toBD: Conv[BigDecimal]
    Definition Classes
    MappingBuilders
  25. implicit def toColumnStructure(s: Seq[String]): Seq[(String, String)]

    Converts Seq("foo", "bar") to Seq("foo" -> "foo", "bar" -> "bar") to make operations on column names easier

  26. implicit def toColumnStructure(s: String): (String, String)

    Converts Csv(data).columnStructure("Name") to Csv(data).columnStructure("Name" -> "Name")

  27. implicit val toInt: Conv[Int]
    Definition Classes
    MappingBuilders
  28. implicit def toRowTransformer(mapping: (String, (String) ⇒ (Row) ⇒ (Row) ⇒ Row)): (Row) ⇒ (Row) ⇒ Row
    Definition Classes
    MappingBuilders
  29. implicit def toStringCompare(s: String): (String, Comparator[String])

    Converts sort(input, "Surname") into sort(input, "Surname" -> Comparator[String])

  30. implicit def toValidation(columnAssertion: (String, (String) ⇒ Boolean)): (Row) ⇒ (Row) ⇒ Row

    Used in Cvs.assertAndAbort() and Csv.assertAndReport().

    Used in Cvs.assertAndAbort() and Csv.assertAndReport(). Converts , to build simple validation checks

  31. def top5(csv: Csv): String
    Definition Classes
    PrettyPrinters
  32. def write(csv: Csv, path: String, parser: Parser = Parsers.basic): Unit
    Definition Classes
    DataSinkImplicits
  33. object Csv extends Serializable
  34. object EmptyRow extends Row
  35. object FieldDiffer extends Differentiator[(String, String)] with Product with Serializable
  36. object X
    Definition Classes
    MappingBuilders
  37. object NaiveRowDiffer extends Differentiator[Row]
  38. object NaiveRowOrdering extends Ordering[Row]
  39. object Parsers
  40. object RegexTwoPassParser
  41. object Row extends Serializable
  42. object StringDiffer extends Differentiator[String] with Product with Serializable
  43. object Validations

Inherited from PrettyPrinters

Inherited from DataSinkImplicits

Inherited from DataSourceImplicits

Inherited from MappingBuilders

Inherited from AnyRef

Inherited from Any

Ungrouped