colossus.parsing

Combinators

object Combinators

Streaming Parser Combinators

Overview

A Parser[T] is an object that consumes a stream of bytes to produce a result of type T.

A Combinator is a "higher-order" parser that takes one or more parsers to produce a new parser

The Stream parsers are very fast and efficient, but because of this they need to make some tradeoffs. They are mutable, not thread safe, and in general are designed for network protocols, which tend to have very deterministic grammars.

The Parser Rules:

1. A parser must greedily consume the data stream until it produces a result 2. When a parser consumes the last byte necessary to produce a result, it must stop consuming the stream and return the new result while resetting its state

Examples

Use any parser by itself:

val parser = bytes(4)
val data = DataBuffer(ByteString("aaaabbbbccc")
parser.parse(data) // Some(ByteString(97, 97, 97, 97))
parser.parse(data) >> {bytes => bytes.utf8String} // Some("bbbb")
parser.parse(data) // None

Combine two parsers

val parser = bytes(3) ~ bytes(2) >> {case a ~ b => a.ut8String + ":" + b.utf8String}
parser.parse(DataBuffer(ByteString("abc"))) // None
parser.parse(DataBuffer(ByteString("defgh"))) // Some("abc:de")
Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. Combinators
  2. AnyRef
  3. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Type Members

  1. implicit final class ByteArrayOps extends AnyVal

  2. class ChainedParser[A, B] extends Parser[~[A, B]]

  3. class FastArrayBuilder extends FastArrayBuilding

  4. trait FastArrayBuilding extends AnyRef

    A very fast dynamically growable array builder.

  5. class FlatMapParser[A, B] extends Parser[B]

  6. class FoldZeroParser[T, U] extends Parser[U]

  7. class LineParser[T] extends Parser[T] with FastArrayBuilding

    Parse a single line of data.

  8. class MapParser[A, B] extends Parser[B]

  9. trait Parser[+T] extends AnyRef

  10. class RepeatZeroParser[T] extends Parser[Array[T]]

    Repeat a parser, accumulating the results until the value returned by the parser matches the type's Zero value.

  11. case class ~[+A, +B](a: A, b: B) extends Product with Serializable

Value Members

  1. final def !=(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  2. final def !=(arg0: Any): Boolean

    Definition Classes
    Any

  3. final def ##(): Int

    Definition Classes
    AnyRef → Any

  4. final def ==(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  5. final def ==(arg0: Any): Boolean

    Definition Classes
    Any

  6. final def asInstanceOf[T0]: T0

    Definition Classes
    Any

  7. val byte: Parser[Byte]

    parse a single byte

  8. def bytes(num: Int): Parser[Array[Byte]]

  9. def bytes(num: Parser[Int]): Parser[Array[Byte]]

  10. def bytes(num: Int, maxSize: DataSize, maxInitBufferSize: DataSize): Parser[Array[Byte]]

  11. def bytes(num: Parser[Int], maxSize: DataSize, maxInitBufferSize: DataSize): Parser[Array[Byte]]

    read a fixed number bytes, prefixed by a length

  12. def bytesUntil(terminus: Array[Byte], includeTerminusInData: Boolean = false, sizeHint: Int = 32): Parser[Array[Byte]]

    Keep reading bytes until the terminus is encounted.

    Keep reading bytes until the terminus is encounted. This accounts for possible partial terminus in the data. The terminus is NOT included in the returned value

  13. def bytesUntilEOS: Parser[ByteString]

    Read in an unknown number of bytes, ended only when endOfStream is called

    Read in an unknown number of bytes, ended only when endOfStream is called

    be aware this parser has no max size and will read in data forever if endOfStream is never called

  14. def clone(): AnyRef

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  15. def const[T](t: T): Parser[T]

    Creates a parser that will always return the same value without consuming any data.

    Creates a parser that will always return the same value without consuming any data. Useful when flatMapping parsers

  16. def delimitedString(delimiter: Byte, terminus: Byte): Parser[Vector[String]]

    Parse a series of ascii strings separated by a single-byte delimiter and terminated by a byte

  17. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  18. def equals(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any

  19. def finalize(): Unit

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )

  20. def foldZero[T, U](parser: Parser[T], init: ⇒ U)(folder: (T, U) ⇒ U)(implicit zero: Zero[T]): FoldZeroParser[T, U]

  21. final def getClass(): Class[_]

    Definition Classes
    AnyRef → Any

  22. def hashCode(): Int

    Definition Classes
    AnyRef → Any

  23. def int: Parser[Int]

  24. def intUntil(terminus: Byte, base: Int = 10): Parser[Long]

    Parses the ASCII representation of an integer, keeps going until the terminus is encountered

  25. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any

  26. def line[T](constructor: (Array[Byte]) ⇒ T, includeNewLine: Boolean): Parser[T]

  27. def line(includeNewline: Boolean): Parser[Array[Byte]]

  28. def line: Parser[Array[Byte]]

  29. def literal(lit: ByteString): Parser[ByteString]

  30. def long: Parser[Long]

  31. def maxSize[T](size: DataSize, parser: Parser[T]): Parser[T]

    Creates a parser that wraps another parser and will throw an exception if more than size data is required to parse a single object.

    Creates a parser that wraps another parser and will throw an exception if more than size data is required to parse a single object. See the ParserSizeTracker for more details.

  32. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  33. final def notify(): Unit

    Definition Classes
    AnyRef

  34. final def notifyAll(): Unit

    Definition Classes
    AnyRef

  35. def repeat[T](times: Long, parser: Parser[T]): Parser[Vector[T]]

    Repeat a pattern a fixed number of times

    Repeat a pattern a fixed number of times

    times

    the number of times to parse the pattern

    parser

    the parser for the pattern

    returns

    the parsed sequence

  36. def repeat[T](times: Parser[Long], parser: Parser[T]): Parser[Vector[T]]

    Parse a pattern multiple times based on a numeric prefix

    Parse a pattern multiple times based on a numeric prefix

    This is useful for any situation where the repeated pattern is prefixed by the number of repetitions, for example num:[obj1][obj2][obj3]. In situations where the pattern doesn't immediately follow the number, you'll have to do it yourself, something like 

    intUntil(':') ~ otherParser |> {case num ~ other => repeat(num, patternParser)

    }

    intUntil(':') ~ otherParser |> {case num ~ other => repeat(num, patternParser) }}}

    times

    parser for the number of times to repeat the pattern

    parser

    the parser that will parse a single instance of the pattern

    returns

    the parsed sequence

  37. def repeatUntil[T](parser: Parser[T], terminus: Byte): Parser[Vector[T]]

    Repeatedly parse a pattern until a terminal byte is reached

    Repeatedly parse a pattern until a terminal byte is reached

    Before calling parser this will examine the next byte. If the byte matches the terminus, it will return the built sequence. Otherwise it will pass control to parser (including the examined byte) until the parser returns a result.

    Notice that the terminal byte is consumed, so if we have 

    val parser = repeatUntil(bytes(2), ':')
parser.parse(DataBuffer(ByteString("aabbcc:ddee")))

    the bytes remaining in the buffer after parsing are just ddee.

    parser

    the parser repeat

    terminus

    the byte to singal to stop repeating

    returns

    the parsed sequence

  38. def repeatUntilEOS[T](parser: Parser[T]): Parser[Seq[T]]

    Create a parser that will repeat the given parser forever until endOfStream() is called.

    Create a parser that will repeat the given parser forever until endOfStream() is called. The results from each call to the given parser are accumulated and returned at the end of the stream.

  39. def repeatZero[T](parser: Parser[T])(implicit arg0: ClassTag[T], zero: Zero[T]): RepeatZeroParser[T]

    Repeat using a parser until it returns a zero value.

    Repeat using a parser until it returns a zero value. An array of non-zero values is returned

  40. def short: Parser[Short]

  41. def skip[T](n: Int): Parser[Unit]

    creates a parser that will skip over n bytes.

    creates a parser that will skip over n bytes. You generally only want to do this inside a peek parser

  42. def stringUntil(terminus: Byte, toLower: Boolean = false, minSize: Option[Int] = None, allowWhiteSpace: Boolean = true, ltrim: Boolean = false): Parser[String]

    Parse a string until a designated byte is encountered

    Parse a string until a designated byte is encountered

    Limited filtering is currently supported, all of which happens during the reading.

    terminus

    reading will stop when this byte is encountered

    toLower

    if true any characters in the range A-Z will be lowercased before insertion

    minSize

    specify a minimum size

    allowWhiteSpace

    throw a ParseException if any whitespace is encountered before the terminus. If the terminus is a whitespace character, it will not be counted

    ltrim

    trim leading whitespace

  43. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef

  44. def toString(): String

    Definition Classes
    AnyRef → Any

  45. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  46. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  47. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from AnyRef

Inherited from Any

Ungrouped