Create a lexer for "raw" tokens.
Create a lexer for "raw" tokens.
rawTokenise
Convert the given Scala source code into a list of "raw" tokens.
Convert the given Scala source code into a list of "raw" tokens.
This includes whitespace and comment tokens. No NEWLINE or NEWLINES tokens are inferred. The final token will be of type EOF.
-- if true, no exceptions will be thrown when malformed tokens are encountered.
-- the version of Scala to assume as the source type (e.g. "2.9.1"). This can affect the interpretation of certain tokens (for example, floating point literals).
Convert the given Scala source code into a list of tokens.
Convert the given Scala source code into a list of tokens.
NEWLINE or NEWLINES tokens are inferred, and whitespace and comments are absorbed into the token they precede. The final token will be of type EOF.
-- if true, no exceptions will be thrown when malformed tokens are encountered.
-- the version of Scala to assume as the source type (e.g. "2.9.1"). This can affect the interpretation of certain tokens (for example, floating point literals).