JavaScanner

convert name, base to double value

convert name to long value

Generate an error at the given offset

Clear buffer and set name and token If target is different from this, don't treat identifiers as end tokens

signal an error where the input ended in the middle of a token

Is current token first one after a newline?

Is token a COLON, after having converted COLONEOL to COLON? The conversion means that indentation is not significant after : anymore. So, warning: this is a side-effecting operation.

Is last character a unicode escape \uxxxx?

A new reader that takes off at the current character position

Advance one character; reducing CR;LF pairs to just LF

Advance one character, leaving CR;LF pairs intact. This is for use in multi-line strings, so there are no "potential line ends" here.

append Unicode character to "litBuf" buffer

Clear buffer and set string

we need one token lookahead

the base of a number

the last read character

The offset one past the last read character

the last error offset

The offset before the last read character

the offset of the character following the token preceding this one

the offset of the newline immediately preceding the token, or -1 if token is not preceded by a newline.

The start offset of the current line

A character buffer for literals

the name of an identifier

the offset of the first character of the current token

the string value of a literal

the next token