ScriptParser

This is the parse entry point for code which is not self-contained, e.g. a script which is a series of template statements. They will be swaddled in Trees until the AST is equivalent to the one returned by compilationUnit().

Consume one token of the specified type, or signal an error if it is not there.

semi = nl {nl} | ;' nl =\n' // where allowed

AccessQualifier ::= "[" (id | this) "]"

Always add the syntactic mod, but check and conditionally add semantic mod.flags

Adjust start of annotation or constructor to offset of preceding @ or new

Annotation ::= `@' SimpleType1 {ParArgumentExprs}

AnnotType ::= SimpleType {Annotation}

ArgTypes ::= Type {,' Type} | NamedTypeArg {,' NamedTypeArg} NamedTypeArg ::= id `=' Type

ArgumentExprs ::= ParArgumentExprs | [nl] BlockExpr

ArgumentExprss ::= {ArgumentExprs}

ArgumentPatterns ::= ‘(’ [Patterns] ‘)’ | ‘(’ [Patterns ‘,’] PatVar ‘*’ ‘)’

If the last read offset is strictly greater than start, assign tree the span from start to last read offset, with given point. If the last offset is less than or equal to start, the tree t did not consume any source for its construction. In this case, don't assign a span yet, but wait for its span to be determined by setChildSpans when the parent node is positioned.

Positions tree. If t does not have a span yet, set its span to the given one.

Binding ::= (id | _') [:' Type]

Block ::= BlockStatSeq

BlockExpr ::= <<< (CaseClauses | Block) >>>

BlockStatSeq ::= { BlockStat semi } [Expr] BlockStat ::= Import | Annotations [implicit] [lazy] Def | Annotations LocalModifiers TmplDef | Extension | Expr1 |

Parse brace-enclosed body and rewrite it to be an indentation region instead, if possible. If possible means:

1. not inside (...), [...], case ... =>
2. opening brace { is at end of line
3. closing brace } is at start of line
4. there is at least one token between the braces
5. the closing brace is also at the end of the line, or it is followed by one of then, else, do, catch, finally, yield, or match.
6. the opening brace does not follow a =>. The reason for this condition is that rewriting back to braces does not work after => (since in most cases braces are omitted after a => it would be annoying if braces were inserted).

CaptureRef ::= ident | this

CaptureSet ::= { CaptureRef {, CaptureRef} } -- under captureChecking

CaseClause ::= ‘case’ Pattern [Guard] `=>' Block ExprCaseClause ::= ‘case’ Pattern [Guard] ‘=>’ Expr

CaseClauses ::= CaseClause {CaseClause} TypeCaseClauses ::= TypeCaseClause {TypeCaseClause}

[`extends' ConstrApps]

Check that this is not the start of a statement that's indented relative to the current region.

ClassConstr ::= [ClsTypeParamClause] [ConstrMods] ClsParamClauses

ClassDef ::= id ClassConstr TemplateOpt

The region to eliminate when replacing a closing ) or } that starts a new line The ) or } precedes in.lastOffset.

Expr ::= [‘implicit’] FunParams =>' Expr BlockResult ::= implicit id [:' InfixType] `=>' Block // Scala2 only

{, }

{, }

currentRegion.commasExpected has to be set separately.

CompilationUnit ::= {package QualId semi} TopStatSeq

ConstrApps ::= ConstrApp ({‘,’ ConstrApp} | {‘with’ ConstrApp})

ConstrMods ::= {Annotation} [AccessModifier]

ContextTypes ::= FunArgType {‘,’ FunArgType}

Convert tree to formal parameter

Convert tree to formal parameter list

Convert (qual)ident to type identifier

DefDef ::= DefSig [‘:’ Type] ‘=’ Expr | this ParamClause ParamClauses =' ConstrExpr DefDcl ::= DefSig:' Type DefSig ::= id [DefTypeParamClause] DefParamClauses | ExtParamClause [nl] [‘.’] id DefParamClauses

Def ::= val PatDef | var VarDef | def DefDef | type {nl} TypeDcl | TmplDef Dcl ::= val ValDcl | var ValDcl | def DefDcl | type {nl} TypeDcl EnumCase ::= case' (id ClassConstr [extends' ConstrApps]] | ids)

DotSelectors ::= { `.' id }

Drop (...) or { ... }, replacing the closing element with endStr

Drop current token, if it is a then or do.

EnumCase = case' (id ClassConstr [extends' ConstrApps] | ids)

EnumDef ::= id ClassConstr InheritClauses EnumBody

Enumerator ::= Generator | Guard {Guard} | Pattern1 `=' Expr

Enumerators ::= Generator {semi Enumerator | Guard}

in.offset, except if this is at a new line, in which case lastOffset is preferred.

ExprsInParens ::= ExprInParens {,' ExprInParens} Bindings ::= Binding {,' Binding}

ExtMethod ::= {Annotation [nl]} {Modifier} ‘def’ DefDef | Export

ExtMethods ::= ExtMethod | [nl] ‘{’ ExtMethod {semi ExtMethod ‘}’

Extension ::= ‘extension’ [DefTypeParamClause] {UsingParamClause} ‘(’ DefParam ‘)’ {UsingParamClause} ExtMethods

The position of the first XML literal encountered while parsing, NoSourcePosition if there were no XML literals.

In the tokens following the current one, does query precede any of the tokens that

• must start a statement, or
• separate two statements, or
• continue a statement (e.g. else, catch`), or
• terminate the current scope?

Under captureChecking language import: is the following token sequence a capture set {ref1, ..., refN} followed by a token that can start a type?

Is the following sequence the generators of a for-expression enclosed in (...)?

Are the next token the "GivenSig" part of a given definition, i.e. an identifier followed by type and value parameters, followed by :?

Is the token sequence following the current : token classified as a lambda? This is the case if the input starts with an identifier, a wildcard, or something enclosed in (...) or [...], and this is followed by a => or ?=> and an INDENT.

When encountering a :, is that in the binding of a lambda?

Is current ident a *, and is it followed by a ), , ), ,EOF? The latter two are not syntactically valid, but we need to include them here for error recovery.

ForExpr ::= ‘for’ ‘(’ Enumerators ‘)’ {nl} [‘do‘ | ‘yield’] Expr | ‘for’ ‘{’ Enumerators ‘}’ {nl} [‘do‘ | ‘yield’] Expr | ‘for’ Enumerators (‘do‘ | ‘yield’) Expr

FunParamClause ::= ‘(’ TypedFunParam {‘,’ TypedFunParam } ‘)’

FunParams ::= Bindings | id | _' Bindings ::=(' [[‘erased’] Binding {,' Binding}])'

Generator ::= [‘case’] Pattern `<-' Expr

GivenDef ::= [GivenSig] (AnnotType [‘=’ Expr] | StructuralInstance) GivenSig ::= [id] [DefTypeParamClause] {UsingParamClauses} ‘:’

Guard ::= if PostfixExpr

Accept identifier and return its name as a term name.

if'(' Expr )' {nl} Expr [[semi] else Expr]if' Expr `then' Expr [[semi] else Expr]

ImportExpr ::= SimpleRef {‘.’ id} ‘.’ ImportSpec | SimpleRef ‘as’ id ImportSpec ::= NamedSelector | WildcardSelector | ‘{’ ImportSelectors ‘}’ ImportSelectors ::= NamedSelector [‘,’ ImportSelectors] | WildCardSelector {‘,’ WildCardSelector} NamedSelector ::= id [‘as’ (id | ‘_’)] WildCardSelector ::= ‘*' | ‘given’ [InfixType]

Import ::= import' ImportExpr {‘,’ ImportExpr} Export ::=export' ImportExpr {‘,’ ImportExpr}

Issue an error at current offset that input is incomplete

Parse indentation region body and rewrite it to be in braces instead

operand { infixop operand | MatchClause } [postfixop],

respecting rules of associativity and precedence.

InfixPattern ::= SimplePattern {id [nl] SimplePattern}

InfixType ::= RefinedType {id [nl] RefinedType}

True if we are seeing a lambda argument after a colon of the form: : (params) => body

Is offset1 less or equally indented than offset2? This is the case if the characters between the preceding end-of-line and offset1 are a prefix of the characters between the preceding end-of-line and offset2.

Is current token a hard or soft modifier (in modifier position or not)?

A '$' identifier is treated as a splice if followed by a {. A longer identifier starting with $ is treated as a splice/id combination in a quoted block '{...'

Literal ::= SimpleLiteral | processedStringLiteral | symbolLiteral | ‘null’

Create a tree representing a packaging

MatchClause ::= match'{' CaseClauses `}'

match'{' TypeCaseClauses `}'

MixinQualifier ::= [' id]'

Create an import node and handle source version imports

{Annotation} {Modifier} Modifiers ::= {Modifier} LocalModifiers ::= {LocalModifier} AccessModifier ::= (private | protected) [AccessQualifier] Modifier ::= LocalModifier | AccessModifier | override | opaque LocalModifier ::= abstract | final | sealed | open | implicit | lazy | erased | inline | transparent

SimpleExpr ::= ‘new’ ConstrApp {with ConstrApp} [TemplateBody] | ‘new’ TemplateBody

ObjectDef ::= id TemplateOpt

Packaging ::= package QualId [nl] {' TopStatSeq}'

ParArgumentExprs ::= (' [‘using’] [ExprsInParens])' | (' [ExprsInParens,'] PostfixExpr `*' ')'

ParArgumentExprss ::= {ParArgumentExprs}

Special treatment for arguments to primary constructor annotations. (...) is considered an argument only if it does not look like a formal parameter list, i.e. does not start with ( <annot>* <mod>* ident : Furthermore, () is considered a annotation argument only if it comes first.

ClsParamClause ::= ‘(’ [‘erased’] ClsParams ‘)’ | UsingClsParamClause UsingClsParamClause::= ‘(’ ‘using’ [‘erased’] (ClsParams | ContextTypes) ‘)’ ClsParams ::= ClsParam {‘,’ ClsParam} ClsParam ::= {Annotation}

DefParamClause ::= ‘(’ [‘erased’] DefParams ‘)’ | UsingParamClause UsingParamClause ::= ‘(’ ‘using’ [‘erased’] (DefParams | ContextTypes) ‘)’ DefParams ::= DefParam {‘,’ DefParam} DefParam ::= {Annotation} [‘inline’] Param

Param ::= id :' ParamType [=' Expr]

ClsParamClauses ::= {ClsParamClause} [[nl] ‘(’ [‘implicit’] ClsParams ‘)’] DefParamClauses ::= {DefParamClause} [[nl] ‘(’ [‘implicit’] DefParams ‘)’]

ParamType ::= ParamValueType | =>' ParamValueType | [CaptureSet]->' ParamValueType

ParamValueType ::= Type [`*']

PatDef ::= ids [‘:’ Type] ‘=’ Expr | Pattern2 [‘:’ Type] ‘=’ Expr VarDef ::= PatDef | id {,' id}:' Type ='_' (deprecated in 3.x) ValDcl ::= id {,' id}:' Type VarDcl ::= id {,' id}:' Type

Pattern ::= Pattern1 { `|' Pattern1 }

Pattern1 ::= Pattern2 [Ascription]

Pattern3 ::= InfixPattern | PatVar ‘*’

Patterns ::= Pattern [`,' Pattern]

PostfixExpr ::= InfixExpr [id [nl]] InfixExpr ::= PrefixExpr | InfixExpr id [nl] InfixExpr | InfixExpr id : IndentedExpr | InfixExpr MatchClause

QualId ::= id {`.' id}

RefineStatSeq ::= RefineStat {semi RefineStat} RefineStat ::= ‘val’ VarDcl | ‘def’ DefDcl | ‘type’ {nl} TypeDcl (in reality we admit Defs and vars and filter them out afterwards in checkLegal)

Refinement ::= {' RefineStatSeq}'

rewrite code with (...) around the source code of t

Accept identifier or match clause acting as a selector on given tree t

SelfInvocation ::= this ArgumentExprs {ArgumentExprs}

SelfType ::= id [‘:’ [CaptureSet] InfixType] ‘=>’ | ‘this’ ‘:’ [CaptureSet] InfixType ‘=>’

SimpleExpr ::= ‘new’ ConstrApp {with ConstrApp} [TemplateBody] | ‘new’ TemplateBody | BlockExpr | ExprSplice | Quoted | quoteId | SimpleExpr1 [_] SimpleExpr1 ::= literal | xmlLiteral | SimpleRef | ( [ExprsInParens] ) | SimpleExpr . id | SimpleExpr . MatchClause | SimpleExpr (TypeArgs | NamedTypeArgs) | SimpleExpr1 ArgumentExprs | SimpleExpr1 : ColonArgument -- under language.experimental.fewerBraces ColonArgument ::= indent (CaseClauses | Block) outdent | FunParams (‘=>’ | ‘?=>’) ColonArgBody | HkTypeParamClause ‘=>’ ColonArgBody ColonArgBody ::= indent (CaseClauses | Block) outdent Quoted ::= ‘'’ ‘{’ Block ‘}’ | ‘'’ ‘[’ Type ‘]’

SimpleLiteral ::= [‘-’] integerLiteral | [‘-’] floatingPointLiteral | booleanLiteral | characterLiteral | stringLiteral

SimplePattern ::= PatVar | Literal | Quoted | XmlPattern | (' [Patterns])' | SimplePattern1 [TypeArgs] [ArgumentPatterns] SimplePattern1 ::= SimpleRef | SimplePattern1 .' id PatVar ::= id |_'

SimpleRef ::= id | [id ‘.’] ‘this’ | [id ‘.’] ‘super’ [ClassQualifier] ‘.’ id

SimpleType ::= SimpleLiteral | ‘?’ SubtypeBounds | SimpleType1 | SimpleType ‘(’ Singletons ‘)’ -- under language.experimental.dependent, checked in Typer Singletons ::= Singleton {‘,’ Singleton}

SimpleType1 ::= id | Singleton .' id | Singleton.' type | ‘(’ ArgTypes ‘)’ | Refinement | TypeSplice -- deprecated syntax (since 3.0.0) | SimpleType1 TypeArgs | SimpleType1 `#' id

Singleton ::= SimpleRef | SimpleLiteral | Singleton ‘.’ id -- not yet | Singleton ‘(’ Singletons ‘)’ -- not yet | Singleton ‘[’ Types ‘]’

Skip on error to next safe point.

ExprSplice ::= ‘$’ spliceId -- if inside quoted block | ‘$’ ‘{’ Block ‘}’ -- unless inside quoted pattern | ‘$’ ‘{’ Pattern ‘}’ -- when inside quoted pattern TypeSplice ::= ‘$’ spliceId -- if inside quoted type | ‘$’ ‘{’ Block ‘}’ -- unless inside quoted type pattern | ‘$’ ‘{’ Pattern ‘}’ -- when inside quoted type pattern

The block in a quote or splice

The region to eliminate when replacing an opening ( or { that ends a line. The ( or { is at in.offset.

Parse statement separators and end markers. Ensure that there is at least one statement separator unless the next token terminates a statement´sequence.

Parse body while checking (under -no-indent) that a { is not missing before it. This is done as follows: If the next token S is indented relative to the current region, and the end of body is followed by a new line and another statement, check that that other statement is indented less than S

Unconditionally issue an error at given span, without updating lastErrorOffset.

Issue an error at given offset if beyond last error offset and update lastErrorOffset.

If at end of file, issue an incompleteInputError. Otherwise issue a syntax error and skip to next safe point.

Template ::= InheritClauses [TemplateBody] InheritClauses ::= [‘extends’ ConstrApps] [‘derives’ QualId {‘,’ QualId}]

TemplateBody ::= [nl] {' TemplateStatSeq}' EnumBody ::= [nl] ‘{’ [SelfType] EnumStat {semi EnumStat} ‘}’

TemplateOpt = [Template]

TemplateStatSeq ::= [SelfType] TemplateStat {semi TemplateStat} TemplateStat ::= Import | Export | Annotations Modifiers Def | Annotations Modifiers Dcl | Extension | Expr1 | EnumStat ::= TemplateStat | Annotations Modifiers EnumCase

Accept identifier and return Ident with its name as a term name.

TmplDef ::= ([‘case’] ‘class’ | ‘trait’) ClassDef | [‘case’] ‘object’ ObjectDef | ‘enum’ EnumDef | ‘given’ GivenDef

Does the current conditional expression continue after the initially parsed (...) region?

TopStatSeq ::= TopStat {semi TopStat} TopStat ::= Import | Export | Annotations Modifiers Def | Packaging | package object objectDef | Extension |

Same as typ, but if this results in a wildcard it emits a syntax error and returns a tree for type Any instead.

Type ::= FunType | HkTypeParamClause ‘=>>’ Type | FunParamClause ‘=>>’ Type | MatchType | InfixType | CaptureSet Type -- under captureChecking FunType ::= (MonoFunType | PolyFunType) MonoFunType ::= FunTypeArgs (‘=>’ | ‘?=>’) Type | (‘->’ | ‘?->’ ) Type -- under pureFunctions PolyFunType ::= HKTypeParamClause '=>' Type | HKTypeParamClause ‘->’ Type -- under pureFunctions FunTypeArgs ::= InfixType | (' [ [ ‘[using]’ ‘['erased'] FunArgType {,' FunArgType } ] `)' | '(' [ ‘[using]’ ‘['erased'] TypedFunParam {',' TypedFunParam } ')'

TypeArgs ::= [' Type {,' Type} ]' NamedTypeArgs ::=[' NamedTypeArg {,' NamedTypeArg}]'

TypeBounds ::= [>:' Type] [<:' Type]

TypeCaseClause ::= ‘case’ (InfixType | ‘_’) ‘=>’ Type [semi]

TypeDcl ::= id [TypeParamClause] {FunParamClause} TypeBounds [‘=’ Type]

Accept identifier and return Ident with its name as a type name.

TypeParamBounds ::= TypeBounds {<%' Type} {:' Type}

ClsTypeParamClause::= ‘[’ ClsTypeParam {‘,’ ClsTypeParam} ‘]’ ClsTypeParam ::= {Annotation} [‘+’ | ‘-’] id [HkTypeParamClause] TypeParamBounds

DefTypeParamClause::= ‘[’ DefTypeParam {‘,’ DefTypeParam} ‘]’ DefTypeParam ::= {Annotation} id [HkTypeParamClause] TypeParamBounds

TypTypeParamClause::= ‘[’ TypTypeParam {‘,’ TypTypeParam} ‘]’ TypTypeParam ::= {Annotation} id [HkTypePamClause] TypeBounds

HkTypeParamClause ::= ‘[’ HkTypeParam {‘,’ HkTypeParam} ‘]’ HkTypeParam ::= {Annotation} [‘+’ | ‘-’] (id [HkTypePamClause] | ‘_’) TypeBounds

TypedFunParam ::= id ':' Type

If all other characters on the same line as span are blanks, widen to the whole line.

{withConstrApp} but no EOL allowed afterwith`.

with Template, with EOL interpreted

WithType ::= AnnotType {`with' AnnotType} (deprecated)

Wrap annotation or constructor in New(...).

ConstrApp ::= SimpleType1 {Annotation} {ParArgumentExprs}

ConstrExpr ::= SelfInvocation | {' SelfInvocation {semi BlockStat}}'

Expr ::= [implicit'] FunParams (‘=>’ | ‘?=>’) Expr | HkTypeParamClause ‘=>’ Expr | Expr1 FunParams ::= Bindings | id |' ExprInParens ::= PostfixExpr :' Type | Expr BlockResult ::= [‘implicit’] FunParams (‘=>’ | ‘?=>’) Block | HkTypeParamClause ‘=>’ Block | Expr1 Expr1 ::= [‘inline’]if' (' Expr)' {nl} Expr [[semi] else Expr] | [‘inline’] if' Exprthen' Expr [[semi] else Expr] | while'(' Expr )' {nl} Expr |while' Expr do' Expr |try' Expr Catches [finally' Expr] |try' Expr [finally' Expr] |throw' Expr | return' [Expr] | ForExpr | [SimpleExpr.'] id =' Expr | PrefixOperator SimpleExpr=' Expr | SimpleExpr1 ArgumentExprs =' Expr | PostfixExpr [Ascription] | ‘inline’ InfixExpr MatchClause Bindings ::=(' [Binding {,' Binding}])' Binding ::= (id | _') [:' Type] Ascription ::= :' [CaptureSet] InfixType |:' Annotation {Annotation} | :'' `*' Catches ::= ‘catch’ (Expr | ExprCaseClause)

FunArgType ::= Type | =>' Type | [CaptureSet]->' Type

The offset where the last syntax error was reported, or if a skip to a safepoint occurred afterwards, the offset of the safe point.

Pattern2 ::= [id `@'] Pattern3

The implicit parameters introduced by _ in the current expression. Parameters appear in reverse order.