LexingCommon (JRuby Core 9.2.5.0 API)

java.lang.Object
- org.jruby.lexer.LexingCommon

Direct Known Subclasses:

RipperLexer, RubyLexer
```
public abstract class LexingCommon
extends java.lang.Object
```
Code and constants common to both ripper and main parser.

Field Summary

Fields
Modifier and Type	Field and Description
`protected boolean`	`__end__seen`
`protected ByteList`	`AMPERSAND`
`static ByteList`	`AMPERSAND_AMPERSAND`
`ByteList`	`AMPERSAND_DOT`
`static org.jcodings.Encoding`	`ASCII8BIT_ENCODING`
`protected ByteList`	`BACKSLASH`
`ByteList`	`BACKTICK`
`ByteList`	`BANG`
`protected ByteList`	`BANG_EQ`
`protected ByteList`	`BANG_TILDE`
`static ByteList`	`BEGIN_DOC_MARKER`
`protected int`	`braceNest`
`static ByteList`	`CALL`
`protected ByteList`	`CARET`
`protected StackState`	`cmdArgumentState`
`static ByteList`	`CODING`
`protected ByteList`	`COLON`
`protected ByteList`	`COLON_COLON`
`protected ByteList`	`COMMA`
`boolean`	`commandStart`
`protected StackState`	`conditionState`
`static ByteList`	`DOLLAR_BANG`
`static ByteList`	`DOLLAR_DOT`
`static ByteList`	`DOLLAR_UNDERSCORE`
`ByteList`	`DOT`
`protected ByteList`	`DOT_DOT`
`protected ByteList`	`DOT_DOT_DOT`
`static ByteList`	`END_DOC_MARKER`
`static ByteList`	`END_MARKER`
`static int`	`EOF`
`boolean`	`eofp`
`protected ByteList`	`EQ`
`ByteList`	`EQ_EQ`
`ByteList`	`EQ_EQ_EQ`
`protected ByteList`	`EQ_GT`
`protected ByteList`	`EQ_TILDE`
`static int`	`EXPR_ARG`
`static int`	`EXPR_ARG_ANY`
`static int`	`EXPR_BEG`
`static int`	`EXPR_BEG_ANY`
`static int`	`EXPR_CLASS`
`static int`	`EXPR_CMDARG`
`static int`	`EXPR_DOT`
`static int`	`EXPR_END`
`static int`	`EXPR_END_ANY`
`static int`	`EXPR_ENDARG`
`static int`	`EXPR_ENDFN`
`static int`	`EXPR_FITEM`
`static int`	`EXPR_FNAME`
`static int`	`EXPR_LABEL`
`static int`	`EXPR_LABELED`
`static int`	`EXPR_MID`
`static int`	`EXPR_VALUE`
`protected ByteList`	`GT`
`protected ByteList`	`GT_EQ`
`protected ByteList`	`GT_GT`
`protected boolean`	`has_shebang`
`protected int`	`heredoc_end`
`protected int`	`heredoc_indent`
`protected int`	`heredoc_line_indent`
`boolean`	`inKwarg`
`protected int`	`last_cr_line`
`protected int`	`last_state`
`protected ByteList`	`LBRACKET`
`static ByteList`	`LBRACKET_RBRACKET`
`protected ByteList`	`LBRACKET_RBRACKET_EQ`
`protected ByteList`	`LCURLY`
`ByteList`	`lex_lastline`
`protected ByteList`	`lex_nextline`
`int`	`lex_p`
`protected int`	`lex_pbeg`
`int`	`lex_pend`
`protected int`	`lex_state`
`ByteList`	`lexb`
`protected int`	`line_count`
`protected int`	`line_offset`
`protected ByteList`	`LT`
`protected ByteList`	`LT_EQ`
`protected ByteList`	`LT_EQ_RT`
`protected ByteList`	`LT_LT`
`protected ByteList`	`MINUS`
`protected ByteList`	`MINUS_AT`
`protected ByteList`	`MINUS_GT`
`protected ByteList`	`OR`
`static ByteList`	`OR_OR`
`protected int`	`parenNest`
`protected ByteList`	`PERCENT`
`protected ByteList`	`PLUS`
`protected ByteList`	`PLUS_AT`
`protected ByteList`	`Q`
`protected ByteList`	`QQ`
`protected ByteList`	`QUESTION`
`protected ByteList`	`RBRACKET`
`protected ByteList`	`RCURLY`
`protected ByteList`	`RPAREN`
`protected int`	`ruby_sourceline`
`protected ByteList`	`SEMICOLON`
`protected ByteList`	`SLASH`
`protected LexerSource`	`src`
`protected ByteList`	`STAR`
`protected ByteList`	`STAR_STAR`
`static int`	`str_dquote`
`static int`	`str_dsym`
`static int`	`str_dword`
`static int`	`STR_FUNC_ESCAPE`
`static int`	`STR_FUNC_EXPAND`
`static int`	`STR_FUNC_INDENT`
`static int`	`STR_FUNC_LABEL`
`static int`	`STR_FUNC_LIST`
`static int`	`STR_FUNC_QWORDS`
`static int`	`STR_FUNC_REGEXP`
`static int`	`STR_FUNC_SYMBOL`
`static int`	`STR_FUNC_TERM`
`static int`	`str_label`
`static int`	`str_regexp`
`static int`	`str_squote`
`static int`	`str_ssym`
`static int`	`str_sword`
`static int`	`str_xquote`
`static int`	`SUFFIX_ALL`
`static int`	`SUFFIX_I`
`static int`	`SUFFIX_R`
`static int`	`TAB_WIDTH`
`protected ByteList`	`TILDE`
`protected int`	`token`
`protected boolean`	`tokenSeen`
`ISourcePosition`	`tokline`
`int`	`tokp`
`static org.jcodings.Encoding`	`USASCII_ENCODING`
`static org.jcodings.Encoding`	`UTF8_ENCODING`
`protected java.lang.Object`	`yaccValue`

Constructor Summary

Constructors
Constructor and Description

LexingCommon(LexerSource src)

Constructors
Constructor and Description
`LexingCommon(LexerSource src)`

Method Summary

All Methods Static Methods Instance Methods Abstract Methods Concrete Methods Deprecated Methods
Modifier and Type	Method and Description
`protected abstract void`	`ambiguousOperator(java.lang.String op, java.lang.String syn)`
`void`	`checkRegexpFragment(Ruby runtime, ByteList value, RegexpOptions options)`
`void`	`checkRegexpSyntax(Ruby runtime, ByteList value, RegexpOptions options)`
`int`	`column()`
`protected boolean`	`comment_at_top()`
`abstract void`	`compile_error(java.lang.String message)`
`java.lang.String`	`createAsEncodedString(byte[] bytes, int start, int length, org.jcodings.Encoding encoding)`
`ByteList`	`createTokenByteList()`
`ByteList`	`createTokenByteList(int start)`
`java.lang.String`	`createTokenString()`
`java.lang.String`	`createTokenString(int start)`
`static int`	`dedent_string(ByteList string, int width)`
`protected void`	`flush()`
`int`	`getBraceNest()`
`StackState`	`getCmdArgumentState()`
`StackState`	`getConditionState()`
`ByteList`	`getCurrentArg()`
`java.lang.String`	`getCurrentLine()`
`org.jcodings.Encoding`	`getEncoding()`
`java.lang.String`	`getFile()`
`int`	`getHeredocIndent()`
`int`	`getHeredocLineIndent()`
`int`	`getLeftParenBegin()`
`int`	`getLineOffset()`
`ISourcePosition`	`getPosition()`
`int`	`getRubySourceline()`
`int`	`getState()`
`int`	`getTokenCR()`
`int`	`incrementParenNest()`
`protected boolean`	`isAfterOperator()`
`protected boolean`	`isARG()`
`boolean`	`isASCII()`
`boolean`	`isASCII(int c)`
`protected boolean`	`isBEG()`
`protected boolean`	`isEND()`
`boolean`	`isEndSeen()`
`boolean`	`isGlobalCharPunct(int c)`
`static boolean`	`isHexChar(int c)`
`boolean`	`isIdentifierChar(int c)` This is a valid character for an identifier?
`protected boolean`	`isLabelPossible(boolean commandState)`
`boolean`	`isLabelSuffix()`
`static boolean`	`isLexState(int state, int mask)`
`protected boolean`	`isLexStateAll(int state, int mask)`
`protected boolean`	`isNext_identchar()`
`static boolean`	`isOctChar(int c)`
`static boolean`	`isSpace(int c)`
`protected boolean`	`isSpaceArg(int c, boolean spaceSeen)`
`void`	`lex_goto_eol()`
`int`	`lineno()`
`protected void`	`magicCommentEncoding(ByteList encoding)`
`static int`	`magicCommentMarker(ByteList str, int begin)`
`protected abstract void`	`mismatchedRegexpEncodingError(org.jcodings.Encoding optionEncoding, org.jcodings.Encoding encoding)`
`void`	`newtok(boolean unreadOnce)`
`abstract int`	`nextc()`
`protected int`	`numberLiteralSuffix(int mask)`
`protected boolean`	`onMagicComment(java.lang.String name, ByteList value)`
`protected char`	`optionsEncodingChar(org.jcodings.Encoding optionEncoding)`
`int`	`p(int offset)`
`boolean`	`parser_magic_comment(ByteList magicLine)`
`void`	`parser_prepare()`
`protected abstract RegexpOptions`	`parseRegexpFlags()`
`protected RegexpOptions`	`parseRegexpFlags(java.lang.StringBuilder unknownFlags)`
`boolean`	`peek(int c)`
`protected boolean`	`peek(int c, int n)`
`int`	`peekVariableName(int tSTRING_DVAR, int tSTRING_DBEG)`
`int`	`precise_mbclen()`
`void`	`printState()`
`void`	`pushback(int c)`
`void`	`reset()`
`void`	`resetStacks()`
`protected char`	`scanOct(int count)`
`protected void`	`set_file_encoding(int str, int send)`
`void`	`setBraceNest(int nest)`
`protected abstract void`	`setCompileOptionFlag(java.lang.String name, ByteList value)`
`void`	`setCurrentArg(ByteList current_arg)`
`void`	`setCurrentEncoding(org.jcodings.Encoding encoding)`
`protected abstract void`	`setEncoding(ByteList name)`
`void`	`setEncoding(org.jcodings.Encoding encoding)`
`void`	`setHeredocIndent(int heredoc_indent)`
`void`	`setHeredocLineIndent(int heredoc_line_indent)`
`void`	`setLeftParenBegin(int value)`
`void`	`setRegexpEncoding(Ruby runtime, ByteList value, RegexpOptions options)`
`void`	`setRubySourceline(int line)`
`void`	`setSource(LexerSource source)` Allow the parser to set the source for its lexer.
`void`	`setState(int state)`
`protected abstract void`	`setTokenInfo(java.lang.String name, ByteList value)`
`void`	`setValue(java.lang.Object yaccValue)`
`protected boolean`	`strncmp(ByteList one, ByteList two, int length)`
`boolean`	`tokadd_ident(int c)`
`boolean`	`tokadd_mbchar(int first_byte)` This differs from MRI in a few ways.
`boolean`	`tokadd_mbchar(int first_byte, ByteList buffer)`
`void`	`tokAdd(int first_byte, ByteList buffer)`
`void`	`tokaddmbc(int codepoint, ByteList buffer)` This looks deceptively like tokadd_mbchar(int, ByteList) but it differs in that it uses the bytelists encoding and the first parameter is a full codepoint and not the first byte of a mbc sequence.
`void`	`tokCopy(int length, ByteList buffer)`
`int`	`token()` Last token read from the lexer at the end of a call to yylex()
`abstract int`	`tokenize_ident(int result)`
`boolean`	`update_heredoc_indent(int c)`
`void`	`validateFormalIdentifier(ByteList identifier)`
`void`	`validateFormalIdentifier(java.lang.String identifier)` Deprecated.
`java.lang.Object`	`value()` Value of last token (if it is a token which has a value).
`protected void`	`warn_balanced(int c, boolean spaceSeen, java.lang.String op, java.lang.String syn)`
`boolean`	`was_bol()`
`boolean`	`whole_match_p(ByteList eos, boolean indent)`

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

EXPR_BEG
```
public static final int EXPR_BEG
```
See Also:

Constant Field Values

EXPR_END
```
public static final int EXPR_END
```
See Also:

Constant Field Values

EXPR_ENDARG
```
public static final int EXPR_ENDARG
```
See Also:

Constant Field Values

EXPR_ENDFN
```
public static final int EXPR_ENDFN
```
See Also:

Constant Field Values

EXPR_ARG
```
public static final int EXPR_ARG
```
See Also:

Constant Field Values

EXPR_CMDARG
```
public static final int EXPR_CMDARG
```
See Also:

Constant Field Values

EXPR_MID
```
public static final int EXPR_MID
```
See Also:

Constant Field Values

EXPR_FNAME
```
public static final int EXPR_FNAME
```
See Also:

Constant Field Values

EXPR_DOT
```
public static final int EXPR_DOT
```
See Also:

Constant Field Values

EXPR_CLASS
```
public static final int EXPR_CLASS
```
See Also:

Constant Field Values

EXPR_LABEL
```
public static final int EXPR_LABEL
```
See Also:

Constant Field Values

EXPR_LABELED
```
public static final int EXPR_LABELED
```
See Also:

Constant Field Values

EXPR_FITEM
```
public static final int EXPR_FITEM
```
See Also:

Constant Field Values

EXPR_VALUE
```
public static final int EXPR_VALUE
```
See Also:

Constant Field Values

EXPR_BEG_ANY
```
public static final int EXPR_BEG_ANY
```
See Also:

Constant Field Values

EXPR_ARG_ANY
```
public static final int EXPR_ARG_ANY
```
See Also:

Constant Field Values

EXPR_END_ANY
```
public static final int EXPR_END_ANY
```
See Also:

Constant Field Values

braceNest
```
protected int braceNest
```

commandStart
```
public boolean commandStart
```

conditionState
```
protected StackState conditionState
```

cmdArgumentState
```
protected StackState cmdArgumentState
```

__end__seen
```
protected boolean __end__seen
```

eofp
```
public boolean eofp
```

has_shebang
```
protected boolean has_shebang
```

heredoc_end
```
protected int heredoc_end
```

heredoc_indent
```
protected int heredoc_indent
```

heredoc_line_indent
```
protected int heredoc_line_indent
```

inKwarg
```
public boolean inKwarg
```

last_cr_line
```
protected int last_cr_line
```

last_state
```
protected int last_state
```

lexb
```
public ByteList lexb
```

lex_lastline
```
public ByteList lex_lastline
```

lex_nextline
```
protected ByteList lex_nextline
```

lex_p
```
public int lex_p
```

lex_pbeg
```
protected int lex_pbeg
```

lex_pend
```
public int lex_pend
```

lex_state
```
protected int lex_state
```

line_count
```
protected int line_count
```

line_offset
```
protected int line_offset
```

parenNest
```
protected int parenNest
```

ruby_sourceline
```
protected int ruby_sourceline
```

src
```
protected LexerSource src
```

token
```
protected int token
```

tokenSeen
```
protected boolean tokenSeen
```

tokline
```
public ISourcePosition tokline
```

tokp
```
public int tokp
```

yaccValue
```
protected java.lang.Object yaccValue
```

BACKTICK
```
public final ByteList BACKTICK
```

EQ_EQ_EQ
```
public final ByteList EQ_EQ_EQ
```

EQ_EQ
```
public final ByteList EQ_EQ
```

EQ_TILDE
```
protected ByteList EQ_TILDE
```

EQ_GT
```
protected ByteList EQ_GT
```

EQ
```
protected ByteList EQ
```

AMPERSAND_AMPERSAND

public static final ByteList AMPERSAND_AMPERSAND

AMPERSAND
```
protected ByteList AMPERSAND
```

AMPERSAND_DOT
```
public ByteList AMPERSAND_DOT
```

BANG
```
public final ByteList BANG
```

BANG_EQ
```
protected ByteList BANG_EQ
```

BANG_TILDE
```
protected ByteList BANG_TILDE
```

CARET
```
protected ByteList CARET
```

COLON_COLON
```
protected ByteList COLON_COLON
```

COLON
```
protected ByteList COLON
```

COMMA
```
protected ByteList COMMA
```

DOT_DOT_DOT
```
protected ByteList DOT_DOT_DOT
```

DOT_DOT
```
protected ByteList DOT_DOT
```

DOT
```
public ByteList DOT
```

GT_EQ
```
protected ByteList GT_EQ
```

GT_GT
```
protected ByteList GT_GT
```

GT
```
protected ByteList GT
```

LBRACKET_RBRACKET_EQ

protected ByteList LBRACKET_RBRACKET_EQ

LBRACKET_RBRACKET

public static ByteList LBRACKET_RBRACKET

LBRACKET
```
protected ByteList LBRACKET
```

LCURLY
```
protected ByteList LCURLY
```

LT_EQ_RT
```
protected ByteList LT_EQ_RT
```

LT_EQ
```
protected ByteList LT_EQ
```

LT_LT
```
protected ByteList LT_LT
```

LT
```
protected ByteList LT
```

MINUS_AT
```
protected ByteList MINUS_AT
```

MINUS
```
protected ByteList MINUS
```

MINUS_GT
```
protected ByteList MINUS_GT
```

PERCENT
```
protected ByteList PERCENT
```

OR_OR
```
public static final ByteList OR_OR
```

OR
```
protected ByteList OR
```

PLUS_AT
```
protected ByteList PLUS_AT
```

PLUS
```
protected ByteList PLUS
```

QUESTION
```
protected ByteList QUESTION
```

RBRACKET
```
protected ByteList RBRACKET
```

RCURLY
```
protected ByteList RCURLY
```

RPAREN
```
protected ByteList RPAREN
```

Q
```
protected ByteList Q
```

SLASH
```
protected ByteList SLASH
```

STAR
```
protected ByteList STAR
```

STAR_STAR
```
protected ByteList STAR_STAR
```

TILDE
```
protected ByteList TILDE
```

QQ
```
protected ByteList QQ
```

SEMICOLON
```
protected ByteList SEMICOLON
```

BACKSLASH
```
protected ByteList BACKSLASH
```

CALL
```
public static final ByteList CALL
```

DOLLAR_BANG

public static final ByteList DOLLAR_BANG

DOLLAR_UNDERSCORE

public static final ByteList DOLLAR_UNDERSCORE

DOLLAR_DOT

public static final ByteList DOLLAR_DOT

TAB_WIDTH
```
public static final int TAB_WIDTH
```
See Also:

Constant Field Values

STR_FUNC_ESCAPE
```
public static final int STR_FUNC_ESCAPE
```
See Also:

Constant Field Values

STR_FUNC_EXPAND
```
public static final int STR_FUNC_EXPAND
```
See Also:

Constant Field Values

STR_FUNC_REGEXP
```
public static final int STR_FUNC_REGEXP
```
See Also:

Constant Field Values

STR_FUNC_QWORDS
```
public static final int STR_FUNC_QWORDS
```
See Also:

Constant Field Values

STR_FUNC_SYMBOL
```
public static final int STR_FUNC_SYMBOL
```
See Also:

Constant Field Values

STR_FUNC_INDENT
```
public static final int STR_FUNC_INDENT
```
See Also:

Constant Field Values

STR_FUNC_LABEL
```
public static final int STR_FUNC_LABEL
```
See Also:

Constant Field Values

STR_FUNC_LIST
```
public static final int STR_FUNC_LIST
```
See Also:

Constant Field Values

STR_FUNC_TERM
```
public static final int STR_FUNC_TERM
```
See Also:

Constant Field Values

str_label
```
public static final int str_label
```
See Also:

Constant Field Values

str_squote
```
public static final int str_squote
```
See Also:

Constant Field Values

str_dquote
```
public static final int str_dquote
```
See Also:

Constant Field Values

str_xquote
```
public static final int str_xquote
```
See Also:

Constant Field Values

str_regexp
```
public static final int str_regexp
```
See Also:

Constant Field Values

str_sword
```
public static final int str_sword
```
See Also:

Constant Field Values

str_dword
```
public static final int str_dword
```
See Also:

Constant Field Values

str_ssym
```
public static final int str_ssym
```
See Also:

Constant Field Values

str_dsym
```
public static final int str_dsym
```
See Also:

Constant Field Values

EOF
```
public static final int EOF
```
See Also:

Constant Field Values

END_MARKER
```
public static ByteList END_MARKER
```

BEGIN_DOC_MARKER

public static ByteList BEGIN_DOC_MARKER

END_DOC_MARKER
```
public static ByteList END_DOC_MARKER
```

CODING
```
public static ByteList CODING
```

UTF8_ENCODING

public static final org.jcodings.Encoding UTF8_ENCODING

USASCII_ENCODING

public static final org.jcodings.Encoding USASCII_ENCODING

ASCII8BIT_ENCODING

public static final org.jcodings.Encoding ASCII8BIT_ENCODING

SUFFIX_R
```
public static final int SUFFIX_R
```
See Also:

Constant Field Values

SUFFIX_I
```
public static final int SUFFIX_I
```
See Also:

Constant Field Values

SUFFIX_ALL
```
public static final int SUFFIX_ALL
```
See Also:

Constant Field Values

Constructor Detail
- LexingCommon
```
public LexingCommon(LexerSource src)
```

Method Detail

column
```
public int column()
```

comment_at_top
```
protected boolean comment_at_top()
```

getRubySourceline
```
public int getRubySourceline()
```

setRubySourceline

public void setRubySourceline(int line)

createTokenByteList
```
public ByteList createTokenByteList()
```

createTokenByteList

public ByteList createTokenByteList(int start)

createTokenString

public java.lang.String createTokenString(int start)

createAsEncodedString

public java.lang.String createAsEncodedString(byte[] bytes,
                                              int start,
                                              int length,
                                              org.jcodings.Encoding encoding)

createTokenString

public java.lang.String createTokenString()

dedent_string

public static int dedent_string(ByteList string,
                                int width)

flush
```
protected void flush()
```

getBraceNest
```
public int getBraceNest()
```

getCmdArgumentState

public StackState getCmdArgumentState()

getConditionState
```
public StackState getConditionState()
```

getCurrentArg
```
public ByteList getCurrentArg()
```

getCurrentLine

public java.lang.String getCurrentLine()

getEncoding

public org.jcodings.Encoding getEncoding()

getFile
```
public java.lang.String getFile()
```

getHeredocIndent
```
public int getHeredocIndent()
```

getHeredocLineIndent
```
public int getHeredocLineIndent()
```

getLeftParenBegin
```
public int getLeftParenBegin()
```

getPosition
```
public ISourcePosition getPosition()
```

getLineOffset
```
public int getLineOffset()
```

getState
```
public int getState()
```

getTokenCR
```
public int getTokenCR()
```

incrementParenNest
```
public int incrementParenNest()
```

isEndSeen
```
public boolean isEndSeen()
```

isASCII
```
public boolean isASCII()
```

isASCII
```
public boolean isASCII(int c)
```

peekVariableName

public int peekVariableName(int tSTRING_DVAR,
                            int tSTRING_DBEG)
                     throws java.io.IOException

Throws:: java.io.IOException

isGlobalCharPunct

public boolean isGlobalCharPunct(int c)

isIdentifierChar
```
public boolean isIdentifierChar(int c)
```
This is a valid character for an identifier?

Parameters:

c - is character to be compared

Returns:

whether c is an identifier or not mri: is_identchar

lex_goto_eol
```
public void lex_goto_eol()
```

lineno
```
public int lineno()
```

magicCommentEncoding

protected void magicCommentEncoding(ByteList encoding)

newtok

public void newtok(boolean unreadOnce)

numberLiteralSuffix

protected int numberLiteralSuffix(int mask)
                           throws java.io.IOException

Throws:: java.io.IOException

parser_prepare
```
public void parser_prepare()
```

p
```
public int p(int offset)
```

peek
```
public boolean peek(int c)
```

peek

protected boolean peek(int c,
                       int n)

precise_mbclen
```
public int precise_mbclen()
```

printState
```
public void printState()
```

pushback
```
public void pushback(int c)
```

reset
```
public void reset()
```

resetStacks
```
public void resetStacks()
```

scanOct

protected char scanOct(int count)
                throws java.io.IOException

Throws:: java.io.IOException

setCurrentArg

public void setCurrentArg(ByteList current_arg)

setCurrentEncoding

public void setCurrentEncoding(org.jcodings.Encoding encoding)

setEncoding

public void setEncoding(org.jcodings.Encoding encoding)

set_file_encoding

protected void set_file_encoding(int str,
                                 int send)

setHeredocLineIndent

public void setHeredocLineIndent(int heredoc_line_indent)

setHeredocIndent

public void setHeredocIndent(int heredoc_indent)

setBraceNest
```
public void setBraceNest(int nest)
```

setLeftParenBegin

public void setLeftParenBegin(int value)

setSource
```
public void setSource(LexerSource source)
```
Allow the parser to set the source for its lexer.

Parameters:

source - where the lexer gets raw data

setState
```
public void setState(int state)
```

setValue

public void setValue(java.lang.Object yaccValue)

strncmp

protected boolean strncmp(ByteList one,
                          ByteList two,
                          int length)

tokAdd

public void tokAdd(int first_byte,
                   ByteList buffer)

tokCopy

public void tokCopy(int length,
                    ByteList buffer)

tokadd_ident
```
public boolean tokadd_ident(int c)
```

tokadd_mbchar
```
public boolean tokadd_mbchar(int first_byte)
```
This differs from MRI in a few ways. This version does not apply value to a separate token buffer. It is for use when we know we will not be omitting or including ant non-syntactical characters. Use tokadd_mbchar(int, ByteList) if the string differs from actual source. Secondly, this returns a boolean instead of the first byte passed. MRI only used the return value as a success/failure code to return EOF. Because this version does not use a separate token buffer we only just increment lex_p. When we reach end of the token it will just get the bytes directly from source directly.

tokadd_mbchar

public boolean tokadd_mbchar(int first_byte,
                             ByteList buffer)

tokaddmbc
```
public void tokaddmbc(int codepoint,
                      ByteList buffer)
```
This looks deceptively like tokadd_mbchar(int, ByteList) but it differs in that it uses the bytelists encoding and the first parameter is a full codepoint and not the first byte of a mbc sequence.

token
```
public int token()
```
Last token read from the lexer at the end of a call to yylex()

Returns:

last token read

update_heredoc_indent

public boolean update_heredoc_indent(int c)

validateFormalIdentifier

public void validateFormalIdentifier(ByteList identifier)

validateFormalIdentifier

@Deprecated
public void validateFormalIdentifier(java.lang.String identifier)

Deprecated.

value
```
public java.lang.Object value()
```
Value of last token (if it is a token which has a value).

Returns:

value of last value-laden token

warn_balanced

protected void warn_balanced(int c,
                             boolean spaceSeen,
                             java.lang.String op,
                             java.lang.String syn)

was_bol
```
public boolean was_bol()
```

whole_match_p

public boolean whole_match_p(ByteList eos,
                             boolean indent)

ambiguousOperator

protected abstract void ambiguousOperator(java.lang.String op,
                                          java.lang.String syn)

compile_error

public abstract void compile_error(java.lang.String message)

nextc
```
public abstract int nextc()
```

setCompileOptionFlag

protected abstract void setCompileOptionFlag(java.lang.String name,
                                             ByteList value)

setEncoding

protected abstract void setEncoding(ByteList name)

setTokenInfo

protected abstract void setTokenInfo(java.lang.String name,
                                     ByteList value)

tokenize_ident

public abstract int tokenize_ident(int result)

isHexChar
```
public static boolean isHexChar(int c)
```
Parameters:

c - the character to test

Returns:

true if character is a hex value (0-9a-f)

isLexState

public static boolean isLexState(int state,
                                 int mask)

isLexStateAll

protected boolean isLexStateAll(int state,
                                int mask)

isARG
```
protected boolean isARG()
```

isBEG
```
protected boolean isBEG()
```

isEND
```
protected boolean isEND()
```

isLabelPossible

protected boolean isLabelPossible(boolean commandState)

isLabelSuffix
```
public boolean isLabelSuffix()
```

isAfterOperator
```
protected boolean isAfterOperator()
```

isNext_identchar

protected boolean isNext_identchar()
                            throws java.io.IOException

Throws:: java.io.IOException

isOctChar
```
public static boolean isOctChar(int c)
```
Parameters:

c - the character to test

Returns:

true if character is an octal value (0-7)

isSpace
```
public static boolean isSpace(int c)
```

isSpaceArg

protected boolean isSpaceArg(int c,
                             boolean spaceSeen)

magicCommentMarker

public static int magicCommentMarker(ByteList str,
                                     int begin)

parser_magic_comment

public boolean parser_magic_comment(ByteList magicLine)

onMagicComment

protected boolean onMagicComment(java.lang.String name,
                                 ByteList value)

parseRegexpFlags

protected abstract RegexpOptions parseRegexpFlags()
                                           throws java.io.IOException

Throws:: java.io.IOException

parseRegexpFlags

protected RegexpOptions parseRegexpFlags(java.lang.StringBuilder unknownFlags)
                                  throws java.io.IOException

Throws:: java.io.IOException

checkRegexpFragment

public void checkRegexpFragment(Ruby runtime,
                                ByteList value,
                                RegexpOptions options)

checkRegexpSyntax

public void checkRegexpSyntax(Ruby runtime,
                              ByteList value,
                              RegexpOptions options)

mismatchedRegexpEncodingError

protected abstract void mismatchedRegexpEncodingError(org.jcodings.Encoding optionEncoding,
                                                      org.jcodings.Encoding encoding)

setRegexpEncoding

public void setRegexpEncoding(Ruby runtime,
                              ByteList value,
                              RegexpOptions options)

optionsEncodingChar

protected char optionsEncodingChar(org.jcodings.Encoding optionEncoding)

Class LexingCommon

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

EXPR_BEG

EXPR_END

EXPR_ENDARG

EXPR_ENDFN

EXPR_ARG

EXPR_CMDARG

EXPR_MID

EXPR_FNAME

EXPR_DOT

EXPR_CLASS

EXPR_LABEL

EXPR_LABELED

EXPR_FITEM

EXPR_VALUE

EXPR_BEG_ANY

EXPR_ARG_ANY

EXPR_END_ANY

braceNest

commandStart

conditionState

cmdArgumentState

__end__seen

eofp

has_shebang

heredoc_end

heredoc_indent

heredoc_line_indent

inKwarg

last_cr_line

last_state

lexb

lex_lastline

lex_nextline

lex_p

lex_pbeg

lex_pend

lex_state

line_count

line_offset

parenNest

ruby_sourceline

src

token

tokenSeen

tokline

tokp

yaccValue

BACKTICK

EQ_EQ_EQ

EQ_EQ

EQ_TILDE

EQ_GT

EQ

AMPERSAND_AMPERSAND

AMPERSAND

AMPERSAND_DOT

BANG

BANG_EQ

BANG_TILDE

CARET

COLON_COLON

COLON

COMMA

DOT_DOT_DOT

DOT_DOT

DOT

GT_EQ

GT_GT

GT

LBRACKET_RBRACKET_EQ

LBRACKET_RBRACKET

LBRACKET

LCURLY

LT_EQ_RT

endseen