Token (Simple Binary Encoding (SBE))

java.lang.Object
- uk.co.real_logic.sbe.ir.Token

```
public class Token
extends Object
```
Class to encapsulate a token of information for the message schema stream. This Intermediate Representation (IR) is intended to be language, schema, platform independent.
Processing and optimization could be run over a list of Tokens to perform various functions
- re-ordering of fields based on encodedLength
- padding of fields in order to provide expansion room
- computing offsets of individual fields
- etc.
IR could be used to generate code or other specifications. It should be possible to do the following:
- generate a FIX/SBE schema from IR
- generate an ASN.1 spec from IR
- generate a GPB spec from IR
- etc.
IR could be serialized to storage or network via code generated by SBE. Then read back in to a List of Tokens.
The entire IR of an entity is a List of Token objects. The order of this list is very important. Encoding of fields is done by nodes pointing to specific encoding PrimitiveType objects. Each encoding node contains encodedLength, offset, byte order, and Encoding. Entities relevant to the encoding such as fields, messages, repeating groups, etc. are encapsulated in the list as nodes themselves. Although, they will in most cases never be serialized. The boundaries of these entities are delimited by BEGIN and END Signal values in the node Encoding. A list structure like this allows for each concatenation of encodings as well as easy traversal.
An example encoding of a message headerStructure might be like this.
- Token 0 - Signal = BEGIN_MESSAGE, schemaId = 100
- Token 1 - Signal = BEGIN_FIELD, schemaId = 25
- Token 2 - Signal = ENCODING, PrimitiveType = uint32, encodedLength = 4, offset = 0
- Token 3 - Signal = END_FIELD
- Token 4 - Signal = END_MESSAGE

Nested Class Summary

Nested Classes
Modifier and Type Class and Description

static class Token.Builder
Builder for Token which can simplify construction.

Nested Classes
Modifier and Type	Class and Description
`static class`	`Token.Builder` Builder for `Token` which can simplify construction.

Field Summary

Fields
Modifier and Type Field and Description

static int INVALID_ID
Invalid ID value.

static int VARIABLE_LENGTH
Length not determined

Fields
Modifier and Type	Field and Description
`static int`	`INVALID_ID` Invalid ID value.
`static int`	`VARIABLE_LENGTH` Length not determined

Constructor Summary

Constructors
Constructor and Description
`Token(Signal signal, String name, String referencedName, String description, String packageName, int id, int version, int deprecated, int encodedLength, int offset, int componentTokenCount, Encoding encoding)` Construct an `Token` by providing values for all fields.

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`String`	`applicableTypeName()` Get the name of the type that should be applied in context.
`int`	`arrayLength()` The number of encoded primitives in this type.
`int`	`componentTokenCount()` The number of tokens that make up this component.
`void`	`componentTokenCount(int componentTokenCount)` Set the number of tokens this component has.
`int`	`deprecated()` The version in which this context was deprecated.
`String`	`description()` Description for what the token is to be used for.
`int`	`encodedLength()` The encodedLength of this token in bytes.
`void`	`encodedLength(int encodedLength)` Set the encoded length for this node.
`Encoding`	`encoding()` Return the `Encoding` of the `Token`.
`int`	`id()` Return the ID of the token assigned by the specification
`boolean`	`isConstantEncoding()` Is the encoding presence is a constant or not?
`boolean`	`isOptionalEncoding()` Is the encoding presence is optional or not?
`CharSequence`	`matchOnLength(Supplier<CharSequence> one, Supplier<CharSequence> many)` Match which approach to take based on the length of the token.
`String`	`name()` Return the name of the token
`int`	`offset()` The offset for this token in the message.
`String`	`packageName()` Return the packageName of the token
`String`	`referencedName()` Get the name of the type when this is from a reference.
`Signal`	`signal()` Signal the role of this token.
`String`	`toString()`
`int`	`version()` The version context for this token.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

- Field Detail
  - INVALID_ID
```
public static final int INVALID_ID
```
    Invalid ID value.
    
    See Also:
    
    Constant Field Values
  - VARIABLE_LENGTH
```
public static final int VARIABLE_LENGTH
```
    Length not determined
    
    See Also:
    
    Constant Field Values
- Constructor Detail
  - Token
```
public Token(Signal signal,
             String name,
             String referencedName,
             String description,
             String packageName,
             int id,
             int version,
             int deprecated,
             int encodedLength,
             int offset,
             int componentTokenCount,
             Encoding encoding)
```
    Construct an Token by providing values for all fields.
    
    Parameters:
    
    signal - for the token role.
    
    name - of the token in the message.
    
    referencedName - of the type when created from a ref in a composite.
    
    description - of what the token is for.
    
    packageName - of the token in the message. Use null, except for BEGIN_MESSAGE tokens for types that require an explicit package.
    
    id - as the identifier in the message declaration.
    
    version - application within the template.
    
    deprecated - as of this version.
    
    encodedLength - of the component part.
    
    offset - in the underlying message as octets.
    
    componentTokenCount - number of tokens in this component.
    
    encoding - of the primitive field.
- Method Detail
  - signal
```
public Signal signal()
```
    Signal the role of this token.
    
    Returns:
    
    the Signal for the token.
  - name
```
public String name()
```
    Return the name of the token
    
    Returns:
    
    name of the token
  - packageName
```
public String packageName()
```
    Return the packageName of the token
    
    Returns:
    
    packageName of the token or null, if it was not set explicitly.
  - referencedName
```
public String referencedName()
```
    Get the name of the type when this is from a reference.
    
    Returns:
    
    the name of the type when this is from a reference.
  - description
```
public String description()
```
    Description for what the token is to be used for.
    
    Returns:
    
    description for what the token is to be used for.
  - id
```
public int id()
```
    Return the ID of the token assigned by the specification
    
    Returns:
    
    ID of the token assigned by the specification
  - version
```
public int version()
```
    The version context for this token. This is the schema version in which the type was introduced.
    
    Returns:
    
    version for this type.
  - deprecated
```
public int deprecated()
```
    The version in which this context was deprecated.
    
    Returns:
    
    the version in which this context was deprecated.
  - applicableTypeName
```
public String applicableTypeName()
```
    Get the name of the type that should be applied in context.
    
    Returns:
    
    the name of the type that should be applied in context.
  - encodedLength
```
public int encodedLength()
```
    The encodedLength of this token in bytes.
    
    Returns:
    
    the encodedLength of this node. A value of 0 means the node has no encodedLength when encoded. A value of VARIABLE_LENGTH means this node represents a variable length field.
  - encodedLength
```
public void encodedLength(int encodedLength)
```
    Set the encoded length for this node. See encodedLength().
    
    Parameters:
    
    encodedLength - that is overriding existing value.
  - arrayLength
```
public int arrayLength()
```
    The number of encoded primitives in this type.
    
    Returns:
    
    number of encoded primitives in this type.
  - matchOnLength
```
public CharSequence matchOnLength(Supplier<CharSequence> one,
                                  Supplier<CharSequence> many)
```
    Match which approach to take based on the length of the token. If length is zero then an empty String is returned.
    
    Parameters:
    
    one - to be used when length is one.
    
    many - to be used when length is greater than one.
    
    Returns:
    
    the CharSequence representing the token depending on the length.
  - offset
```
public int offset()
```
    The offset for this token in the message.
    
    Returns:
    
    the offset of this Token. A value of 0 means the node has no relevant offset. A value of VARIABLE_LENGTH means this node's true offset is dependent on variable length fields ahead of it in the encoding.
  - componentTokenCount
```
public int componentTokenCount()
```
    The number of tokens that make up this component.
    
    Returns:
    
    the number of tokens that make up this component.
  - componentTokenCount
```
public void componentTokenCount(int componentTokenCount)
```
    Set the number of tokens this component has.
    
    Parameters:
    
    componentTokenCount - the number of tokens this component has.
  - encoding
```
public Encoding encoding()
```
    Return the Encoding of the Token.
    
    Returns:
    
    encoding of the Token
  - isConstantEncoding
```
public boolean isConstantEncoding()
```
    Is the encoding presence is a constant or not?
    
    Returns:
    
    true if the encoding presence is a constant or false if not.
  - isOptionalEncoding
```
public boolean isOptionalEncoding()
```
    Is the encoding presence is optional or not?
    
    Returns:
    
    true if the encoding presence is optional or false if not.
  - toString
```
public String toString()
```
    Overrides:
    
    toString in class Object

Class Token

Nested Class Summary

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

INVALID_ID

VARIABLE_LENGTH

Constructor Detail

Token

Method Detail

signal

name

packageName

referencedName

description

id

version

deprecated

applicableTypeName

encodedLength

encodedLength

arrayLength

matchOnLength

offset

componentTokenCount

componentTokenCount

encoding

isConstantEncoding

isOptionalEncoding

toString