Token (Simple Binary Encoding)

java.lang.Object
- uk.co.real_logic.sbe.ir.Token

```
public class Token
extends java.lang.Object
```
Class to encapsulate a token of information for the message schema stream. This Intermediate Representation (IR) is intended to be language, schema, platform independent. Processing and optimization could be run over a list of Tokens to perform various functions
- re-ordering of fields based on encodedLength
- padding of fields in order to provide expansion room
- computing offsets of individual fields
- etc.
IR could be used to generate code or other specifications. It should be possible to do the following:
- generate a FIX/SBE schema from IR
- generate an ASN.1 spec from IR
- generate a GPB spec from IR
- etc.
IR could be serialized to storage or network via code generated by SBE. Then read back in to a List of Tokens. The entire IR of an entity is a List of Token objects. The order of this list is very important. Encoding of fields is done by nodes pointing to specific encoding PrimitiveType objects. Each encoding node contains encodedLength, offset, byte order, and Encoding. Entities relevant to the encoding such as fields, messages, repeating groups, etc. are encapsulated in the list as nodes themselves. Although, they will in most cases never be serialized. The boundaries of these entities are delimited by BEGIN and END Signal values in the node Encoding. A list structure like this allows for each concatenation of encodings as well as easy traversal. An example encoding of a message headerStructure might be like this.
- Token 0 - Signal = BEGIN_MESSAGE, schemaId = 100
- Token 1 - Signal = BEGIN_FIELD, schemaId = 25
- Token 2 - Signal = ENCODING, PrimitiveType = uint32, encodedLength = 4, offset = 0
- Token 3 - Signal = END_FIELD
- Token 4 - Signal = END_MESSAGE

Nested Class Summary

Nested Classes
Modifier and Type Class and Description

static class Token.Builder

static class Token.VersionContext
Indicates how the version field should be interpreted.

Nested Classes
Modifier and Type	Class and Description
`static class`	`Token.Builder`
`static class`	`Token.VersionContext` Indicates how the version field should be interpreted.

Field Summary

Fields
Modifier and Type	Field and Description
`static int`	`INVALID_ID` Invalid ID value.
`static int`	`UNKNOWN_OFFSET` Offset not computed or set
`static int`	`VARIABLE_LENGTH` Length not determined

Constructor Summary

Constructors
Constructor and Description
`Token(Signal signal, java.lang.String name, java.lang.String referencedName, java.lang.String description, int id, int version, int deprecated, int encodedLength, int offset, int componentTokenCount, Encoding encoding)` Construct an `Token` by providing values for all fields.

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`java.lang.String`	`applicableTypeName()` Get the name of the type that should be applied in context.
`int`	`arrayLength()` The number of encoded primitives in this type.
`int`	`componentTokenCount()` The number of tokens that make up this component.
`void`	`componentTokenCount(int componentTokenCount)` Set the number of tokens this component has.
`int`	`deprecated()` The version in which this context was deprecated.
`java.lang.String`	`description()` Description for what the token is to be used for.
`int`	`encodedLength()` The encodedLength of this token in bytes.
`void`	`encodedLength(int encodedLength)` Set the encoded length for this node.
`Encoding`	`encoding()` Return the `Encoding` of the `Token`.
`int`	`id()` Return the ID of the token assigned by the specification
`boolean`	`isConstantEncoding()`
`boolean`	`isOptionalEncoding()`
`java.lang.CharSequence`	`matchOnLength(java.util.function.Supplier<java.lang.CharSequence> one, java.util.function.Supplier<java.lang.CharSequence> many)` Match which approach to take based on the length of the token.
`java.lang.String`	`name()` Return the name of the token
`int`	`offset()` The offset for this token in the message.
`java.lang.String`	`referencedName()` Get the name of the type when this is from a reference.
`Signal`	`signal()` Signal the role of this token.
`java.lang.String`	`toString()`
`int`	`version()` The version context for this token.
`Token.VersionContext`	`versionContext()` The context in which the version field should be interpreted.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

- Field Detail
  - INVALID_ID
```
public static final int INVALID_ID
```
    Invalid ID value.
    
    See Also:
    
    Constant Field Values
  - VARIABLE_LENGTH
```
public static final int VARIABLE_LENGTH
```
    Length not determined
    
    See Also:
    
    Constant Field Values
  - UNKNOWN_OFFSET
```
public static final int UNKNOWN_OFFSET
```
    Offset not computed or set
    
    See Also:
    
    Constant Field Values
- Constructor Detail
  - Token
```
public Token(Signal signal,
             java.lang.String name,
             java.lang.String referencedName,
             java.lang.String description,
             int id,
             int version,
             int deprecated,
             int encodedLength,
             int offset,
             int componentTokenCount,
             Encoding encoding)
```
    Construct an Token by providing values for all fields.
    
    Parameters:
    
    signal - for the token role.
    
    name - of the token in the message.
    
    referencedName - of the type when created from a ref in a composite.
    
    description - of what the token is for.
    
    id - as the identifier in the message declaration.
    
    version - application within the template.
    
    deprecated - as of this version.
    
    encodedLength - of the component part.
    
    offset - in the underlying message as octets.
    
    componentTokenCount - number of tokens in this component.
    
    encoding - of the primitive field.
- Method Detail
  - signal
```
public Signal signal()
```
    Signal the role of this token.
    
    Returns:
    
    the Signal for the token.
  - name
```
public java.lang.String name()
```
    Return the name of the token
    
    Returns:
    
    name of the token
  - referencedName
```
public java.lang.String referencedName()
```
    Get the name of the type when this is from a reference.
    
    Returns:
    
    the name of the type when this is from a reference.
  - description
```
public java.lang.String description()
```
    Description for what the token is to be used for.
    
    Returns:
    
    description for what the token is to be used for.
  - id
```
public int id()
```
    Return the ID of the token assigned by the specification
    
    Returns:
    
    ID of the token assigned by the specification
  - version
```
public int version()
```
    The version context for this token.
    
    Returns:
    
    version context for this token.
    
    See Also:
    
    versionContext()
  - deprecated
```
public int deprecated()
```
    The version in which this context was deprecated.
    
    Returns:
    
    the version in which this context was deprecated.
  - applicableTypeName
```
public java.lang.String applicableTypeName()
```
    Get the name of the type that should be applied in context.
    
    Returns:
    
    the name of the type that should be applied in context.
  - versionContext
```
public Token.VersionContext versionContext()
```
    The context in which the version field should be interpreted.
    
    Returns:
    
    context in which the version field should be interpreted.
  - encodedLength
```
public int encodedLength()
```
    The encodedLength of this token in bytes.
    
    Returns:
    
    the encodedLength of this node. A value of 0 means the node has no encodedLength when encoded. A value of VARIABLE_LENGTH means this node represents a variable length field.
  - encodedLength
```
public void encodedLength(int encodedLength)
```
    Set the encoded length for this node. See encodedLength().
    
    Parameters:
    
    encodedLength - that is overriding existing value.
  - arrayLength
```
public int arrayLength()
```
    The number of encoded primitives in this type.
    
    Returns:
    
    number of encoded primitives in this type.
  - matchOnLength
```
public java.lang.CharSequence matchOnLength(java.util.function.Supplier<java.lang.CharSequence> one,
                                            java.util.function.Supplier<java.lang.CharSequence> many)
```
    Match which approach to take based on the length of the token. If length is zero then an empty String is returned.
    
    Parameters:
    
    one - to be used when length is one.
    
    many - to be used when length is greater than one.
    
    Returns:
    
    the CharSequence representing the token depending on the length.
  - offset
```
public int offset()
```
    The offset for this token in the message.
    
    Returns:
    
    the offset of this Token. A value of 0 means the node has no relevant offset. A value of UNKNOWN_OFFSET means this nodes true offset is dependent on variable length fields ahead of it in the encoding.
  - componentTokenCount
```
public int componentTokenCount()
```
    The number of tokens that make up this component.
    
    Returns:
    
    the number of tokens that make up this component.
  - componentTokenCount
```
public void componentTokenCount(int componentTokenCount)
```
    Set the number of tokens this component has.
    
    Parameters:
    
    componentTokenCount - the number of tokens this component has.
  - encoding
```
public Encoding encoding()
```
    Return the Encoding of the Token.
    
    Returns:
    
    encoding of the Token
  - isConstantEncoding
```
public boolean isConstantEncoding()
```
  - isOptionalEncoding
```
public boolean isOptionalEncoding()
```
  - toString
```
public java.lang.String toString()
```
    Overrides:
    
    toString in class java.lang.Object

Class Token

Nested Class Summary

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

INVALID_ID

VARIABLE_LENGTH

UNKNOWN_OFFSET

Constructor Detail

Token

Method Detail

signal

name

referencedName

description

id

version

deprecated

applicableTypeName

versionContext

encodedLength

encodedLength

arrayLength

matchOnLength

offset

componentTokenCount

componentTokenCount

encoding

isConstantEncoding

isOptionalEncoding

toString