Package com.didalgo.gpt3

Record Class Encoding.Of

java.lang.Object

java.lang.Record

com.didalgo.gpt3.Encoding.Of

All Implemented Interfaces:

Encoding

Enclosing interface:

Encoding

public static record Encoding.Of(String tiktokenFilename, Map<ByteSequence,Integer> mergeableRanks, Map<String,Integer> specialTokens, Pattern pattern)
extends Record
implements Encoding

Nested Class Summary

Nested classes/interfaces inherited from interface com.didalgo.gpt3.Encoding

Encoding.Lookup, Encoding.Of

Field Summary

Fields inherited from interface com.didalgo.gpt3.Encoding

CL100K_BASE, ENDOFPROMPT, ENDOFTEXT, FIM_MIDDLE, FIM_PREFIX, FIM_SUFFIX, O200K_BASE, P50K_BASE, P50K_EDIT, R50K_BASE

Constructor Summary

Constructors

Constructor	Description
Of(String tiktokenFilename, Map<ByteSequence,Integer> mergeableRanks, Map<String,Integer> specialTokens, Pattern pattern)	Creates an instance of a Of record class.

Method Summary

Modifier and Type	Method	Description
final boolean	equals(Object o)	Indicates whether some other object is "equal to" this one.
final int	hashCode()	Returns a hash code value for this object.
Map<ByteSequence,Integer>	mergeableRanks()	Returns the value of the mergeableRanks record component.
Pattern	pattern()	Returns the value of the pattern record component.
Map<String,Integer>	specialTokens()	Returns the value of the specialTokens record component.
String	tiktokenFilename()	Returns the value of the tiktokenFilename record component.
final String	toString()	Returns a string representation of this record class.

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Details

Of

public Of(String tiktokenFilename,
 Map<ByteSequence,Integer> mergeableRanks,
 Map<String,Integer> specialTokens,
 Pattern pattern)

Creates an instance of a Of record class.

Parameters:

tiktokenFilename - the value for the tiktokenFilename record component

mergeableRanks - the value for the mergeableRanks record component

specialTokens - the value for the specialTokens record component

pattern - the value for the pattern record component

Method Details

mergeableRanks

public Map<ByteSequence,Integer> mergeableRanks()

Returns the value of the mergeableRanks record component.

Specified by:

mergeableRanks in interface Encoding

Returns:

the value of the mergeableRanks record component

toString

public final String toString()

Returns a string representation of this record class. The representation contains the name of the class, followed by the name and value of each of the record components.

Specified by:

toString in class Record

Returns:

a string representation of this object

hashCode

public final int hashCode()

Returns a hash code value for this object. The value is derived from the hash code of each of the record components.

Specified by:

hashCode in class Record

Returns:

a hash code value for this object

equals

public final boolean equals(Object o)

Indicates whether some other object is "equal to" this one. The objects are equal if the other object is of the same class and if all the record components are equal. All components in this record class are compared with Objects::equals(Object,Object).

Specified by:

equals in class Record

Parameters:

o - the object with which to compare

Returns:

true if this object is the same as the o argument; false otherwise.

tiktokenFilename

public String tiktokenFilename()

Returns the value of the tiktokenFilename record component.

Returns:

the value of the tiktokenFilename record component

specialTokens

public Map<String,Integer> specialTokens()

Returns the value of the specialTokens record component.

Specified by:

specialTokens in interface Encoding

Returns:

the value of the specialTokens record component

pattern

public Pattern pattern()

Returns the value of the pattern record component.

Specified by:

pattern in interface Encoding

Returns:

the value of the pattern record component