Package com.yahoo.language.process
Interface Embedder
-
- All Known Implementing Classes:
Embedder.FailingEmbedder
public interface Embedder
An embedder converts a text string to a tensor- Author:
- bratseth
-
-
Nested Class Summary
Nested Classes Modifier and Type Interface Description static class
Embedder.Context
static class
Embedder.FailingEmbedder
-
Field Summary
Fields Modifier and Type Field Description static Embedder
throwsOnUse
An instance of this which throws IllegalStateException if attempted used
-
Method Summary
All Methods Instance Methods Abstract Methods Modifier and Type Method Description List<Integer>
embed(String text, Embedder.Context context)
Converts text into a list of token id's (a vector embedding)com.yahoo.tensor.Tensor
embed(String text, Embedder.Context context, com.yahoo.tensor.TensorType tensorType)
Converts text into tokens in a tensor.
-
-
-
Field Detail
-
throwsOnUse
static final Embedder throwsOnUse
An instance of this which throws IllegalStateException if attempted used
-
-
Method Detail
-
embed
List<Integer> embed(String text, Embedder.Context context)
Converts text into a list of token id's (a vector embedding)- Parameters:
text
- the text to embedcontext
- the context which may influence an embedder's behavior- Returns:
- the text embedded as a list of token ids
- Throws:
IllegalArgumentException
- if the language is not supported by this embedder
-
embed
com.yahoo.tensor.Tensor embed(String text, Embedder.Context context, com.yahoo.tensor.TensorType tensorType)
Converts text into tokens in a tensor. The information contained in the embedding may depend on the tensor type.- Parameters:
text
- the text to embedcontext
- the context which may influence an embedder's behaviortensorType
- the type of the tensor to be returned- Returns:
- the tensor embedding of the text, as the specified tensor type
- Throws:
IllegalArgumentException
- if the language or tensor type is not supported by this embedder
-
-