Package com.yahoo.language.process

Interface Embedder

All Known Implementing Classes:
Embedder.FailingEmbedder
public interface Embedder
An embedder converts a text string to a tensor
Author:
bratseth

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Interface
    Description
    static class 
    Embedder.Context
     
    static class 
    Embedder.FailingEmbedder
     

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    static final String
    defaultEmbedderId
    Name of embedder when none is explicity given
    static final Embedder
    throwsOnUse
    An instance of this which throws IllegalStateException if attempted used

  • Method Summary

    Modifier and Type
    Method
    Description
    default Map<String,Embedder>
    asMap()
    Returns this embedder instance as a map with the default embedder name
    default Map<String,Embedder>
    asMap(String name)
    Returns this embedder instance as a map with the given name
    List<Integer>
    embed(String text, Embedder.Context context)
    Converts text into a list of token id's (a vector embedding)
    com.yahoo.tensor.Tensor
    embed(String text, Embedder.Context context, com.yahoo.tensor.TensorType tensorType)
    Converts text into tokens in a tensor.

  • Field Details

    • defaultEmbedderId

      static final String defaultEmbedderId
      Name of embedder when none is explicity given
      See Also:

    • throwsOnUse

      static final Embedder throwsOnUse
      An instance of this which throws IllegalStateException if attempted used

  • Method Details

    • asMap

      default Map<String,Embedder> asMap()
      Returns this embedder instance as a map with the default embedder name

    • asMap

      default Map<String,Embedder> asMap(String name)
      Returns this embedder instance as a map with the given name

    • embed

      List<Integer> embed(String text, Embedder.Context context)
      Converts text into a list of token id's (a vector embedding)
      Parameters:
      text - the text to embed
      context - the context which may influence an embedder's behavior
      Returns:
      the text embedded as a list of token ids
      Throws:
      IllegalArgumentException - if the language is not supported by this embedder

    • embed

      com.yahoo.tensor.Tensor embed(String text, Embedder.Context context, com.yahoo.tensor.TensorType tensorType)
      Converts text into tokens in a tensor. The information contained in the embedding may depend on the tensor type.
      Parameters:
      text - the text to embed
      context - the context which may influence an embedder's behavior
      tensorType - the type of the tensor to be returned
      Returns:
      the tensor embedding of the text, as the specified tensor type
      Throws:
      IllegalArgumentException - if the language or tensor type is not supported by this embedder