Package ai.djl.modality.nlp.preprocess
Class LowerCaseConvertor
- java.lang.Object
-
- ai.djl.modality.nlp.preprocess.LowerCaseConvertor
-
- All Implemented Interfaces:
TextProcessor
public class LowerCaseConvertor extends java.lang.Object implements TextProcessor
LowerCaseConvertor
converts every character of the input tokens to it's respective lower case character.
-
-
Constructor Summary
Constructors Constructor Description LowerCaseConvertor()
Creates aTextProcessor
that converts input text into lower case character with the default englishLocale
.LowerCaseConvertor(java.util.Locale locale)
Creates aTextProcessor
that converts input text into lower case character given theLocale
.
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description java.util.List<java.lang.String>
preprocess(java.util.List<java.lang.String> tokens)
Applies the preprocessing defined to the given input tokens.
-
-
-
Constructor Detail
-
LowerCaseConvertor
public LowerCaseConvertor(java.util.Locale locale)
Creates aTextProcessor
that converts input text into lower case character given theLocale
.- Parameters:
locale
- the expectedLocale
of the input text
-
LowerCaseConvertor
public LowerCaseConvertor()
Creates aTextProcessor
that converts input text into lower case character with the default englishLocale
.
-
-
Method Detail
-
preprocess
public java.util.List<java.lang.String> preprocess(java.util.List<java.lang.String> tokens)
Applies the preprocessing defined to the given input tokens.- Specified by:
preprocess
in interfaceTextProcessor
- Parameters:
tokens
- the tokens created after the input text is tokenized- Returns:
- the preprocessed tokens
-
-