ai.djl.modality.nlp.preprocess

Class LowerCaseConvertor

java.lang.Object

ai.djl.modality.nlp.preprocess.LowerCaseConvertor

All Implemented Interfaces:

TextProcessor

public class LowerCaseConvertor
extends java.lang.Object
implements TextProcessor

LowerCaseConvertor converts every character of the input tokens to it's respective lower case character.

Constructor Summary

Constructors

Constructor and Description
LowerCaseConvertor() Creates a TextProcessor that converts input text into lower case character with the default english Locale.
LowerCaseConvertor(java.util.Locale locale) Creates a TextProcessor that converts input text into lower case character given the Locale.

Method Summary

Modifier and Type	Method and Description
java.util.List<java.lang.String>	preprocess(java.util.List<java.lang.String> tokens) Applies the preprocessing defined to the given input tokens.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

LowerCaseConvertor

public LowerCaseConvertor(java.util.Locale locale)

Creates a TextProcessor that converts input text into lower case character given the Locale.

Parameters:

locale - the expected Locale of the input text

LowerCaseConvertor

public LowerCaseConvertor()

Creates a TextProcessor that converts input text into lower case character with the default english Locale.

Method Detail

preprocess

public java.util.List<java.lang.String> preprocess(java.util.List<java.lang.String> tokens)

Applies the preprocessing defined to the given input tokens.

Specified by:

preprocess in interface TextProcessor

Parameters:

tokens - the tokens created after the input text is tokenized

Returns:

the preprocessed tokens