ai.djl.modality.nlp.preprocess

Class LambdaProcessor

java.lang.Object

ai.djl.modality.nlp.preprocess.LambdaProcessor

All Implemented Interfaces:

TextProcessor

public class LambdaProcessor
extends java.lang.Object
implements TextProcessor

TextProcessor will apply user defined lambda function on input tokens.

The function can only support single input and output.

Constructor Summary

Constructors

Constructor and Description
LambdaProcessor(java.util.function.Function<java.lang.String,java.lang.String> processor) Creates a LambdaProcessor and specify the function to apply.

Method Summary

Modifier and Type	Method and Description
java.util.List<java.lang.String>	preprocess(java.util.List<java.lang.String> tokens) Applies the preprocessing defined to the given input tokens.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

LambdaProcessor

public LambdaProcessor(java.util.function.Function<java.lang.String,java.lang.String> processor)

Creates a LambdaProcessor and specify the function to apply.

Parameters:

processor - The lambda function to apply on input String

Method Detail

preprocess

public java.util.List<java.lang.String> preprocess(java.util.List<java.lang.String> tokens)

Applies the preprocessing defined to the given input tokens.

Specified by:

preprocess in interface TextProcessor

Parameters:

tokens - the tokens created after the input text is tokenized

Returns:

the preprocessed tokens