Package ai.djl.modality.nlp.preprocess

Class LambdaProcessor

java.lang.Object

ai.djl.modality.nlp.preprocess.LambdaProcessor

All Implemented Interfaces:

TextProcessor

public class LambdaProcessor
extends Object
implements TextProcessor

TextProcessor will apply user defined lambda function on input tokens.

The function can only support single input and output.

Constructor Summary

Constructors

Constructor	Description
LambdaProcessor(Function<String,String> processor)	Creates a LambdaProcessor and specify the function to apply.

Method Summary

Modifier and Type	Method	Description
List<String>	preprocess(List<String> tokens)	Applies the preprocessing defined to the given input tokens.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

LambdaProcessor

public LambdaProcessor(Function<String,String> processor)

Creates a LambdaProcessor and specify the function to apply.

Parameters:

processor - The lambda function to apply on input String

Method Details

preprocess

public List<String> preprocess(List<String> tokens)

Applies the preprocessing defined to the given input tokens.

Specified by:

preprocess in interface TextProcessor

Parameters:

tokens - the tokens created after the input text is tokenized

Returns:

the preprocessed tokens