Package org.predict4all.nlp.ngram

Class NGramDictionaryGenerator

java.lang.Object

org.predict4all.nlp.ngram.NGramDictionaryGenerator

public class NGramDictionaryGenerator
extends java.lang.Object

Use this generator to train an ngram model.
It will load texts from a TrainingCorpus and generate a ngram file that could be later opened with a StaticNGramTrieDictionary

Constructor Summary

Constructors

Constructor	Description
NGramDictionaryGenerator(LanguageModel languageModel, TrainingConfiguration trainingConfiguration, WordDictionary wordDictionary)

Method Summary

Modifier and Type	Method	Description
java.util.Map<java.lang.Integer,Pair<java.lang.Integer,java.lang.Integer>>	executeNGramTraining(TrainingCorpus corpus, java.io.File ngramOutputFile, java.util.function.Consumer<java.util.List<TrainerTask>> blockingTaskExecutor)
java.lang.String	getDebugPrefix()
NGramDebugger	getNgramDebugAfterPruning()
NGramDebugger	getNgramDebugBeforePruning()
void	setDebugPrefix(java.lang.String debugPrefix)
void	setNgramDebugAfterPruning(NGramDebugger ngramDebugAfterPruning)
void	setNgramDebugBeforePruning(NGramDebugger ngramDebugBeforePruning)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

NGramDictionaryGenerator

public NGramDictionaryGenerator(LanguageModel languageModel,
                                TrainingConfiguration trainingConfiguration,
                                WordDictionary wordDictionary)

Method Detail

executeNGramTraining

public java.util.Map<java.lang.Integer,Pair<java.lang.Integer,java.lang.Integer>> executeNGramTraining(TrainingCorpus corpus,
                                                                                                                   java.io.File ngramOutputFile,
                                                                                                                   java.util.function.Consumer<java.util.List<TrainerTask>> blockingTaskExecutor)
                                                                                                            throws java.io.IOException

Throws:

java.io.IOException

getNgramDebugBeforePruning

public NGramDebugger getNgramDebugBeforePruning()

setNgramDebugBeforePruning

public void setNgramDebugBeforePruning(NGramDebugger ngramDebugBeforePruning)

getNgramDebugAfterPruning

public NGramDebugger getNgramDebugAfterPruning()

setNgramDebugAfterPruning

public void setNgramDebugAfterPruning(NGramDebugger ngramDebugAfterPruning)

getDebugPrefix

public java.lang.String getDebugPrefix()

setDebugPrefix

public void setDebugPrefix(java.lang.String debugPrefix)