Class TokenConverter


  • public class TokenConverter
    extends java.lang.Object
    This token converter will convert input token list to another token list, with matched TokenMatcher pattern. It is useful to convert a specific raw input to whole token matching specific cases. See TokenMatcher implementations.
    • Constructor Detail

      • TokenConverter

        public TokenConverter​(TokenMatcher[] termMatchers)
    • Method Detail

      • executeTermDetection

        public java.util.List<Token> executeTermDetection​(java.util.List<Token> inputTokenList)
                                                   throws java.io.IOException
        Throws:
        java.io.IOException
      • executeTokenPatternMatching

        public java.util.List<TrainerTask> executeTokenPatternMatching​(TrainingCorpus corpus)
                                                                throws java.io.IOException
        Throws:
        java.io.IOException