Packages

p

epic

preprocess

package preprocess

TODO

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. preprocess
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Type Members

  1. class JavaSentenceSegmenter extends SentenceSegmenter

    A Sentence Segmenter backed by Java's BreakIterator.

    A Sentence Segmenter backed by Java's BreakIterator. Given an input string, it will return an iterator over sentences

  2. class JavaWordTokenizer extends Tokenizer

    A Word Segmenter backed by Java's BreakIterator.

    A Word Segmenter backed by Java's BreakIterator. Given an input string, it will return an iterator over sentences Doesn't return spaces, does return punctuation.

  3. case class MLSentenceSegmenter (inf: ClassificationInference) extends SentenceSegmenter with Serializable with Product with Serializable
    Annotations
    @SerialVersionUID()
  4. class NewLineSentenceSegmenter extends SentenceSegmenter
  5. case class RegexSearchTokenizer (pattern: String) extends Tokenizer with Product with Serializable

    Finds all occurrences of the given pattern in the document.

  6. case class RegexSplitTokenizer (pattern: String) extends Tokenizer with Product with Serializable

    Splits the input document according to the given pattern.

    Splits the input document according to the given pattern. Does not return the splits.

  7. class SegmentingIterator extends Iterator[Span]
  8. trait SentenceSegmenter extends StringAnalysisFunction[Any, Sentence] with (String) ⇒ Iterable[String] with Serializable

  9. class StreamSentenceSegmenter extends AnyRef

    TODO

  10. trait Tokenizer extends StringAnalysisFunction[Sentence, Token] with Serializable with (String) ⇒ IndexedSeq[String]

    Abstract trait for tokenizers, which annotate sentence-segmented text with tokens.

    Abstract trait for tokenizers, which annotate sentence-segmented text with tokens. Tokenizers work with both raw strings and epic.slab.StringSlabs.

    Annotations
    @SerialVersionUID()
  11. class TreebankTokenizer extends Tokenizer with Serializable
    Annotations
    @SerialVersionUID()
  12. class WhitespaceTokenizer extends RegexSplitTokenizer

    Tokenizes by splitting on the regular expression \s+.

Value Members

  1. def loadContent(url: URL): String
  2. def preprocess(file: File): IndexedSeq[IndexedSeq[String]]
  3. def preprocess(text: String): IndexedSeq[IndexedSeq[String]]
  4. def preprocess(url: URL): IndexedSeq[IndexedSeq[String]]
  5. def tokenize(sentence: String): IndexedSeq[String]
  6. object JavaSentenceSegmenter extends JavaSentenceSegmenter
  7. object JavaWordTokenizer extends JavaWordTokenizer
  8. object MLSentenceSegmenter extends Serializable
  9. object RegexSentenceSegmenter extends SentenceSegmenter

    A simple regex sentence segmenter.

  10. object SegmentSentences
  11. object StreamSentenceSegmenter
  12. object TextExtractor

    Just a simple thing for me to learn Tika

  13. object Textify

    TODO

  14. object TreebankTokenizer extends TreebankTokenizer
  15. object WhitespaceTokenizer extends Serializable

Inherited from AnyRef

Inherited from Any

Ungrouped