hepple.postag

Class POSTagger

java.lang.Object

hepple.postag.POSTagger

public class POSTagger
extends Object

A Java POS Tagger Author: Mark Hepple ([email protected]) Input: An ascii text file in "Brill input format", i.e. one sentence per line, tokens separated by spaces. Output: Same text with each token tagged, i.e. "token" -> "token/tag". Output is just streamed to std-output, so commonly will direct into some target file. Revision: 13/9/00. Version 1.0. Comments: Implements a version of the decision list based tagging method described in: M. Hepple. 2000. Independence and Commitment: Assumptions for Rapid Training and Execution of Rule-based Part-of-Speech Taggers. Proceedings of the 38th Annual Meeting of the Association for Computational Linguistics (ACL-2000). Hong Kong, October 2000. Modified by Niraj Aswani/Ian Roberts to allow explicit specification of the character encoding to use when reading rules and lexicon files. $Id: POSTagger.java 17605 2014-03-09 10:15:34Z markagreenwood $

Field Summary

Fields

Modifier and Type	Field and Description
String[][]	lexBuff
protected Map<String,List<Rule>>	rules
String[]	tagBuff
String[]	wordBuff

Constructor Summary

Constructors

Constructor and Description
POSTagger(URL lexiconURL, URL rulesURL) Construct a POS tagger using the platform's native encoding to read the lexicon and rules files.
POSTagger(URL lexiconURL, URL rulesURL, String encoding) Construct a POS tagger using the specified encoding to read the lexicon and rules files.

Method Summary

Methods

Modifier and Type	Method and Description
protected String[]	classifyWord(String wd) Attempts to classify an unknown word.
Rule	createNewRule(String ruleId) Creates a new rule of the required type according to the provided ID.
static void	main(String[] args) Main method.
protected boolean	oneStep(String word, List<String[]> taggedSentence) Adds a new word to the window of 7 words (on the last position) and tags the word currently in the middle (i.e. on position 3).
void	readRules(URL rulesURL) Reads the rules from the rules input file
List<List<String[]>>	runTagger(List<List<String>> sentences) Runs the tagger over a set of sentences.
void	showRules()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

rules

protected Map<String,List<Rule>> rules

wordBuff

public String[] wordBuff

tagBuff

public String[] tagBuff

lexBuff

public String[][] lexBuff

Constructor Detail

POSTagger

public POSTagger(URL lexiconURL,
         URL rulesURL)
          throws InvalidRuleException,
                 IOException

Construct a POS tagger using the platform's native encoding to read the lexicon and rules files.

Throws:

InvalidRuleException

IOException

POSTagger

public POSTagger(URL lexiconURL,
         URL rulesURL,
         String encoding)
          throws InvalidRuleException,
                 IOException

Construct a POS tagger using the specified encoding to read the lexicon and rules files.

Throws:

InvalidRuleException

IOException

Method Detail

createNewRule

public Rule createNewRule(String ruleId)
                   throws InvalidRuleException

Creates a new rule of the required type according to the provided ID.

Parameters:

ruleId - the ID for the rule to be created

Throws:

InvalidRuleException

runTagger

public List<List<String[]>> runTagger(List<List<String>> sentences)

Runs the tagger over a set of sentences.

Parameters:

sentences - a List of Lists of words to be tagged. Each list is a sentence represented as a list of words.

Returns:

a List of Lists of String[]. A list of tagged sentences, each sentence being itself a list having pairs of strings as elements with the word on the first position and the tag on the second.

oneStep

protected boolean oneStep(String word,
              List<String[]> taggedSentence)

Adds a new word to the window of 7 words (on the last position) and tags the word currently in the middle (i.e. on position 3). This function also reads the word on the first position and adds its tag to the taggedSentence structure as this word would be lost at the next advance. If this word completes a sentence then it returns true otherwise it returns false.

Parameters:

word - the new word

taggedSentence - a List of pairs of strings representing the results of tagging the current sentence so far.

Returns:

returns true if a full sentence is now tagged, otherwise false.

readRules

public void readRules(URL rulesURL)
               throws IOException,
                      InvalidRuleException

Reads the rules from the rules input file

Throws:

IOException

InvalidRuleException

showRules

public void showRules()

classifyWord

protected String[] classifyWord(String wd)

Attempts to classify an unknown word.

Parameters:

wd - the word to be classified

main

public static void main(String[] args)

Main method. Runs the tagger using the arguments to find the resources to be used for initialisation and the input file.