PythonTokenizer (PMD Python 7.0.0-rc4 API)

java.lang.Object
- net.sourceforge.pmd.cpd.impl.TokenizerBase<JavaccToken>
- - net.sourceforge.pmd.cpd.impl.JavaCCTokenizer
  - - net.sourceforge.pmd.lang.python.cpd.PythonTokenizer

All Implemented Interfaces:

Tokenizer
```
public class PythonTokenizer
extends JavaCCTokenizer
```
The Python tokenizer.

Constructor Summary

Constructors
Constructor Description

PythonTokenizer()

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method	Description
`protected String`	`getImage(JavaccToken token)`
`protected TokenManager<JavaccToken>`	`makeLexerImpl(TextDocument doc)`

Methods inherited from class net.sourceforge.pmd.cpd.impl.TokenizerBase
filterTokenStream, processToken, tokenize

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Constructor Detail
  - PythonTokenizer
```
public PythonTokenizer()
```
- Method Detail
  - makeLexerImpl
```
protected TokenManager<JavaccToken> makeLexerImpl(TextDocument doc)
```
    Specified by:
    
    makeLexerImpl in class TokenizerBase<JavaccToken>
  - getImage
```
protected String getImage(JavaccToken token)
```
    Overrides:
    
    getImage in class TokenizerBase<JavaccToken>