AbstractTokenizer (PMD Core 6.29.0 API)

java.lang.Object
- net.sourceforge.pmd.cpd.AbstractTokenizer

All Implemented Interfaces:

Tokenizer
```
@Deprecated
public abstract class AbstractTokenizer
extends Object
implements Tokenizer
```
Deprecated.
Use an AnyTokenizer instead, it's basically as powerful.

Author:

Zev Blut [email protected], Romain PELISSE [email protected]

Field Summary

Fields
Modifier and Type	Field	Description
`protected List<String>`	`ignorableCharacter`	Deprecated.
`protected List<String>`	`ignorableStmt`	Deprecated.
`protected char`	`oneLineCommentChar`	Deprecated.
`protected Character`	`spanMultipleLinesLineContinuationCharacter`	Deprecated.
`protected boolean`	`spanMultipleLinesString`	Deprecated.
`protected List<String>`	`stringToken`	Deprecated.

Fields inherited from interface net.sourceforge.pmd.cpd.Tokenizer
DEFAULT_SKIP_BLOCKS_PATTERN, IGNORE_ANNOTATIONS, IGNORE_IDENTIFIERS, IGNORE_LITERALS, IGNORE_USINGS, OPTION_SKIP_BLOCKS, OPTION_SKIP_BLOCKS_PATTERN

Constructor Summary

Constructors
Constructor Description

AbstractTokenizer()
Deprecated.

Method Summary

All Methods Instance Methods Concrete Methods Deprecated Methods
Modifier and Type Method Description

void tokenize(SourceCode tokens, Tokens tokenEntries)
Deprecated.
- Methods inherited from class java.lang.Object
  clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

stringToken
```
protected List<String> stringToken
```
Deprecated.

ignorableCharacter

protected List<String> ignorableCharacter

Deprecated.

ignorableStmt
```
protected List<String> ignorableStmt
```
Deprecated.

oneLineCommentChar
```
protected char oneLineCommentChar
```
Deprecated.

spanMultipleLinesString

protected boolean spanMultipleLinesString

Deprecated.

spanMultipleLinesLineContinuationCharacter

protected Character spanMultipleLinesLineContinuationCharacter

Deprecated.

Constructor Detail
- AbstractTokenizer
```
public AbstractTokenizer()
```
  Deprecated.

Method Detail

tokenize

public void tokenize(SourceCode tokens,
                     Tokens tokenEntries)

Deprecated.

Specified by:: tokenize in interface Tokenizer