Tokenizer (Apache Jena)

java.lang.Object
- org.apache.jena.util.Tokenizer

```
public class Tokenizer
extends java.lang.Object
```
A tokenizer, similar to java's StringTokenizer but allows for quoted character strings which can include other separators.

Constructor Summary

Constructors
Constructor	Description
`Tokenizer(java.lang.String str, java.lang.String delim, java.lang.String literalDelim, boolean returnDelims)`	Constructor.

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method	Description
`boolean`	`hasMoreTokens()`	Test if there are more tokens which can be returned.
`java.lang.String`	`nextToken()`	Return the next token.

Methods inherited from class java.lang.Object
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Constructor Detail
  - Tokenizer
```
public Tokenizer(java.lang.String str,
                 java.lang.String delim,
                 java.lang.String literalDelim,
                 boolean returnDelims)
```
    Constructor.
    
    Parameters:
    
    str - the source string to be parsed
    
    delim - The set of delimiter characters
    
    literalDelim - Literal string delimiters
    
    returnDelims - If true then delimiters should be returned as tokens
- Method Detail
  - nextToken
```
public java.lang.String nextToken()
```
    Return the next token.
    
    Throws:
    
    java.util.NoSuchElementException - if there are no more tokens available
  - hasMoreTokens
```
public boolean hasMoreTokens()
```
    Test if there are more tokens which can be returned.