Package org.apache.nifi.stream.io.util

Class StreamDemarcator

java.lang.Object

org.apache.nifi.stream.io.util.AbstractDemarcator

org.apache.nifi.stream.io.util.StreamDemarcator

All Implemented Interfaces:

Closeable, AutoCloseable

public class StreamDemarcator
extends AbstractDemarcator

The StreamDemarcator class takes an input stream and demarcates it so it could be read (see nextToken()) as individual byte[] demarcated by the provided delimiter (see 'delimiterBytes'). If delimiter is not provided the entire stream will be read into a single token which may result in OutOfMemoryError if stream is too large. The 'maxDataSize' controls the maximum size of the buffer that accumulates a token.

NOTE: Not intended for multi-thread usage hence not Thread-safe.

Field Summary

Fields

Modifier and Type	Field	Description
private final byte[]	delimiterBytes

Fields inherited from class org.apache.nifi.stream.io.util.AbstractDemarcator

availableBytesLength, buffer, index, INIT_BUFFER_SIZE, mark, offset

Constructor Summary

Constructors

Constructor	Description
StreamDemarcator(InputStream is, byte[] delimiterBytes, int maxDataSize)	Constructs a new instance
StreamDemarcator(InputStream is, byte[] delimiterBytes, int maxDataSize, int initialBufferSize)	Constructs a new instance

Method Summary

Modifier and Type	Method	Description
byte[]	nextToken()	Will read the next data token from the InputStream returning null when it reaches the end of the stream.
private void	validate(byte[] delimiterBytes)	Validates prerequisites for constructor arguments

Methods inherited from class org.apache.nifi.stream.io.util.AbstractDemarcator

close, extractDataToken, fill

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

delimiterBytes

private final byte[] delimiterBytes

Constructor Details

StreamDemarcator

public StreamDemarcator(InputStream is,
 byte[] delimiterBytes,
 int maxDataSize)

Constructs a new instance

Parameters:

is - instance of InputStream representing the data

delimiterBytes - byte array representing delimiter bytes used to split the input stream. Can be 'null'. NOTE: the 'null' is allowed only for convenience and consistency since without delimiter this class is no different then BufferedReader which reads the entire stream into a byte array and there may be a more efficient ways to do that (if that is the case).

maxDataSize - maximum size of data derived from the input stream. This means that neither InputStream nor its individual tokens (if delimiter is used) can ever be greater then this size.

StreamDemarcator

public StreamDemarcator(InputStream is,
 byte[] delimiterBytes,
 int maxDataSize,
 int initialBufferSize)

Constructs a new instance

Parameters:

is - instance of InputStream representing the data

delimiterBytes - byte array representing delimiter bytes used to split the input stream. Can be 'null'. NOTE: the 'null' is allowed only for convenience and consistency since without delimiter this class is no different then BufferedReader which reads the entire stream into a byte array and there may be a more efficient ways to do that (if that is the case).

maxDataSize - maximum size of data derived from the input stream. This means that neither InputStream nor its individual tokens (if delimiter is used) can ever be greater then this size.

initialBufferSize - initial size of the buffer used to buffer InputStream or its parts (if delimiter is used) to create its byte[] representation. Must be positive integer. The buffer will grow automatically as needed up to the Integer.MAX_VALUE;

Method Details

nextToken

public byte[] nextToken()
                 throws IOException

Will read the next data token from the InputStream returning null when it reaches the end of the stream.

Throws:

IOException - if unable to read from the stream

validate

private void validate(byte[] delimiterBytes)

Validates prerequisites for constructor arguments