org.apache.hadoop.hive.ql.exec.vector

Class BytesColumnVector

java.lang.Object

org.apache.hadoop.hive.ql.exec.vector.ColumnVector

org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector

public class BytesColumnVector
extends ColumnVector

This class supports string and binary data by value reference -- i.e. each field is explicitly present, as opposed to provided by a dictionary reference. In some cases, all the values will be in the same byte array to begin with, but this need not be the case. If each value is in a separate byte array to start with, or not all of the values are in the same original byte array, you can still assign data by reference into this column vector. This gives flexibility to use this in multiple situations.

When setting data by reference, the caller is responsible for allocating the byte arrays used to hold the data. You can also set data by value, as long as you call the initBuffer() method first. You can mix "by value" and "by reference" in the same column vector, though that use is probably not typical.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.hive.ql.exec.vector.ColumnVector

ColumnVector.Type

Field Summary

Fields

Modifier and Type	Field and Description
int[]	length
int[]	start
byte[][]	vector

Fields inherited from class org.apache.hadoop.hive.ql.exec.vector.ColumnVector

isNull, isRepeating, noNulls, type

Constructor Summary

Constructors

Constructor and Description
BytesColumnVector() Use this constructor for normal operation.
BytesColumnVector(int size) Don't call this constructor except for testing purposes.

Method Summary

Modifier and Type	Method and Description
int	bufferSize()
void	copySelected(boolean selectedInUse, int[] sel, int size, ColumnVector outputColVector) Copy the current object contents into the output.
void	ensureSize(int size, boolean preserveData) Ensure the ColumnVector can hold at least size values.
void	ensureValPreallocated(int length) Ensures that we have space allocated for the next value, which has size length bytes.
void	fill(byte[] value)
void	fillWithNulls()
void	flatten(boolean selectedInUse, int[] sel, int size) Simplify vector by brute-force flattening noNulls and isRepeating This can be used to reduce combinatorial explosion of code paths in VectorExpressions with many arguments, at the expense of loss of some performance.
byte[]	getValPreallocatedBytes()
int	getValPreallocatedStart()
void	init() Initialize the column vector.
void	initBuffer() Initialize buffer to default size.
void	initBuffer(int estimatedValueSize) You must call initBuffer first before using setVal().
void	reset() Additional reset work for BytesColumnVector (releasing scratch bytes for by value strings).
void	setConcat(int elementNum, byte[] leftSourceBuf, int leftStart, int leftLen, byte[] rightSourceBuf, int rightStart, int rightLen) Set a field to the concatenation of two string values.
void	setElement(int outputElementNum, int inputElementNum, ColumnVector inputColVector) Set the element in this column vector from the given input vector.
void	setRef(int elementNum, byte[] sourceBuf, int start, int length) Set a field by reference.
void	setVal(int elementNum, byte[] sourceBuf) Set a field by actually copying in to a local buffer.
void	setVal(int elementNum, byte[] sourceBuf, int start, int length) Set a field by actually copying in to a local buffer.
void	setValPreallocated(int elementNum, int length) Set the length of the preallocated values bytes used.
void	shallowCopyTo(ColumnVector otherCv) Shallow copy of the contents of this vector to the other vector; replaces other vector's values.
void	stringifyValue(StringBuilder buffer, int row) Print the value for this column into the given string builder.
String	toString(int row)

Methods inherited from class org.apache.hadoop.hive.ql.exec.vector.ColumnVector

decRef, flattenNoNulls, flattenPush, flattenRepeatingNulls, getRef, incRef, setRepeating, unFlatten

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

vector

public byte[][] vector

start

public int[] start

length

public int[] length

Constructor Detail

BytesColumnVector

public BytesColumnVector()

Use this constructor for normal operation. All column vectors should be the default size normally.

BytesColumnVector

public BytesColumnVector(int size)

Don't call this constructor except for testing purposes.

Parameters:

size - number of elements in the column vector

Method Detail

reset

public void reset()

Additional reset work for BytesColumnVector (releasing scratch bytes for by value strings).

Overrides:

reset in class ColumnVector

setRef

public void setRef(int elementNum,
                   byte[] sourceBuf,
                   int start,
                   int length)

Set a field by reference. This is a FAST version that assumes the caller has checked to make sure the sourceBuf is not null and elementNum is correctly adjusted for isRepeating. And, that the isNull entry has been set. Only the output entry fields will be set by this method.

Parameters:

elementNum - index within column vector to set

sourceBuf - container of source data

start - start byte position within source

length - length of source byte sequence

initBuffer

public void initBuffer(int estimatedValueSize)

You must call initBuffer first before using setVal(). Provide the estimated number of bytes needed to hold a full column vector worth of byte string data.

Parameters:

estimatedValueSize - Estimated size of buffer space needed per row

initBuffer

public void initBuffer()

Initialize buffer to default size.

bufferSize

public int bufferSize()

Returns:

amount of buffer space currently allocated

setVal

public void setVal(int elementNum,
                   byte[] sourceBuf,
                   int start,
                   int length)

Set a field by actually copying in to a local buffer. If you must actually copy data in to the array, use this method. DO NOT USE this method unless it's not practical to set data by reference with setRef(). Setting data by reference tends to run a lot faster than copying data in. This is a FAST version that assumes the caller has checked to make sure the sourceBuf is not null and elementNum is correctly adjusted for isRepeating. And, that the isNull entry has been set. Only the output entry fields will be set by this method.

Parameters:

elementNum - index within column vector to set

sourceBuf - container of source data

start - start byte position within source

length - length of source byte sequence

setVal

public void setVal(int elementNum,
                   byte[] sourceBuf)

Set a field by actually copying in to a local buffer. If you must actually copy data in to the array, use this method. DO NOT USE this method unless it's not practical to set data by reference with setRef(). Setting data by reference tends to run a lot faster than copying data in. This is a FAST version that assumes the caller has checked to make sure the sourceBuf is not null and elementNum is correctly adjusted for isRepeating. And, that the isNull entry has been set. Only the output entry fields will be set by this method.

Parameters:

elementNum - index within column vector to set

sourceBuf - container of source data

ensureValPreallocated

public void ensureValPreallocated(int length)

Ensures that we have space allocated for the next value, which has size length bytes. Updates currentValue and currentOffset for this value. Always use before getValPreallocatedBytes, getValPreallocatedStart. setValPreallocated must be called to actually reserve the bytes.

getValPreallocatedBytes

public byte[] getValPreallocatedBytes()

getValPreallocatedStart

public int getValPreallocatedStart()

setValPreallocated

public void setValPreallocated(int elementNum,
                               int length)

Set the length of the preallocated values bytes used.

Parameters:

elementNum -

length -

setConcat

public void setConcat(int elementNum,
                      byte[] leftSourceBuf,
                      int leftStart,
                      int leftLen,
                      byte[] rightSourceBuf,
                      int rightStart,
                      int rightLen)

Set a field to the concatenation of two string values. Result data is copied into the internal buffer.

Parameters:

elementNum - index within column vector to set

leftSourceBuf - container of left argument

leftStart - start of left argument

leftLen - length of left argument

rightSourceBuf - container of right argument

rightStart - start of right argument

rightLen - length of right arugment

copySelected

public void copySelected(boolean selectedInUse,
                         int[] sel,
                         int size,
                         ColumnVector outputColVector)

Copy the current object contents into the output. Only copy selected entries, as indicated by selectedInUse and the sel array.

Specified by:

copySelected in class ColumnVector

flatten

public void flatten(boolean selectedInUse,
                    int[] sel,
                    int size)

Simplify vector by brute-force flattening noNulls and isRepeating This can be used to reduce combinatorial explosion of code paths in VectorExpressions with many arguments, at the expense of loss of some performance.

Specified by:

flatten in class ColumnVector

fill

public void fill(byte[] value)

fillWithNulls

public void fillWithNulls()

setElement

public void setElement(int outputElementNum,
                       int inputElementNum,
                       ColumnVector inputColVector)

Set the element in this column vector from the given input vector. The inputElementNum will be adjusted to 0 if the input column has isRepeating set. On the other hand, the outElementNum must have been adjusted to 0 in ADVANCE when the output has isRepeating set. IMPORTANT: if the output entry is marked as NULL, this method will do NOTHING. This supports the caller to do output NULL processing in advance that may cause the output results operation to be ignored. Thus, make sure the output isNull entry is set in ADVANCE. The inputColVector noNulls and isNull entry will be examined. The output will only be set if the input is NOT NULL. I.e. noNulls || !isNull[inputElementNum] where inputElementNum may have been adjusted to 0 for isRepeating. If the input entry is NULL or out-of-range, the output will be marked as NULL. I.e. set output noNull = false and isNull[outElementNum] = true. An example of out-of-range is the DecimalColumnVector which can find the input decimal does not fit in the output precision/scale. (Since we return immediately if the output entry is NULL, we have no need and do not mark the output entry to NOT NULL).

Specified by:

setElement in class ColumnVector

init

public void init()

Description copied from class: ColumnVector

Initialize the column vector. This method can be overridden by specific column vector types. Use this method only if the individual type of the column vector is not known, otherwise its preferable to call specific initialization methods.

Overrides:

init in class ColumnVector

toString

public String toString(int row)

stringifyValue

public void stringifyValue(StringBuilder buffer,
                           int row)

Description copied from class: ColumnVector

Print the value for this column into the given string builder.

Specified by:

stringifyValue in class ColumnVector

Parameters:

buffer - the buffer to print into

row - the id of the row to print

ensureSize

public void ensureSize(int size,
                       boolean preserveData)

Description copied from class: ColumnVector

Ensure the ColumnVector can hold at least size values. This method is deliberately *not* recursive because the complex types can easily have more (or less) children than the upper levels.

Overrides:

ensureSize in class ColumnVector

Parameters:

size - the new minimum size

preserveData - should the old data be preserved?

shallowCopyTo

public void shallowCopyTo(ColumnVector otherCv)

Description copied from class: ColumnVector

Shallow copy of the contents of this vector to the other vector; replaces other vector's values.

Overrides:

shallowCopyTo in class ColumnVector