org.apache.arrow.vector.complex

Class ListVector

java.lang.Object

org.apache.arrow.vector.BaseValueVector

org.apache.arrow.vector.complex.BaseRepeatedValueVector

org.apache.arrow.vector.complex.ListVector

All Implemented Interfaces:

Closeable, AutoCloseable, Iterable<ValueVector>, PromotableVector, RepeatedValueVector, FieldVector, ValueVector

public class ListVector
extends BaseRepeatedValueVector
implements FieldVector, PromotableVector

Field Summary

Fields

Modifier and Type	Field and Description
protected ArrowBuf	validityBuffer

Fields inherited from class org.apache.arrow.vector.complex.BaseRepeatedValueVector

DATA_VECTOR_NAME, DEFAULT_DATA_VECTOR, OFFSET_WIDTH, offsetAllocationSizeInBytes, offsetBuffer, valueCount, vector

Fields inherited from class org.apache.arrow.vector.BaseValueVector

allocator, INITIAL_VALUE_ALLOCATION, MAX_ALLOCATION_SIZE, MAX_ALLOCATION_SIZE_PROPERTY, name

Fields inherited from interface org.apache.arrow.vector.complex.RepeatedValueVector

DEFAULT_REPEAT_PER_RECORD

Constructor Summary

Constructors

Constructor and Description
ListVector(String name, BufferAllocator allocator, CallBack callBack) Deprecated.
ListVector(String name, BufferAllocator allocator, DictionaryEncoding dictionary, CallBack callBack) Deprecated.
ListVector(String name, BufferAllocator allocator, FieldType fieldType, CallBack callBack)

Method Summary

Methods

Modifier and Type	Method and Description
<T extends ValueVector> AddOrGetResult<T>	addOrGetVector(FieldType fieldType)
void	allocateNew() Same as allocateNewSafe().
boolean	allocateNewSafe() Allocate memory for the vector.
void	clear() Release the underlying ArrowBuf and reset the ValueVector to empty.
void	copyFrom(int inIndex, int outIndex, ListVector from) Copy a cell value from a particular index in source vector to a particular position in this vector
void	copyFromSafe(int inIndex, int outIndex, ListVector from) Same as copyFrom(int, int, ListVector) except that it handles the case when the capacity of the vector needs to be expanded before copy.
static ListVector	empty(String name, BufferAllocator allocator)
void	endValue(int index, int size) End the current value
ArrowBuf[]	getBuffers(boolean clear) Return the underlying buffers associated with this vector.
int	getBufferSize() Get the size (number of bytes) of underlying buffers used by this vector
List<FieldVector>	getChildrenFromFields() the returned list is the same size as the list passed to initializeChildrenFromFields
ArrowBuf	getDataBuffer() Gets the underlying buffer associated with data vector
long	getDataBufferAddress() Gets the starting address of the underlying buffer associated with data vector
FieldVector	getDataVector() Get the inner data vector for this list vector
Field	getField() Get information about how this field is materialized.
List<ArrowBuf>	getFieldBuffers() Get the buffers belonging to this vector
List<BufferBacked>	getFieldInnerVectors() Deprecated.
int	getLastSet()
Types.MinorType	getMinorType()
int	getNullCount() Get the number of elements that are null in the vector
Object	getObject(int index) Get the element in the list vector at a particular index
ArrowBuf	getOffsetBuffer() Gets the underlying buffer associated with offset vector
long	getOffsetBufferAddress() Gets the starting address of the underlying buffer associated with offset vector
UnionListReader	getReader()
TransferPair	getTransferPair(String ref, BufferAllocator allocator)
TransferPair	getTransferPair(String ref, BufferAllocator allocator, CallBack callBack)
ArrowBuf	getValidityBuffer() Gets the underlying buffer associated with validity vector
long	getValidityBufferAddress() Gets the starting address of the underlying buffer associated with validity vector
int	getValueCapacity() Get the current value capacity for the vector
UnionListWriter	getWriter()
void	initializeChildrenFromFields(List<Field> children) Initializes the child vectors to be later loaded with loadBuffers
boolean	isNull(int index) Check if element at given index is null.
int	isSet(int index) Same as isNull(int).
void	loadFieldBuffers(ArrowFieldNode fieldNode, List<ArrowBuf> ownBuffers) Load the buffers of this vector with provided source buffers.
TransferPair	makeTransferPair(ValueVector target) makes a new transfer pair used to transfer underlying buffers
UnionVector	promoteToUnion()
void	reAlloc() Resize the vector to increase the capacity.
void	setLastSet(int value)
void	setNotNull(int index)
void	setValueCount(int valueCount) Sets the value count for the vector
int	startNewValue(int index) Start a new value in the list vector

Methods inherited from class org.apache.arrow.vector.complex.BaseRepeatedValueVector

allocateOffsetBuffer, getBufferSizeFor, getInnerValueCount, getInnerValueCountAt, getOffsetBufferValueCapacity, getOffsetVector, getValueCount, isEmpty, iterator, reallocOffsetBuffer, replaceDataVector, setInitialCapacity, size

Methods inherited from class org.apache.arrow.vector.BaseValueVector

checkBufRefs, close, compareTypes, getAllocator, getTransferPair, getValidityBufferSizeFromCount, releaseBuffer, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.arrow.vector.ValueVector

close, getAllocator, getBufferSizeFor, getTransferPair, getValueCount, setInitialCapacity

Methods inherited from interface java.lang.Iterable

iterator

Field Detail

validityBuffer

protected ArrowBuf validityBuffer

Constructor Detail

ListVector

@Deprecated
public ListVector(String name,
                     BufferAllocator allocator,
                     CallBack callBack)

Deprecated.

ListVector

@Deprecated
public ListVector(String name,
                     BufferAllocator allocator,
                     DictionaryEncoding dictionary,
                     CallBack callBack)

Deprecated.

ListVector

public ListVector(String name,
          BufferAllocator allocator,
          FieldType fieldType,
          CallBack callBack)

Method Detail

empty

public static ListVector empty(String name,
               BufferAllocator allocator)

initializeChildrenFromFields

public void initializeChildrenFromFields(List<Field> children)

Description copied from interface: FieldVector

Initializes the child vectors to be later loaded with loadBuffers

Specified by:

initializeChildrenFromFields in interface FieldVector

Parameters:

children - the schema

getChildrenFromFields

public List<FieldVector> getChildrenFromFields()

Description copied from interface: FieldVector

the returned list is the same size as the list passed to initializeChildrenFromFields

Specified by:

getChildrenFromFields in interface FieldVector

Returns:

the children according to schema (empty for primitive types)

loadFieldBuffers

public void loadFieldBuffers(ArrowFieldNode fieldNode,
                    List<ArrowBuf> ownBuffers)

Load the buffers of this vector with provided source buffers. The caller manages the source buffers and populates them before invoking this method.

Specified by:

loadFieldBuffers in interface FieldVector

Parameters:

fieldNode - the fieldNode indicating the value count

ownBuffers - the buffers for this Field (own buffers only, children not included)

getFieldBuffers

public List<ArrowBuf> getFieldBuffers()

Get the buffers belonging to this vector

Specified by:

getFieldBuffers in interface FieldVector

Returns:

the inner buffers.

getFieldInnerVectors

@Deprecated
public List<BufferBacked> getFieldInnerVectors()

Deprecated.

Specified by:

getFieldInnerVectors in interface FieldVector

Returns:

the inner vectors for this field as defined by the TypeLayout

getWriter

public UnionListWriter getWriter()

allocateNew

public void allocateNew()
                 throws OutOfMemoryException

Same as allocateNewSafe().

Specified by:

allocateNew in interface ValueVector

Throws:

OutOfMemoryException - Thrown if no memory can be allocated.

allocateNewSafe

public boolean allocateNewSafe()

Allocate memory for the vector. We internally use a default value count of 4096 to allocate memory for at least these many elements in the vector.

Specified by:

allocateNewSafe in interface ValueVector

Overrides:

allocateNewSafe in class BaseRepeatedValueVector

Returns:

false if memory allocation fails, true otherwise.

reAlloc

public void reAlloc()

Resize the vector to increase the capacity. The internal behavior is to double the current value capacity.

Specified by:

reAlloc in interface ValueVector

Overrides:

reAlloc in class BaseRepeatedValueVector

copyFromSafe

public void copyFromSafe(int inIndex,
                int outIndex,
                ListVector from)

Same as copyFrom(int, int, ListVector) except that it handles the case when the capacity of the vector needs to be expanded before copy.

Parameters:

inIndex - position to copy from in source vector

outIndex - position to copy to in this vector

from - source vector

copyFrom

public void copyFrom(int inIndex,
            int outIndex,
            ListVector from)

Copy a cell value from a particular index in source vector to a particular position in this vector

Parameters:

inIndex - position to copy from in source vector

outIndex - position to copy to in this vector

from - source vector

getDataVector

public FieldVector getDataVector()

Get the inner data vector for this list vector

Specified by:

getDataVector in interface RepeatedValueVector

Overrides:

getDataVector in class BaseRepeatedValueVector

Returns:

data vector

getTransferPair

public TransferPair getTransferPair(String ref,
                           BufferAllocator allocator)

Specified by:

getTransferPair in interface ValueVector

getTransferPair

public TransferPair getTransferPair(String ref,
                           BufferAllocator allocator,
                           CallBack callBack)

Specified by:

getTransferPair in interface ValueVector

makeTransferPair

public TransferPair makeTransferPair(ValueVector target)

Description copied from interface: ValueVector

makes a new transfer pair used to transfer underlying buffers

Specified by:

makeTransferPair in interface ValueVector

Parameters:

target - the target for the transfer

Returns:

a new transfer pair that is used to transfer underlying buffers into the target vector.

getValidityBufferAddress

public long getValidityBufferAddress()

Description copied from interface: FieldVector

Gets the starting address of the underlying buffer associated with validity vector

Specified by:

getValidityBufferAddress in interface FieldVector

Returns:

buffer address

getDataBufferAddress

public long getDataBufferAddress()

Description copied from interface: FieldVector

Gets the starting address of the underlying buffer associated with data vector

Specified by:

getDataBufferAddress in interface FieldVector

Returns:

buffer address

getOffsetBufferAddress

public long getOffsetBufferAddress()

Description copied from interface: FieldVector

Gets the starting address of the underlying buffer associated with offset vector

Specified by:

getOffsetBufferAddress in interface FieldVector

Returns:

buffer address

getValidityBuffer

public ArrowBuf getValidityBuffer()

Description copied from interface: ValueVector

Gets the underlying buffer associated with validity vector

Specified by:

getValidityBuffer in interface ValueVector

Returns:

buffer

getDataBuffer

public ArrowBuf getDataBuffer()

Description copied from interface: ValueVector

Gets the underlying buffer associated with data vector

Specified by:

getDataBuffer in interface ValueVector

Returns:

buffer

getOffsetBuffer

public ArrowBuf getOffsetBuffer()

Description copied from interface: ValueVector

Gets the underlying buffer associated with offset vector

Specified by:

getOffsetBuffer in interface ValueVector

Returns:

buffer

getReader

public UnionListReader getReader()

Specified by:

getReader in interface ValueVector

Returns:

a field reader that supports reading values from this vector.

addOrGetVector

public <T extends ValueVector> AddOrGetResult<T> addOrGetVector(FieldType fieldType)

Specified by:

addOrGetVector in interface PromotableVector

Overrides:

addOrGetVector in class BaseRepeatedValueVector

getBufferSize

public int getBufferSize()

Get the size (number of bytes) of underlying buffers used by this vector

Specified by:

getBufferSize in interface ValueVector

Overrides:

getBufferSize in class BaseRepeatedValueVector

Returns:

size of underlying buffers.

getField

public Field getField()

Description copied from interface: ValueVector

Get information about how this field is materialized.

Specified by:

getField in interface ValueVector

Returns:

the field corresponding to this vector

getMinorType

public Types.MinorType getMinorType()

Specified by:

getMinorType in interface ValueVector

clear

public void clear()

Description copied from interface: ValueVector

Release the underlying ArrowBuf and reset the ValueVector to empty.

Specified by:

clear in interface ValueVector

Overrides:

clear in class BaseRepeatedValueVector

getBuffers

public ArrowBuf[] getBuffers(boolean clear)

Return the underlying buffers associated with this vector. Note that this doesn't impact the reference counts for this buffer so it only should be used for in-context access. Also note that this buffer changes regularly thus external classes shouldn't hold a reference to it (unless they change it).

Specified by:

getBuffers in interface ValueVector

Overrides:

getBuffers in class BaseRepeatedValueVector

Parameters:

clear - Whether to clear vector before returning; the buffers will still be refcounted but the returned array will be the only reference to them

Returns:

The underlying buffers that is used by this vector instance.

promoteToUnion

public UnionVector promoteToUnion()

Specified by:

promoteToUnion in interface PromotableVector

getObject

public Object getObject(int index)

Get the element in the list vector at a particular index

Specified by:

getObject in interface ValueVector

Parameters:

index - position of the element

Returns:

Object at given position

isNull

public boolean isNull(int index)

Check if element at given index is null.

Specified by:

isNull in interface ValueVector

Overrides:

isNull in class BaseRepeatedValueVector

Parameters:

index - position of element

Returns:

true if element at given index is null, false otherwise

isSet

public int isSet(int index)

Same as isNull(int).

Parameters:

index - position of element

Returns:

1 if element at given index is not null, 0 otherwise

getNullCount

public int getNullCount()

Get the number of elements that are null in the vector

Specified by:

getNullCount in interface ValueVector

Returns:

the number of null elements.

getValueCapacity

public int getValueCapacity()

Get the current value capacity for the vector

Specified by:

getValueCapacity in interface ValueVector

Overrides:

getValueCapacity in class BaseRepeatedValueVector

Returns:

number of elements that vector can hold.

setNotNull

public void setNotNull(int index)

startNewValue

public int startNewValue(int index)

Start a new value in the list vector

Overrides:

startNewValue in class BaseRepeatedValueVector

Parameters:

index - index of the value to start

endValue

public void endValue(int index,
            int size)

End the current value

Parameters:

index - index of the value to end

size - number of elements in the list that was written

setValueCount

public void setValueCount(int valueCount)

Sets the value count for the vector

Specified by:

setValueCount in interface ValueVector

Overrides:

setValueCount in class BaseRepeatedValueVector

Parameters:

valueCount - value count

setLastSet

public void setLastSet(int value)

getLastSet

public int getLastSet()