org.apache.arrow.c

Class Data

java.lang.Object

org.apache.arrow.c.Data

public final class Data
extends Object

Functions for working with the C data interface.

This API is EXPERIMENTAL. Note that currently only 64bit systems are supported.

Method Summary

Modifier and Type	Method and Description
static void	exportArrayStream(BufferAllocator allocator, ArrowReader reader, ArrowArrayStream out) Export a reader as an ArrowArrayStream using the C Stream Interface.
static void	exportField(BufferAllocator allocator, Field field, DictionaryProvider provider, ArrowSchema out) Export Java Field using the C data interface format.
static void	exportSchema(BufferAllocator allocator, Schema schema, DictionaryProvider provider, ArrowSchema out) Export Java Schema using the C data interface format.
static void	exportTable(BufferAllocator allocator, Table table, ArrowArray out) Export the current contents of a Java Table using the C data interface format.
static void	exportTable(BufferAllocator allocator, Table table, DictionaryProvider provider, ArrowArray out) Export the current contents of a Java Table using the C data interface format.
static void	exportTable(BufferAllocator allocator, Table table, DictionaryProvider provider, ArrowArray out, ArrowSchema outSchema) Export the current contents of a Java Table using the C data interface format.
static void	exportVector(BufferAllocator allocator, FieldVector vector, DictionaryProvider provider, ArrowArray out) Export Java FieldVector using the C data interface format.
static void	exportVector(BufferAllocator allocator, FieldVector vector, DictionaryProvider provider, ArrowArray out, ArrowSchema outSchema) Export Java FieldVector using the C data interface format.
static void	exportVectorSchemaRoot(BufferAllocator allocator, VectorSchemaRoot vsr, DictionaryProvider provider, ArrowArray out) Export the current contents of a Java VectorSchemaRoot using the C data interface format.
static void	exportVectorSchemaRoot(BufferAllocator allocator, VectorSchemaRoot vsr, DictionaryProvider provider, ArrowArray out, ArrowSchema outSchema) Export the current contents of a Java VectorSchemaRoot using the C data interface format.
static ArrowReader	importArrayStream(BufferAllocator allocator, ArrowArrayStream stream) Import an ArrowArrayStream as an ArrowReader.
static Field	importField(BufferAllocator allocator, ArrowSchema schema, CDataDictionaryProvider provider) Import Java Field from the C data interface.
static void	importIntoVector(BufferAllocator allocator, ArrowArray array, FieldVector vector, DictionaryProvider provider) Import Java vector from the C data interface.
static void	importIntoVectorSchemaRoot(BufferAllocator allocator, ArrowArray array, VectorSchemaRoot root, DictionaryProvider provider) Import record batch from the C data interface into vector schema root.
static Schema	importSchema(BufferAllocator allocator, ArrowSchema schema, CDataDictionaryProvider provider) Import Java Schema from the C data interface.
static FieldVector	importVector(BufferAllocator allocator, ArrowArray array, ArrowSchema schema, CDataDictionaryProvider provider) Import Java vector and its type from the C data interface.
static VectorSchemaRoot	importVectorSchemaRoot(BufferAllocator allocator, ArrowArray array, ArrowSchema schema, CDataDictionaryProvider provider) Import Java vector schema root from the C data interface.
static VectorSchemaRoot	importVectorSchemaRoot(BufferAllocator allocator, ArrowSchema schema, CDataDictionaryProvider provider) Import Java vector schema root from a C data interface Schema.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

exportField

public static void exportField(BufferAllocator allocator,
                               Field field,
                               DictionaryProvider provider,
                               ArrowSchema out)

Export Java Field using the C data interface format.

Parameters:

allocator - Buffer allocator for allocating C data interface fields

field - Field object to export

provider - Dictionary provider for dictionary encoded fields (optional)

out - C struct where to export the field

exportSchema

public static void exportSchema(BufferAllocator allocator,
                                Schema schema,
                                DictionaryProvider provider,
                                ArrowSchema out)

Export Java Schema using the C data interface format.

Parameters:

allocator - Buffer allocator for allocating C data interface fields

schema - Schema object to export

provider - Dictionary provider for dictionary encoded fields (optional)

out - C struct where to export the field

exportVector

public static void exportVector(BufferAllocator allocator,
                                FieldVector vector,
                                DictionaryProvider provider,
                                ArrowArray out)

Export Java FieldVector using the C data interface format.

The resulting ArrowArray struct keeps the array data and buffers alive until its release callback is called by the consumer.

Parameters:

allocator - Buffer allocator for allocating C data interface fields

vector - Vector object to export

provider - Dictionary provider for dictionary encoded vectors (optional)

out - C struct where to export the array

exportVector

public static void exportVector(BufferAllocator allocator,
                                FieldVector vector,
                                DictionaryProvider provider,
                                ArrowArray out,
                                ArrowSchema outSchema)

Export Java FieldVector using the C data interface format.

The resulting ArrowArray struct keeps the array data and buffers alive until its release callback is called by the consumer.

Parameters:

allocator - Buffer allocator for allocating C data interface fields

vector - Vector object to export

provider - Dictionary provider for dictionary encoded vectors (optional)

out - C struct where to export the array

outSchema - C struct where to export the array type (optional)

exportTable

public static void exportTable(BufferAllocator allocator,
                               Table table,
                               ArrowArray out)

Export the current contents of a Java Table using the C data interface format.

The table is exported as if it were a struct array. The resulting ArrowArray struct keeps the record batch data and buffers alive until its release callback is called by the consumer.

Parameters:

allocator - Buffer allocator for allocating C data interface fields

table - Table to export

out - C struct where to export the record batch

exportTable

public static void exportTable(BufferAllocator allocator,
                               Table table,
                               DictionaryProvider provider,
                               ArrowArray out)

Export the current contents of a Java Table using the C data interface format.

The table is exported as if it were a struct array. The resulting ArrowArray struct keeps the record batch data and buffers alive until its release callback is called by the consumer.

Parameters:

allocator - Buffer allocator for allocating C data interface fields

table - Table to export

provider - Dictionary provider for dictionary encoded vectors (optional)

out - C struct where to export the record batch

exportTable

public static void exportTable(BufferAllocator allocator,
                               Table table,
                               DictionaryProvider provider,
                               ArrowArray out,
                               ArrowSchema outSchema)

Export the current contents of a Java Table using the C data interface format.

The table is exported as if it were a struct array. The resulting ArrowArray struct keeps the record batch data and buffers alive until its release callback is called by the consumer.

Parameters:

allocator - Buffer allocator for allocating C data interface fields

table - Table to export

provider - Dictionary provider for dictionary encoded vectors (optional)

out - C struct where to export the record batch

outSchema - C struct where to export the record batch schema (optional)

exportVectorSchemaRoot

public static void exportVectorSchemaRoot(BufferAllocator allocator,
                                          VectorSchemaRoot vsr,
                                          DictionaryProvider provider,
                                          ArrowArray out)

Export the current contents of a Java VectorSchemaRoot using the C data interface format.

The vector schema root is exported as if it were a struct array. The resulting ArrowArray struct keeps the record batch data and buffers alive until its release callback is called by the consumer.

Parameters:

allocator - Buffer allocator for allocating C data interface fields

vsr - Vector schema root to export

provider - Dictionary provider for dictionary encoded vectors (optional)

out - C struct where to export the record batch

exportVectorSchemaRoot

public static void exportVectorSchemaRoot(BufferAllocator allocator,
                                          VectorSchemaRoot vsr,
                                          DictionaryProvider provider,
                                          ArrowArray out,
                                          ArrowSchema outSchema)

Export the current contents of a Java VectorSchemaRoot using the C data interface format.

The vector schema root is exported as if it were a struct array. The resulting ArrowArray struct keeps the record batch data and buffers alive until its release callback is called by the consumer.

Parameters:

allocator - Buffer allocator for allocating C data interface fields

vsr - Vector schema root to export

provider - Dictionary provider for dictionary encoded vectors (optional)

out - C struct where to export the record batch

outSchema - C struct where to export the record batch schema (optional)

exportArrayStream

public static void exportArrayStream(BufferAllocator allocator,
                                     ArrowReader reader,
                                     ArrowArrayStream out)

Export a reader as an ArrowArrayStream using the C Stream Interface.

Parameters:

allocator - Buffer allocator for allocating C data interface fields

reader - Reader to export

out - C struct to export the stream

importField

public static Field importField(BufferAllocator allocator,
                                ArrowSchema schema,
                                CDataDictionaryProvider provider)

Import Java Field from the C data interface.

The given ArrowSchema struct is released (as per the C data interface specification), even if this function fails.

Parameters:

allocator - Buffer allocator for allocating dictionary vectors

schema - C data interface struct representing the field [inout]

provider - A dictionary provider will be initialized with empty dictionary vectors (optional)

Returns:

Imported field object

importSchema

public static Schema importSchema(BufferAllocator allocator,
                                  ArrowSchema schema,
                                  CDataDictionaryProvider provider)

Import Java Schema from the C data interface.

The given ArrowSchema struct is released (as per the C data interface specification), even if this function fails.

Parameters:

allocator - Buffer allocator for allocating dictionary vectors

schema - C data interface struct representing the field

provider - A dictionary provider will be initialized with empty dictionary vectors (optional)

Returns:

Imported schema object

importIntoVector

public static void importIntoVector(BufferAllocator allocator,
                                    ArrowArray array,
                                    FieldVector vector,
                                    DictionaryProvider provider)

Import Java vector from the C data interface.

The ArrowArray struct has its contents moved (as per the C data interface specification) to a private object held alive by the resulting array.

Parameters:

allocator - Buffer allocator

array - C data interface struct holding the array data

vector - Imported vector object [out]

provider - Dictionary provider to load dictionary vectors to (optional)

importVector

public static FieldVector importVector(BufferAllocator allocator,
                                       ArrowArray array,
                                       ArrowSchema schema,
                                       CDataDictionaryProvider provider)

Import Java vector and its type from the C data interface.

The ArrowArray struct has its contents moved (as per the C data interface specification) to a private object held alive by the resulting vector. The ArrowSchema struct is released, even if this function fails.

Parameters:

allocator - Buffer allocator for allocating the output FieldVector

array - C data interface struct holding the array data

schema - C data interface struct holding the array type

provider - Dictionary provider to load dictionary vectors to (optional)

Returns:

Imported vector object

importIntoVectorSchemaRoot

public static void importIntoVectorSchemaRoot(BufferAllocator allocator,
                                              ArrowArray array,
                                              VectorSchemaRoot root,
                                              DictionaryProvider provider)

Import record batch from the C data interface into vector schema root. The ArrowArray struct has its contents moved (as per the C data interface specification) to a private object held alive by the resulting vector schema root. The schema of the vector schema root must match the input array (undefined behavior otherwise).

Parameters:

allocator - Buffer allocator

array - C data interface struct holding the record batch data

root - vector schema root to load into

provider - Dictionary provider to load dictionary vectors to (optional)

importVectorSchemaRoot

public static VectorSchemaRoot importVectorSchemaRoot(BufferAllocator allocator,
                                                      ArrowSchema schema,
                                                      CDataDictionaryProvider provider)

Import Java vector schema root from a C data interface Schema. The type represented by the ArrowSchema struct must be a struct type array. The ArrowSchema struct is released, even if this function fails.

Parameters:

allocator - Buffer allocator for allocating the output VectorSchemaRoot

schema - C data interface struct holding the record batch schema

provider - Dictionary provider to load dictionary vectors to (optional)

Returns:

Imported vector schema root

importVectorSchemaRoot

public static VectorSchemaRoot importVectorSchemaRoot(BufferAllocator allocator,
                                                      ArrowArray array,
                                                      ArrowSchema schema,
                                                      CDataDictionaryProvider provider)

Import Java vector schema root from the C data interface. The type represented by the ArrowSchema struct must be a struct type array. The ArrowArray struct has its contents moved (as per the C data interface specification) to a private object held alive by the resulting record batch. The ArrowSchema struct is released, even if this function fails. Prefer importIntoVectorSchemaRoot(org.apache.arrow.memory.BufferAllocator, org.apache.arrow.c.ArrowArray, org.apache.arrow.vector.VectorSchemaRoot, org.apache.arrow.vector.dictionary.DictionaryProvider) for loading array data while reusing the same vector schema root.

Parameters:

allocator - Buffer allocator for allocating the output VectorSchemaRoot

array - C data interface struct holding the record batch data (optional)

schema - C data interface struct holding the record batch schema

provider - Dictionary provider to load dictionary vectors to (optional)

Returns:

Imported vector schema root

importArrayStream

public static ArrowReader importArrayStream(BufferAllocator allocator,
                                            ArrowArrayStream stream)

Import an ArrowArrayStream as an ArrowReader.

Parameters:

allocator - Buffer allocator for allocating the output data.

stream - C stream interface struct to import.

Returns:

Imported reader