org.apache.druid.segment.virtual

Class NestedFieldVirtualColumn

java.lang.Object

org.apache.druid.segment.virtual.NestedFieldVirtualColumn

All Implemented Interfaces:

Cacheable, VirtualColumn

public class NestedFieldVirtualColumn
extends Object
implements VirtualColumn

Optimized virtual column that can make direct selectors into a NestedDataComplexColumn or any associated nested fields (NestedFieldDictionaryEncodedColumn) including using their indexes.

This virtual column is used for the SQL operators JSON_VALUE (if processFromRaw is set to false) or JSON_QUERY (if it is true), and accepts 'JSONPath' or 'jq' syntax string representations of paths, or a parsed list of NestedPathPart in order to determine what should be selected from the column.

Type information for nested fields is completely absent in the SQL planner, so it guesses the best it can to set expectedType from the context of how something is being used, e.g. an aggregators default type or an explicit cast, or, if using the 'RETURNING' syntax which explicitly specifies type. This might not be the same as if it had actual type information, but, we try to stick with whatever we chose there to do the best we can for now.

Since capabilities(ColumnInspector, String) is determined by the expectedType, the results will be best effor cast to the expected type if the column is not natively the expected type so that this column can fulfill the contract of the type of selector that is likely to be created to read this column.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	NestedFieldVirtualColumn.FieldDimensionSelector
static class	NestedFieldVirtualColumn.RawFieldColumnSelector Process the "raw" data to extract values with NestedPathFinder.find(Object, List), wrapping the result in StructuredData
static class	NestedFieldVirtualColumn.RawFieldLiteralColumnValueSelector Process the "raw" data to extract non-complex values.
static class	NestedFieldVirtualColumn.RawFieldVectorObjectSelector Process the "raw" data to extract vectors of values with NestedPathFinder.find(Object, List), wrapping the result in StructuredData

Constructor Summary

Constructors

Constructor and Description
NestedFieldVirtualColumn(String columnName, String outputName, ColumnType expectedType, List<NestedPathPart> parts, Boolean processFromRaw, String path, Boolean useJqSyntax)
NestedFieldVirtualColumn(String columnName, String path, String outputName)
NestedFieldVirtualColumn(String columnName, String path, String outputName, ColumnType expectedType)

Method Summary

Modifier and Type	Method and Description
boolean	canVectorize(ColumnInspector inspector)
ColumnCapabilities	capabilities(ColumnInspector inspector, String columnName) Return the ColumnCapabilities which best describe the optimal selector to read from this virtual column.
ColumnCapabilities	capabilities(String columnName) This method is deprecated in favor of VirtualColumn.capabilities(ColumnInspector, String), which should be used whenever possible and can support virtual column implementations that need to inspect other columns as inputs.
boolean	equals(Object o)
byte[]	getCacheKey() Get a byte array used as a cache key.
String	getColumnName()
ColumnType	getExpectedType()
ColumnIndexSupplier	getIndexSupplier(String columnName, ColumnSelector selector) Get the ColumnIndexSupplier for the specified virtual column, with the assistance of a ColumnSelector to allow reading things from segments.
String	getOutputName() Output name of this column.
List<NestedPathPart>	getPathParts()
int	hashCode()
boolean	isProcessFromRaw()
ColumnValueSelector<?>	makeColumnValueSelector(String columnName, ColumnSelectorFactory factory) Build a ColumnValueSelector corresponding to this virtual column.
ColumnValueSelector<?>	makeColumnValueSelector(String columnName, ColumnSelector columnSelector, ReadableOffset offset) Returns similar ColumnValueSelector object as returned by VirtualColumn.makeColumnValueSelector(String, ColumnSelectorFactory) except this method has full access to the underlying column and can potentially provide a more efficient implementation.
DimensionSelector	makeDimensionSelector(DimensionSpec dimensionSpec, ColumnSelectorFactory factory) Build a selector corresponding to this virtual column.
DimensionSelector	makeDimensionSelector(DimensionSpec dimensionSpec, ColumnSelector columnSelector, ReadableOffset offset) Returns similar DimensionSelector object as returned by VirtualColumn.makeDimensionSelector(DimensionSpec, ColumnSelectorFactory) except this method has full access to the underlying column and can potentially provide a more efficient implementation.
SingleValueDimensionVectorSelector	makeSingleValueVectorDimensionSelector(DimensionSpec dimensionSpec, ColumnSelector columnSelector, ReadableVectorOffset offset) Returns similar SingleValueDimensionVectorSelector object as returned by VirtualColumn.makeSingleValueVectorDimensionSelector(DimensionSpec, ColumnSelector, ReadableVectorOffset) except this method has full access to the underlying column and can potentially provide a more efficient implementation.
VectorObjectSelector	makeVectorObjectSelector(String columnName, ColumnSelector columnSelector, ReadableVectorOffset offset) Returns similar VectorObjectSelector object as returned by VirtualColumn.makeVectorObjectSelector(String, VectorColumnSelectorFactory) except this method has full access to the underlying column and can potentially provide a more efficient implementation.
VectorValueSelector	makeVectorValueSelector(String columnName, ColumnSelector columnSelector, ReadableVectorOffset offset) Returns similar VectorValueSelector object as returned by VirtualColumn.makeVectorValueSelector(String, VectorColumnSelectorFactory) except this method has full access to the underlying column and can potentially provide a more efficient implementation.
List<String>	requiredColumns() Returns a list of columns that this virtual column will access.
String	toString()
boolean	usesDotNotation() Indicates that this virtual column can be referenced with dot notation.

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.druid.segment.VirtualColumn

makeMultiValueVectorDimensionSelector, makeMultiValueVectorDimensionSelector, makeSingleValueVectorDimensionSelector, makeVectorObjectSelector, makeVectorValueSelector

Constructor Detail

NestedFieldVirtualColumn

public NestedFieldVirtualColumn(String columnName,
                                String outputName,
                                @Nullable
                                ColumnType expectedType,
                                @Nullable
                                List<NestedPathPart> parts,
                                @Nullable
                                Boolean processFromRaw,
                                @Nullable
                                String path,
                                @Nullable
                                Boolean useJqSyntax)

NestedFieldVirtualColumn

public NestedFieldVirtualColumn(String columnName,
                                String path,
                                String outputName)

NestedFieldVirtualColumn

public NestedFieldVirtualColumn(String columnName,
                                String path,
                                String outputName,
                                @Nullable
                                ColumnType expectedType)

Method Detail

getCacheKey

public byte[] getCacheKey()

Description copied from interface: Cacheable

Get a byte array used as a cache key.

Specified by:

getCacheKey in interface Cacheable

Returns:

a cache key

getOutputName

public String getOutputName()

Description copied from interface: VirtualColumn

Output name of this column.

Specified by:

getOutputName in interface VirtualColumn

Returns:

name

getColumnName

public String getColumnName()

getPathParts

public List<NestedPathPart> getPathParts()

getExpectedType

public ColumnType getExpectedType()

isProcessFromRaw

public boolean isProcessFromRaw()

makeDimensionSelector

public DimensionSelector makeDimensionSelector(DimensionSpec dimensionSpec,
                                               ColumnSelectorFactory factory)

Description copied from interface: VirtualColumn

Build a selector corresponding to this virtual column. Also provides the name that the virtual column was referenced with (through DimensionSpec.getDimension(), which is useful if this column uses dot notation. The virtual column is expected to apply any necessary decoration from the dimensionSpec.

Specified by:

makeDimensionSelector in interface VirtualColumn

makeColumnValueSelector

public ColumnValueSelector<?> makeColumnValueSelector(String columnName,
                                                      ColumnSelectorFactory factory)

Description copied from interface: VirtualColumn

Build a ColumnValueSelector corresponding to this virtual column. Also provides the name that the virtual column was referenced with, which is useful if this column uses dot notation.

Specified by:

makeColumnValueSelector in interface VirtualColumn

makeDimensionSelector

@Nullable
public DimensionSelector makeDimensionSelector(DimensionSpec dimensionSpec,
                                                         ColumnSelector columnSelector,
                                                         ReadableOffset offset)

Description copied from interface: VirtualColumn

Returns similar DimensionSelector object as returned by VirtualColumn.makeDimensionSelector(DimensionSpec, ColumnSelectorFactory) except this method has full access to the underlying column and can potentially provide a more efficient implementation. Users of this interface must ensure to first call this method whenever possible. Typically this can not be called in query paths on top of IncrementalIndex which doesn't have columns as in persisted segments.

Specified by:

makeDimensionSelector in interface VirtualColumn

makeColumnValueSelector

@Nullable
public ColumnValueSelector<?> makeColumnValueSelector(String columnName,
                                                                ColumnSelector columnSelector,
                                                                ReadableOffset offset)

Description copied from interface: VirtualColumn

Returns similar ColumnValueSelector object as returned by VirtualColumn.makeColumnValueSelector(String, ColumnSelectorFactory) except this method has full access to the underlying column and can potentially provide a more efficient implementation. Users of this interface must ensure to first call this method whenever possible. Typically this can not be called in query paths on top of IncrementalIndex which doesn't have columns as in persisted segments.

Specified by:

makeColumnValueSelector in interface VirtualColumn

canVectorize

public boolean canVectorize(ColumnInspector inspector)

Specified by:

canVectorize in interface VirtualColumn

makeSingleValueVectorDimensionSelector

@Nullable
public SingleValueDimensionVectorSelector makeSingleValueVectorDimensionSelector(DimensionSpec dimensionSpec,
                                                                                           ColumnSelector columnSelector,
                                                                                           ReadableVectorOffset offset)

Description copied from interface: VirtualColumn

Returns similar SingleValueDimensionVectorSelector object as returned by VirtualColumn.makeSingleValueVectorDimensionSelector(DimensionSpec, ColumnSelector, ReadableVectorOffset) except this method has full access to the underlying column and can potentially provide a more efficient implementation. Users of this interface must ensure to first call this method whenever possible.

Specified by:

makeSingleValueVectorDimensionSelector in interface VirtualColumn

makeVectorObjectSelector

@Nullable
public VectorObjectSelector makeVectorObjectSelector(String columnName,
                                                               ColumnSelector columnSelector,
                                                               ReadableVectorOffset offset)

Description copied from interface: VirtualColumn

Returns similar VectorObjectSelector object as returned by VirtualColumn.makeVectorObjectSelector(String, VectorColumnSelectorFactory) except this method has full access to the underlying column and can potentially provide a more efficient implementation. Users of this interface must ensure to first call this method whenever possible.

Specified by:

makeVectorObjectSelector in interface VirtualColumn

makeVectorValueSelector

@Nullable
public VectorValueSelector makeVectorValueSelector(String columnName,
                                                             ColumnSelector columnSelector,
                                                             ReadableVectorOffset offset)

Description copied from interface: VirtualColumn

Returns similar VectorValueSelector object as returned by VirtualColumn.makeVectorValueSelector(String, VectorColumnSelectorFactory) except this method has full access to the underlying column and can potentially provide a more efficient implementation. Users of this interface must ensure to first call this method whenever possible.

Specified by:

makeVectorValueSelector in interface VirtualColumn

getIndexSupplier

@Nullable
public ColumnIndexSupplier getIndexSupplier(String columnName,
                                                      ColumnSelector selector)

Description copied from interface: VirtualColumn

Get the ColumnIndexSupplier for the specified virtual column, with the assistance of a ColumnSelector to allow reading things from segments. If the virtual column has no indexes, this method will return null, or may also return a non-null supplier whose methods may return null values - having a supplier is no guarantee that the column has indexes.

Specified by:

getIndexSupplier in interface VirtualColumn

capabilities

public ColumnCapabilities capabilities(String columnName)

Description copied from interface: VirtualColumn

This method is deprecated in favor of VirtualColumn.capabilities(ColumnInspector, String), which should be used whenever possible and can support virtual column implementations that need to inspect other columns as inputs. This is a fallback implementation to return the capabilities of this virtual column, which includes a type that corresponds to the best performing base selector supertype (e. g. BaseLongColumnValueSelector) of the object, returned from VirtualColumn.makeColumnValueSelector(String, ColumnSelectorFactory). May vary based on columnName if this column uses dot notation.

Specified by:

capabilities in interface VirtualColumn

Parameters:

columnName - the name this virtual column was referenced with

Returns:

capabilities, must not be null

capabilities

@Nullable
public ColumnCapabilities capabilities(ColumnInspector inspector,
                                                 String columnName)

Description copied from interface: VirtualColumn

Return the ColumnCapabilities which best describe the optimal selector to read from this virtual column.

The ColumnInspector (most likely corresponding to an underlying ColumnSelectorFactory of a query) allows the virtual column to consider this information if necessary to compute its output type details.

Examples of this include the ExpressionVirtualColumn, which takes input from other columns and uses the ColumnInspector to infer the output type of expressions based on the types of the inputs.

Specified by:

capabilities in interface VirtualColumn

Parameters:

inspector - column inspector to provide additional information of other available columns

columnName - the name this virtual column was referenced with

Returns:

capabilities, must not be null

requiredColumns

public List<String> requiredColumns()

Description copied from interface: VirtualColumn

Returns a list of columns that this virtual column will access. This may include the names of other virtual columns. May be empty if a virtual column doesn't access any underlying columns. Does not pass columnName because there is an assumption that the list of columns needed by a dot-notation supporting virtual column will not vary based on the columnName.

Specified by:

requiredColumns in interface VirtualColumn

Returns:

column names

usesDotNotation

public boolean usesDotNotation()

Description copied from interface: VirtualColumn

Indicates that this virtual column can be referenced with dot notation. For example, a virtual column named "foo" could be referred to as "foo.bar" with the Cursor it is registered with. In that case, init will be called with columnName "foo.bar" rather than "foo".

Specified by:

usesDotNotation in interface VirtualColumn

Returns:

whether to use dot notation

equals

public boolean equals(Object o)

Overrides:

equals in class Object

hashCode

public int hashCode()

Overrides:

hashCode in class Object

toString

public String toString()

Overrides:

toString in class Object