Consisting of a top level Index interface and two sub-interfaces which handle read and write operations,
Searcher and Indexer respectively, this defines a secondary index implementation.
Instantiation is done via reflection and implementations must provide a constructor which takes the base
table's ColumnFamilyStore and the IndexMetadata which defines the Index as arguments. e.g:
MyCustomIndex( ColumnFamilyStore baseCfs, IndexMetadata indexDef )
The main interface defines methods for index management, index selection at both write and query time,
as well as validation of values that will ultimately be indexed.
Two sub-interfaces are also defined, which represent single use helpers for short lived tasks at read and write time.
Indexer: an event listener which receives notifications at particular points during an update of a single partition
in the base table.
Searcher: performs queries against the index based on a predicate defined in a RowFilter. An instance
is expected to be single use, being involved in the execution of a single ReadCommand.
The main interface includes factory methods for obtaining instances of both of the sub-interfaces;
The methods defined in the top level interface can be grouped into 3 categories:
Management Tasks:
This group of methods is primarily concerned with maintenance of secondary indexes are are mainly called from
SecondaryIndexManager. It includes methods for registering and un-registering an index, performing maintenance
tasks such as (re)building an index from SSTable data, flushing, invalidating and so forth, as well as some to
retrieve general metadata about the index (index name, any internal tables used for persistence etc).
Several of these maintenance functions have a return type of Callable; the expectation for these methods is
that any work required to be performed by the method be done inside the Callable so that the responsibility for
scheduling its execution can rest with SecondaryIndexManager. For instance, a task like reloading index metadata
following potential updates caused by modifications to the base table may be performed in a blocking way. In
contrast, adding a new index may require it to be built from existing SSTable data, a potentially expensive task
which should be performed asyncronously.
Index Selection:
There are two facets to index selection, write time and read time selection. The former is concerned with
identifying whether an index should be informed about a particular write operation. The latter is about providing
means to use the index for search during query execution.
Validation:
Values that may be written to an index are checked as part of input validation, prior to an update or insert
operation being accepted.
Sub-interfaces:
Update processing:
Indexes are subscribed to the stream of events generated by modifications to the base table. Subscription is
done via first registering the Index with the base table's SecondaryIndexManager. For each partition update, the set
of registered indexes are then filtered based on the properties of the update using the selection methods on the main
interface described above. Each of the indexes in the filtered set then provides an event listener to receive
notifications about the update as it is processed. As such then, a event handler instance is scoped to a single
partition update; SecondaryIndexManager obtains a new handler for every update it processes (via a call to the
factory method, indexerFor. That handler will then receive all events for the update, before being
discarded by the SecondaryIndexManager. Indexer instances are never re-used by SecondaryIndexManager and the
expectation is that each call to indexerFor should return a unique instance, or at least if instances can
be recycled, that a given instance is only used to process a single partition update at a time.
Search:
Each query (i.e. a single ReadCommand) that uses indexes will use a single instance of Index.Searcher. As with
processing of updates, an Index must be registered with the primary table's SecondaryIndexManager to be able to
support queries. During the processing of a ReadCommand, the Expressions in its RowFilter are examined to determine
whether any of them are supported by a registered Index. supportsExpression is used to filter out Indexes which
cannot support a given Expression. After filtering, the set of candidate indexes are ranked according to the result
of getEstimatedResultRows and the most selective (i.e. the one expected to return the smallest number of results) is
chosen. A Searcher instance is then obtained from the searcherFor method & used to perform the actual Index lookup.
Finally, Indexes can define a post processing step to be performed on the coordinator, after results (partitions from
the primary table) have been received from replicas and reconciled. This post processing is defined as a
java.util.functions.BiFunction
, that is a function which takes as
arguments a PartitionIterator (containing the reconciled result rows) and a RowFilter (from the ReadCommand being
executed) and returns another iterator of partitions, possibly having transformed the initial results in some way.
The post processing function is obtained from the Index's postProcessorFor method; the built-in indexes which ship
with Cassandra return a no-op function here.
An optional static method may be provided to validate custom index options (two variants are supported):
public static Map validateOptions(Map options);
The input is the map of index options supplied in the WITH clause of a CREATE INDEX statement.
{@code public static Map validateOptions(Map options, CFMetaData cfm);}
In this version, the base table's metadata is also supplied as an argument.
If both overloaded methods are provided, only the one including the base table's metadata will be invoked.
The validation method should return a map containing any of the supplied options which are not valid for the
implementation. If the returned map is not empty, validation is considered failed and an error is raised.
Alternatively, the implementation may choose to throw an org.apache.cassandra.exceptions.ConfigurationException
if invalid options are encountered.