Package com.apple.foundationdb.record.provider.foundationdb.leaderboard

Maintain leaderboard as multiple time-windowed ranked sets (so that old scores fall off).

The time windows are grouped by type, a client-supplied positive integer. For example, daily and weekly. The built-in ALL_TIME_WINDOW_LEADERBOARD_TYPE type has the value 0.

Each time window has a start and (exclusive) end timestamp, using a client-supplied timebase.

A record is expected to contain multiple timestamped scored. The leaderboard enters the record's best score (client specifies whether this is lowest or highest numerical value) in each time window. This determines the record's rank (leaderboard position) at that time.

Index Representation

The expected index keys are group_fields..., score, timestamp, more_fields.... A single B-tree in the primary index subspace stores these once for all time windows.

The ideal root expression is therefore field(scores_field, fanOut).nest(concat(score, timestamp, context)).

For the sake of simpler type systems, SplitKeyExpression can be used to store timestamped scores in the record in a repeated long field, along with zero or more opaque context values.

The index root expression is then split(list_field/fanOut, n) or concat(group_fields..., split(list_field/fanOut, n), more_fields...).

The primary index entries are group_fields..., score, timestamp, more_fields..., primary_key → context_values.

The root of the secondary index subspace is a directory, TimeWindowLeaderboardDirectory, serialized as Protobuf.

For each directory entry, there is a leaderboard subspace using the entry's subspace key.

There is a RankedSet within the leaderboard subspace for each group, that is, with any grouping keys as a prefix.

The ranked set entries are for score, timestamp, more_fields. When scores match, the timestamp can break the tie: earlier is better (even when high scores come first). more_fields can be used to further break ties at the same timestamp.

A group can have an optional TimeWindowLeaderboardSubDirectory. A sub-directory allows individual groups to have different settings for whether high scores come first. Sub-directories are serialized as Protobuf in the secondary index subspace with key [null, group_fields...]. This does not conflict with leaderboards because null is not a valid leaderboard subspace key (they are automatically assigned integers).

Operations

Updating Time Windows The client is also responsible for keeping a current set of time windows active. This is done with TimeWindowLeaderboardWindowUpdate. The caller specifies:
  • Whether high scores are better or low scores for determining the best score in a given window
  • A timestamp before which expired time windows can be removed from the database
  • A set of per-type specifications for regularly spaced windows, giving a base timestamp, duration and repeat count

A good practice is to probabilistically add time windows in the future, with the chances increasing to certainty as the time when a new window would be needed approaches.

Scanning

The leaderboard index can be scanned BY_VALUE, like an ordinary index, provided the all-time time window is maintained. This means by score ranges, or score matches for time ranges.

Scanning BY_RANK means a range of ranks rather than scores, as with a rankset index. For this, too, the ALL_TIME_LEADERBOARD_TYPE time window is used.

Scanning BY_TIME_WINDOW adds to tuple items at the beginning of the range representing the time window type and target timestamp. The oldest ranked set of the given type containing the timestamp will be used. For example,

final TupleRange top_10_type_2 = new TupleRange(Tuple.from(2, now, 0), Tuple.from(2, now, 9), EndpointType.RANGE_INCLUSIVE, EndpointType.RANGE_INCLUSIVE); final RecordCursor<Message> cursor = <Message>recordStore.scanIndexRecords("LeaderboardIndex", IndexScanType.BY_TIME_WINDOW, top_2_type_2, ScanProperties.FORWARD_SCAN);

Querying

The index can be used to match predicates / sorting for Query.rank(expr). Again, the ALL_TIME_LEADERBOARD_TYPE time window is used.

It can also match Query.timeWindowRank(leaderboardType, leaderboardTimestamp, expr). leaderboardType and leaderboardTimestamp can be strings, in which case they specify the names of runtime parameters containing those values. The rank(s) will be taken from the oldest leaderboard of the specified type containing the specified timestamp.