util
Mutable double to integer map based on an underlying LongIntHashMap.
Wraps the java IdentityHashMap as an immutable scala Map.
Wraps the java IdentityHashMap as an immutable scala Map. Modifications will result in a copy of the wrapped map being created and used with the new instance.
Mutable integer set based on open-addressing.
Mutable integer set based on open-addressing. Primary use-case is deduping
integers so it only supports add and foreach.
Mutable integer map based on open-addressing.
Mutable integer map based on open-addressing. Primary use-case is computing a count for the number of times a particular value was encountered.
Mutable integer map based on open-addressing.
Mutable integer map based on open-addressing. Primary use-case is computing a count for the number of times a particular value was encountered.
Keeps track of canonical references for a type of object.
Keeps track of canonical references for a type of object. Typically used to reduce memory overhead if an application potentially creates many copies of equal objects and will need to keep them around for some period of time.
Mutable integer set based on open-addressing.
Mutable integer set based on open-addressing. Primary use-case is deduping
integers so it only supports add and foreach.
Mutable long to integer map based on open-addressing.
Mutable long to integer map based on open-addressing. Primary use-case is computing a count for the number of times a particular value was encountered.
Does nothing, the original value will always be returned.
Interner that starts with a predefined set of values.
Interner that starts with a predefined set of values. If a request is made to intern a value that is not in the predefined list, then it will be forwarded to the specified fallback interner.
Mutable reference to integer map based on open-addressing.
Mutable reference to integer map based on open-addressing. Primary use-case is computing a count for the number of times a particular value was encountered.
Interval that moves over time in increments of a given unit.
Interval that moves over time in increments of a given unit. When in the middle of a unit it will round to the next even boundary. For example, if the unit is HOURS and it is 10:37, then it will round too 11:00.
The offset and duration must be an even multiple of the unit.
Offset subtracted from the current time, now - offset is used as the end
time for this interval.
The length of the interval. The start time is now - offset - duration.
The unit to use when moving along. This is typically HOURS or DAYS.
Simple immutable hash map implementation intended for use-cases where the number of entries is known to be small.
Simple immutable hash map implementation intended for use-cases where the number of entries is known to be small. This implementation is backed by a single array and uses open addressing with linear probing to resolve conflicts. The underlying array is created to exactly fit the data size so hash collisions tend to be around 50%, but have a fairly low number of probes to find the actual entry. With a cheap equals function for the keys lookups should be fast and there is low memory overhead.
You probably don't want to use this implementation if you expect more than around 50 keys in the map. If you have millions of small immutable maps, such as tag data associated with metrics, it may be a good fit.
Sine function based on timestamps.
Sine function based on timestamps. The sine values will be pre-computed for a single wavelength and then looked up for all others. This can be significantly faster than using the sine function directly for longer spans.
Span of time for the repeating pattern of the wave.
How often to compute the sine value within the wavelength.
Common prefixes used for units or human readable strings.
Common prefixes used for units or human readable strings.
the symbol shown for the prefix
text for the prefix
the multiplication factor for the prefix
Helper functions for interners.
Helper functions for working with lists.
Utility functions for mapping ids or indices to a shard.
Utility functions for mapping ids or indices to a shard. For our purposes, a shard is an instance with a set of server groups. The union of data from all groups comprises a full copy of the overall dataset. To allow for smaller deployment units, an individual group or subset of the groups can be replicated. For redundancy, groups could be replicated all the time. At Netflix, we typically replicate the overall set of server groups in another region or zone instead.
This class specifically focuses on relatively simple sharding schemes where the component making the decision only needs to know the set of instances and a slot for each instance. Edda is one example of a system that provides this information for AWS auto-scaling groups. More complex sharding schemes that require additional infrastructure, e.g, zookeeper, are out of scope here. There are two sharding modes supported by this class:
1. Mapping an id for a tagged item to a shard. This is typically done while data is flowing into the system and each datapoint can be routed based on the id.
2. Mapping an positional index to a shard. This is typically done for loading data that has been processed via Hadoop or similar tools and stored in a fixed number of files. There should be a manifest with an order list of the files for a given time and the position can be used to map to a shard. When using this approach it is recommended to use a [highly composite number][hcn] for the number of files. This makes it easier to pick a number of groups and sizes for the groups such that each instance will get the same number of files.
When mapping this to AWS an overall deployment is typically a set of auto-scaling groups (ASG). Each instance should get the same amount of data if possible given the set of files. Deployments are typically done as a red/black push of one ASG at a time. So the amount of additional capacity during a push is the size of one of these groups if deployments across the groups are performed serially. While multiple ASGs for a particular group are active the data will be replicated across them.
[hcn]: https://en.wikipedia.org/wiki/Highly_composite_number
Utilities for computing and rounding times based on the step size for a dataset.
Delegate to String.intern().
Helper functions for working with strings.
Mutable double to integer map based on an underlying LongIntHashMap. Primary use-case is computing a count for the number of times a particular value was encountered.