Package org.dataloader

Class DataLoader<K,​V>

  • Type Parameters:
    K - type parameter indicating the type of the data load keys
    V - type parameter indicating the type of the data that is returned
    Direct Known Subclasses:
    DelegatingDataLoader
    @PublicApi
@NullMarked
public class DataLoader<K,​V>
extends java.lang.Object
    Data loader is a utility class that allows batch loading of data that is identified by a set of unique keys. For each key that is loaded a separate CompletableFuture is returned, that completes as the batch function completes.

    With batching enabled the execution will start after calling dispatch(), causing the queue of loaded keys to be sent to the batch function, clears the queue, and returns a promise to the values.

    As BatchLoader batch functions are executed the resulting futures are cached using a cache implementation of choice, so they will only execute once. Individual cache keys can be cleared, so they will be re-fetched when referred to again.

    It is also possible to clear the cache entirely, and prime it with values before they are used.

    Both caching and batching can be disabled. Configuration of the data loader is done by providing a DataLoaderOptions instance on creation.

    A call to the batch loader might result in individual exception failures for item with the returned list. if you want to capture these specific item failures then use Try as a return value and create the data loader with DataLoaderFactory.newDataLoaderWithTry(BatchLoader) form. The Try values will be interpreted as either success values or cause the load(Object) promise to complete exceptionally.

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      DataLoader<K,​V> clear​(K key)
      Clears the future with the specified key from the cache, if caching is enabled, so it will be re-fetched on the next load request.
      DataLoader<K,​V> clear​(K key, java.util.function.BiConsumer<java.lang.Void,​java.lang.Throwable> handler)
      Clears the future with the specified key from the cache remote value store, if caching is enabled and a remote store is set, so it will be re-fetched and stored on the next load request.
      DataLoader<K,​V> clearAll()
      Clears the entire cache map of the loader.
      DataLoader<K,​V> clearAll​(java.util.function.BiConsumer<java.lang.Void,​java.lang.Throwable> handler)
      Clears the entire cache map of the loader, and of the cached value store.
      java.util.concurrent.CompletableFuture<java.util.List<V>> dispatch()
      Dispatches the queued load requests to the batch execution function and returns a promise of the result.
      java.util.List<V> dispatchAndJoin()
      Normally dispatch() is an asynchronous operation but this version will 'join' on the results if dispatch and wait for them to complete.
      int dispatchDepth()  
      DispatchResult<V> dispatchWithCounts()
      Dispatches the queued load requests to the batch execution function and returns both the promise of the result and the number of entries that were dispatched.
      java.lang.Object getBatchLoadFunction()  
      java.lang.Object getCacheKey​(K key)
      Gets the object that is used in the internal cache map as key, by applying the cache key function to the provided key.
      CacheMap<java.lang.Object,​V> getCacheMap()
      Gets the cacheMap associated with this data loader passed in via DataLoaderOptions.cacheMap()
      java.util.Optional<java.util.concurrent.CompletableFuture<V>> getIfCompleted​(K key)
      This will return an optional promise to a value previously loaded via a load(Object) call that has in fact been completed or empty if no call has been made for that key or the promise has not completed yet.
      java.util.Optional<java.util.concurrent.CompletableFuture<V>> getIfPresent​(K key)
      This will return an optional promise to a value previously loaded via a load(Object) call or empty if not call has been made for that key.
      java.time.Instant getLastDispatchTime()
      This returns the last instant the data loader was dispatched.
      @Nullable java.lang.String getName()  
      DataLoaderOptions getOptions()  
      Statistics getStatistics()
      Gets the statistics associated with this data loader.
      java.time.Duration getTimeSinceDispatch()
      This returns the Duration since the data loader was dispatched.
      ValueCache<K,​V> getValueCache()
      Gets the valueCache associated with this data loader passed in via DataLoaderOptions.valueCache()
      java.util.concurrent.CompletableFuture<V> load​(@NonNull K key, @Nullable java.lang.Object keyContext)
      Requests to load the data with the specified key asynchronously, and returns a future of the resulting value.
      java.util.concurrent.CompletableFuture<V> load​(K key)
      Requests to load the data with the specified key asynchronously, and returns a future of the resulting value.
      java.util.concurrent.CompletableFuture<java.util.List<V>> loadMany​(java.util.List<K> keys)
      Requests to load the list of data provided by the specified keys asynchronously, and returns a composite future of the resulting values.
      java.util.concurrent.CompletableFuture<java.util.List<V>> loadMany​(java.util.List<K> keys, java.util.List<java.lang.Object> keyContexts)
      Requests to load the list of data provided by the specified keys asynchronously, and returns a composite future of the resulting values.
      java.util.concurrent.CompletableFuture<java.util.Map<K,​V>> loadMany​(java.util.Map<K,​?> keysAndContexts)
      Requests to load the map of data provided by the specified keys asynchronously, and returns a composite future of the resulting values.
      DataLoader<K,​V> prime​(K key, java.lang.Exception error)
      Primes the cache with the given key and error.
      DataLoader<K,​V> prime​(K key, java.util.concurrent.CompletableFuture<V> value)
      Primes the cache with the given key and value.
      DataLoader<K,​V> prime​(K key, V value)
      Primes the cache with the given key and value.
      java.lang.String toString()  
      DataLoader<K,​V> transform​(java.util.function.Consumer<DataLoaderFactory.Builder<K,​V>> builderConsumer)
      This allows you to change the current DataLoader and turn it into a new one

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

    • Method Detail

      • getName

        public @Nullable java.lang.String getName()
        Returns:
        the name of the DataLoader which can be null

      • getBatchLoadFunction

        public java.lang.Object getBatchLoadFunction()
        Returns:
        the batch load interface used to build this DataLoader

      • getLastDispatchTime

        public java.time.Instant getLastDispatchTime()
        This returns the last instant the data loader was dispatched. When the data loader is created this value is set to now.
        Returns:
        the instant since the last dispatch

      • getTimeSinceDispatch

        public java.time.Duration getTimeSinceDispatch()
        This returns the Duration since the data loader was dispatched. When the data loader is created this is zero.
        Returns:
        the time duration since the last dispatch

      • load

        public java.util.concurrent.CompletableFuture<V> load​(K key)
        Requests to load the data with the specified key asynchronously, and returns a future of the resulting value.

        If batching is enabled (the default), you'll have to call dispatch() at a later stage to start batch execution. If you forget this call the future will never be completed (unless already completed, and returned from cache).

        Parameters:
        key - the key to load
        Returns:
        the future of the value

      • getIfPresent

        public java.util.Optional<java.util.concurrent.CompletableFuture<V>> getIfPresent​(K key)
        This will return an optional promise to a value previously loaded via a load(Object) call or empty if not call has been made for that key.

        If you do get a present CompletableFuture it does not mean it has been dispatched and completed yet. It just means it's at least pending and in cache.

        If caching is disabled there will never be a present Optional returned.

        NOTE : This will NOT cause a data load to happen. You must call load(Object) for that to happen.

        Parameters:
        key - the key to check
        Returns:
        an Optional to the future of the value

      • getIfCompleted

        public java.util.Optional<java.util.concurrent.CompletableFuture<V>> getIfCompleted​(K key)
        This will return an optional promise to a value previously loaded via a load(Object) call that has in fact been completed or empty if no call has been made for that key or the promise has not completed yet.

        If you do get a present CompletableFuture it means it has been dispatched and completed. Completed is defined as CompletableFuture.isDone() returning true.

        If caching is disabled there will never be a present Optional returned.

        NOTE : This will NOT cause a data load to happen. You must call load(Object) for that to happen.

        Parameters:
        key - the key to check
        Returns:
        an Optional to the future of the value

      • load

        public java.util.concurrent.CompletableFuture<V> load​(@NonNull K key,
                                                      @Nullable java.lang.Object keyContext)
        Requests to load the data with the specified key asynchronously, and returns a future of the resulting value.

        If batching is enabled (the default), you'll have to call dispatch() at a later stage to start batch execution. If you forget this call the future will never be completed (unless already completed, and returned from cache).

        The key context object may be useful in the batch loader interfaces such as BatchLoaderWithContext or MappedBatchLoaderWithContext to help retrieve data.

        Parameters:
        key - the key to load
        keyContext - a context object that is specific to this key
        Returns:
        the future of the value

      • loadMany

        public java.util.concurrent.CompletableFuture<java.util.List<V>> loadMany​(java.util.List<K> keys)
        Requests to load the list of data provided by the specified keys asynchronously, and returns a composite future of the resulting values.

        If batching is enabled (the default), you'll have to call dispatch() at a later stage to start batch execution. If you forget this call the future will never be completed (unless already completed, and returned from cache).

        Parameters:
        keys - the list of keys to load
        Returns:
        the composite future of the list of values

      • loadMany

        public java.util.concurrent.CompletableFuture<java.util.List<V>> loadMany​(java.util.List<K> keys,
                                                                          java.util.List<java.lang.Object> keyContexts)
        Requests to load the list of data provided by the specified keys asynchronously, and returns a composite future of the resulting values.

        If batching is enabled (the default), you'll have to call dispatch() at a later stage to start batch execution. If you forget this call the future will never be completed (unless already completed, and returned from cache).

        The key context object may be useful in the batch loader interfaces such as BatchLoaderWithContext or MappedBatchLoaderWithContext to help retrieve data.

        Parameters:
        keys - the list of keys to load
        keyContexts - the list of key calling context objects
        Returns:
        the composite future of the list of values

      • loadMany

        public java.util.concurrent.CompletableFuture<java.util.Map<K,​V>> loadMany​(java.util.Map<K,​?> keysAndContexts)
        Requests to load the map of data provided by the specified keys asynchronously, and returns a composite future of the resulting values.

        If batching is enabled (the default), you'll have to call dispatch() at a later stage to start batch execution. If you forget this call the future will never be completed (unless already completed, and returned from cache).

        The key context object may be useful in the batch loader interfaces such as BatchLoaderWithContext or MappedBatchLoaderWithContext to help retrieve data.

        Parameters:
        keysAndContexts - the map of keys to their respective contexts
        Returns:
        the composite future of the map of keys and values

      • dispatch

        public java.util.concurrent.CompletableFuture<java.util.List<V>> dispatch()
        Dispatches the queued load requests to the batch execution function and returns a promise of the result.

        If batching is disabled, or there are no queued requests, then a succeeded promise is returned.

        Returns:
        the promise of the queued load requests

      • dispatchWithCounts

        public DispatchResult<V> dispatchWithCounts()
        Dispatches the queued load requests to the batch execution function and returns both the promise of the result and the number of entries that were dispatched.

        If batching is disabled, or there are no queued requests, then a succeeded promise with no entries dispatched is returned.

        Returns:
        the promise of the queued load requests and the number of keys dispatched.

      • dispatchAndJoin

        public java.util.List<V> dispatchAndJoin()
        Normally dispatch() is an asynchronous operation but this version will 'join' on the results if dispatch and wait for them to complete. If the CompletableFuture callbacks make more calls to this data loader then the dispatchDepth() will be > 0 and this method will loop around and wait for any other extra batch loads to occur.
        Returns:
        the list of all results when the dispatchDepth() reached 0

      • dispatchDepth

        public int dispatchDepth()
        Returns:
        the depth of the batched key loads that need to be dispatched

      • clear

        public DataLoader<K,​V> clear​(K key)
        Clears the future with the specified key from the cache, if caching is enabled, so it will be re-fetched on the next load request.
        Parameters:
        key - the key to remove
        Returns:
        the data loader for fluent coding

      • clear

        public DataLoader<K,​V> clear​(K key,
                                   java.util.function.BiConsumer<java.lang.Void,​java.lang.Throwable> handler)
        Clears the future with the specified key from the cache remote value store, if caching is enabled and a remote store is set, so it will be re-fetched and stored on the next load request.
        Parameters:
        key - the key to remove
        handler - a handler that will be called after the async remote clear completes
        Returns:
        the data loader for fluent coding

      • clearAll

        public DataLoader<K,​V> clearAll()
        Clears the entire cache map of the loader.
        Returns:
        the data loader for fluent coding

      • clearAll

        public DataLoader<K,​V> clearAll​(java.util.function.BiConsumer<java.lang.Void,​java.lang.Throwable> handler)
        Clears the entire cache map of the loader, and of the cached value store.
        Parameters:
        handler - a handler that will be called after the async remote clear all completes
        Returns:
        the data loader for fluent coding

      • prime

        public DataLoader<K,​V> prime​(K key,
                                   V value)
        Primes the cache with the given key and value. Note this will only prime the future cache and not the value store. Use ValueCache.set(Object, Object) if you want to prime it with values before use
        Parameters:
        key - the key
        value - the value
        Returns:
        the data loader for fluent coding

      • prime

        public DataLoader<K,​V> prime​(K key,
                                   java.lang.Exception error)
        Primes the cache with the given key and error.
        Parameters:
        key - the key
        error - the exception to prime instead of a value
        Returns:
        the data loader for fluent coding

      • prime

        public DataLoader<K,​V> prime​(K key,
                                   java.util.concurrent.CompletableFuture<V> value)
        Primes the cache with the given key and value. Note this will only prime the future cache and not the value store. Use ValueCache.set(Object, Object) if you want to prime it with values before use
        Parameters:
        key - the key
        value - the value
        Returns:
        the data loader for fluent coding

      • getCacheKey

        public java.lang.Object getCacheKey​(K key)
        Gets the object that is used in the internal cache map as key, by applying the cache key function to the provided key.

        If no cache key function is present in DataLoaderOptions, then the returned value equals the input key.

        Parameters:
        key - the input key
        Returns:
        the cache key after the input is transformed with the cache key function

      • getCacheMap

        public CacheMap<java.lang.Object,​V> getCacheMap()
        Gets the cacheMap associated with this data loader passed in via DataLoaderOptions.cacheMap()
        Returns:
        the cacheMap of this data loader

      • toString

        public java.lang.String toString()
        Overrides:
        toString in class java.lang.Object