Package org.dataloader

Class DataLoader<K,V extends @Nullable Object>

java.lang.Object
org.dataloader.DataLoader<K,V>
Type Parameters:
K - type parameter indicating the type of the data load keys
V - type parameter indicating the type of the data that is returned
Direct Known Subclasses:
DelegatingDataLoader
@PublicApi @NullMarked public class DataLoader<K,V extends @Nullable Object> extends Object
Data loader is a utility class that allows batch loading of data that is identified by a set of unique keys. For each key that is loaded a separate CompletableFuture is returned, that completes as the batch function completes.

With batching enabled the execution will start after calling dispatch(), causing the queue of loaded keys to be sent to the batch function, clears the queue, and returns a promise to the values.

As BatchLoader batch functions are executed the resulting futures are cached using a cache implementation of choice, so they will only execute once. Individual cache keys can be cleared, so they will be re-fetched when referred to again.

It is also possible to clear the cache entirely, and prime it with values before they are used.

Both caching and batching can be disabled. Configuration of the data loader is done by providing a DataLoaderOptions instance on creation.

A call to the batch loader might result in individual exception failures for item with the returned list. if you want to capture these specific item failures then use Try as a return value and create the data loader with DataLoaderFactory.newDataLoaderWithTry(BatchLoader) form. The Try values will be interpreted as either success values or cause the load(Object) promise to complete exceptionally.

  • Method Details

    • getName

      public @Nullable String getName()
      Returns:
      the name of the DataLoader which can be null

    • getOptions

      public DataLoaderOptions getOptions()
      Returns:
      the options used to build this DataLoader

    • getBatchLoadFunction

      public Object getBatchLoadFunction()
      Returns:
      the batch load interface used to build this DataLoader

    • transform

      public DataLoader<K,V> transform(Consumer<DataLoaderFactory.Builder<K,V>> builderConsumer)
      This allows you to change the current DataLoader and turn it into a new one
      Parameters:
      builderConsumer - the DataLoaderFactory.Builder consumer for changing the DataLoader
      Returns:
      a newly built DataLoader instance

    • getLastDispatchTime

      public Instant getLastDispatchTime()
      This returns the last instant the data loader was dispatched. When the data loader is created this value is set to now.
      Returns:
      the instant since the last dispatch

    • getTimeSinceDispatch

      public Duration getTimeSinceDispatch()
      This returns the Duration since the data loader was dispatched. When the data loader is created this is zero.
      Returns:
      the time duration since the last dispatch

    • load

      public CompletableFuture<V> load(K key)
      Requests to load the data with the specified key asynchronously, and returns a future of the resulting value.

      If batching is enabled (the default), you'll have to call dispatch() at a later stage to start batch execution. If you forget this call the future will never be completed (unless already completed, and returned from cache).

      Parameters:
      key - the key to load
      Returns:
      the future of the value

    • getIfPresent

      public Optional<CompletableFuture<V>> getIfPresent(K key)
      This will return an optional promise to a value previously loaded via a load(Object) call or empty if not call has been made for that key.

      If you do get a present CompletableFuture it does not mean it has been dispatched and completed yet. It just means it's at least pending and in cache.

      If caching is disabled there will never be a present Optional returned.

      NOTE : This will NOT cause a data load to happen. You must call load(Object) for that to happen.

      Parameters:
      key - the key to check
      Returns:
      an Optional to the future of the value

    • getIfCompleted

      public Optional<CompletableFuture<V>> getIfCompleted(K key)
      This will return an optional promise to a value previously loaded via a load(Object) call that has in fact been completed or empty if no call has been made for that key or the promise has not completed yet.

      If you do get a present CompletableFuture it means it has been dispatched and completed. Completed is defined as CompletableFuture.isDone() returning true.

      If caching is disabled there will never be a present Optional returned.

      NOTE : This will NOT cause a data load to happen. You must call load(Object) for that to happen.

      Parameters:
      key - the key to check
      Returns:
      an Optional to the future of the value

    • load

      public CompletableFuture<V> load(@NonNull K key, @Nullable Object keyContext)
      Requests to load the data with the specified key asynchronously, and returns a future of the resulting value.

      If batching is enabled (the default), you'll have to call dispatch() at a later stage to start batch execution. If you forget this call the future will never be completed (unless already completed, and returned from cache).

      The key context object may be useful in the batch loader interfaces such as BatchLoaderWithContext or MappedBatchLoaderWithContext to help retrieve data.

      Parameters:
      key - the key to load
      keyContext - a context object that is specific to this key
      Returns:
      the future of the value

    • loadMany

      public CompletableFuture<List<V>> loadMany(List<K> keys)
      Requests to load the list of data provided by the specified keys asynchronously, and returns a composite future of the resulting values.

      If batching is enabled (the default), you'll have to call dispatch() at a later stage to start batch execution. If you forget this call the future will never be completed (unless already completed, and returned from cache).

      Parameters:
      keys - the list of keys to load
      Returns:
      the composite future of the list of values

    • loadMany

      public CompletableFuture<List<V>> loadMany(List<K> keys, List<Object> keyContexts)
      Requests to load the list of data provided by the specified keys asynchronously, and returns a composite future of the resulting values.

      If batching is enabled (the default), you'll have to call dispatch() at a later stage to start batch execution. If you forget this call the future will never be completed (unless already completed, and returned from cache).

      The key context object may be useful in the batch loader interfaces such as BatchLoaderWithContext or MappedBatchLoaderWithContext to help retrieve data.

      Parameters:
      keys - the list of keys to load
      keyContexts - the list of key calling context objects
      Returns:
      the composite future of the list of values

    • loadMany

      public CompletableFuture<Map<K,V>> loadMany(Map<K,?> keysAndContexts)
      Requests to load the map of data provided by the specified keys asynchronously, and returns a composite future of the resulting values.

      If batching is enabled (the default), you'll have to call dispatch() at a later stage to start batch execution. If you forget this call the future will never be completed (unless already completed, and returned from cache).

      The key context object may be useful in the batch loader interfaces such as BatchLoaderWithContext or MappedBatchLoaderWithContext to help retrieve data.

      Parameters:
      keysAndContexts - the map of keys to their respective contexts
      Returns:
      the composite future of the map of keys and values

    • dispatch

      public CompletableFuture<List<V>> dispatch()
      Dispatches the queued load requests to the batch execution function and returns a promise of the result.

      If batching is disabled, or there are no queued requests, then a succeeded promise is returned.

      Returns:
      the promise of the queued load requests

    • dispatchWithCounts

      public DispatchResult<V> dispatchWithCounts()
      Dispatches the queued load requests to the batch execution function and returns both the promise of the result and the number of entries that were dispatched.

      If batching is disabled, or there are no queued requests, then a succeeded promise with no entries dispatched is returned.

      Returns:
      the promise of the queued load requests and the number of keys dispatched.

    • dispatchAndJoin

      public List<V> dispatchAndJoin()
      Normally dispatch() is an asynchronous operation but this version will 'join' on the results if dispatch and wait for them to complete. If the CompletableFuture callbacks make more calls to this data loader then the dispatchDepth() will be > 0 and this method will loop around and wait for any other extra batch loads to occur.
      Returns:
      the list of all results when the dispatchDepth() reached 0

    • dispatchDepth

      public int dispatchDepth()
      Returns:
      the depth of the batched key loads that need to be dispatched

    • clear

      public DataLoader<K,V> clear(K key)
      Clears the future with the specified key from the cache, if caching is enabled, so it will be re-fetched on the next load request.
      Parameters:
      key - the key to remove
      Returns:
      the data loader for fluent coding

    • clear

      public DataLoader<K,V> clear(K key, BiConsumer<Void,Throwable> handler)
      Clears the future with the specified key from the cache remote value store, if caching is enabled and a remote store is set, so it will be re-fetched and stored on the next load request.
      Parameters:
      key - the key to remove
      handler - a handler that will be called after the async remote clear completes
      Returns:
      the data loader for fluent coding

    • clearAll

      public DataLoader<K,V> clearAll()
      Clears the entire cache map of the loader.
      Returns:
      the data loader for fluent coding

    • clearAll

      public DataLoader<K,V> clearAll(BiConsumer<Void,Throwable> handler)
      Clears the entire cache map of the loader, and of the cached value store.
      Parameters:
      handler - a handler that will be called after the async remote clear all completes
      Returns:
      the data loader for fluent coding

    • prime

      public DataLoader<K,V> prime(K key, V value)
      Primes the cache with the given key and value. Note this will only prime the future cache and not the value store. Use ValueCache.set(Object, Object) if you want to prime it with values before use
      Parameters:
      key - the key
      value - the value
      Returns:
      the data loader for fluent coding

    • prime

      public DataLoader<K,V> prime(K key, Exception error)
      Primes the cache with the given key and error.
      Parameters:
      key - the key
      error - the exception to prime instead of a value
      Returns:
      the data loader for fluent coding

    • prime

      public DataLoader<K,V> prime(K key, CompletableFuture<V> value)
      Primes the cache with the given key and value. Note this will only prime the future cache and not the value store. Use ValueCache.set(Object, Object) if you want to prime it with values before use
      Parameters:
      key - the key
      value - the value
      Returns:
      the data loader for fluent coding

    • getCacheKey

      public Object getCacheKey(K key)
      Gets the object that is used in the internal cache map as key, by applying the cache key function to the provided key.

      If no cache key function is present in DataLoaderOptions, then the returned value equals the input key.

      Parameters:
      key - the input key
      Returns:
      the cache key after the input is transformed with the cache key function

    • getStatistics

      public Statistics getStatistics()
      Gets the statistics associated with this data loader. These will have been gather via the StatisticsCollector passed in via DataLoaderOptions.getStatisticsCollector()
      Returns:
      statistics for this data loader

    • getCacheMap

      public CacheMap<Object,V> getCacheMap()
      Gets the cacheMap associated with this data loader passed in via DataLoaderOptions.cacheMap()
      Returns:
      the cacheMap of this data loader

    • getValueCache

      public ValueCache<K,V> getValueCache()
      Gets the valueCache associated with this data loader passed in via DataLoaderOptions.valueCache()
      Returns:
      the valueCache of this data loader

    • toString

      public String toString()
      Overrides:
      toString in class Object