Concatenate one or more distributed lists to this distributed list.
Apply an associative function to reduce the collection of values to a single value in a key-value-collection distributed list.
Group the values of a distributed list with key-value elements by key.
Turn a distributed list into a normal, non-distributed collection that can be accessed by the client
Apply a specified function to "chunks" of elements from the distributed list to produce zero or more output elements.
Apply a specified function to "chunks" of elements from the distributed list to produce zero or more output elements. The resulting output elements from the many "chunks" form a new distributed list
Create a new distributed list that is keyed based on a specified function.
Build a new DList by applying a partial function to all elements of this DList on which the function is defined
Count the number of elements in the list which satisfy a predicate.
Build a new distributed list from this list without any duplicate elements.
Keep elements from the distributed list that pass a specified predicate function
Keep elements from the distributed list that do not pass a specified predicate function
Converts a distributed list of iterable values into to a distributed list in which all the values are concatenated.
Group the values of a distributed list according to some discriminator function.
Group the values of a distributed list with key-value elements by key.
Group the values of a distributed list with key-value elements by key. And explicitly take the grouping that should be used. This is best used when you're doing things like secondary sorts, or groupings with strange logic (like making sure None's / nulls are sprayed across all reducers
Group the value of a distributed list according to some discriminator function and some grouping function.
the head of the DList as a DObject. This is an unsafe operation
the head of the DList as a DObject containing an Option
Returns if the other DList has the same elements.
Returns if the other DList has the same elements. A DList is unordered so order isn't considered. The Grouping required isn't very special and almost any will work (including grouping designed for secondary sorting) but for completeness, it is required to send two equal As to the same partition, and sortCompare provide total ordering
Create a distributed list containing just the keys of a key-value distributed list.
The length of the distributed list.
For each element of the distributed list produce a new element by applying a specified function.
For each element of the distributed list produce a new element by applying a specified function. The resulting collection of elements form a new distributed list
For each element of the distributed list produce zero or more elements by applying a specified function.
For each element of the distributed list produce zero or more elements by applying a specified function. The resulting collection of elements form a new distributed list
Find the largest element in the distributed list.
Find the largest element in the distributed list.
Find the smallest element in the distributed list.
Find the smallest element in the distributed list.
Partitions this distributed list into a pair of distributed lists according to some predicate.
Partitions this distributed list into a pair of distributed lists according to some predicate. The first distributed list consists of elements that satisfy the predicate and the second of all elements that don't.
Multiply up the elements of this distribute list.
Reduce the elements of this distributed list using the specified associative binary operator.
Reduce the elements of this distributed list using the specified associative binary operator. The order in which the elements are reduced is unspecified and may be non-deterministic
Reduce the elements of this distributed list using the specified associative binary operator and a default value if the list is empty.
Reduce the elements of this distributed list using the specified associative binary operator and a default value if the list is empty. The order in which the elements are reduced is unspecified and may be non-deterministic
Randomly shuffle a DList.
The size of the distributed list.
Sum up the elements of this distribute list.
Create a distributed list containing just the values of a key-value distributed list.
the withFilter method
Add an index (Long) to the DList where the index is between 0 and .
Add an index (Long) to the DList where the index is between 0 and .size-1 of the DList
(Since version 0.7.0) use mapFlatten instead because DList is not a subclass of Iterator and a well-behaved flatMap operation accepts an argument: A => DList[B]
(Since version 0.6.0) use materialise instead
A list that is distributed across multiple machines.
It supports a few Traversable-like methods:
- parallelDo: a 'map' operation transforming elements of the list in parallel - ++: to concatenate 2 DLists - groupByKey: to group a list of (key, value) elements by key, so as to get (key, values) - combine: a parallel 'reduce' operation - materialise: transforms a distributed list into a non-distributed list