T
- the type of the stream elementsA
- the type of arrayP
- the type of predicateC
- the type of consumerPL
- the type of PrimitiveList/ListOT
- the type of OptionalIT
- the type of IndexedS
- the type of of the stream implementing BaseStream
public interface BaseStream<T,A,P,C,PL,OT,IT,S extends BaseStream<T,A,P,C,PL,OT,IT,S>>
extends java.lang.AutoCloseable
Stream
and IntStream
, computing the sum of the weights of the red widgets:
int sum = widgets.stream()
.filter(w -> w.getColor() == RED)
.mapToInt(w -> w.getWeight())
.sum();
See the class documentation for Stream
and the package documentation
for java.util.stream for additional
specification of streams, stream operations, stream pipelines, and
parallelism, which governs the behavior of all stream types.Stream
,
IntStream
,
LongStream
,
DoubleStream
,
java.util.streamModifier and Type | Interface and Description |
---|---|
static class |
BaseStream.Splitor |
Modifier and Type | Method and Description |
---|---|
<R> R |
__(Function<? super S,R> transfer) |
S |
append(S s) |
S |
cached()
This method only run sequentially, even in parallel stream and all elements will be loaded to memory. |
S |
carry(C action)
Same as
peek |
void |
close()
Closes this stream, causing all close handlers for this stream pipeline
to be called.
|
long |
count()
Returns the count of elements in this stream.
|
S |
difference(java.util.Collection<?> c)
This method only run sequentially, even in parallel stream. |
S |
distinct()
Returns a stream consisting of the distinct elements of this stream.
|
S |
dropWhile(P predicate)
Remove the elements until the given predicate returns false.
|
S |
filter(P predicate)
Returns a stream consisting of the elements of this stream that match
the given predicate.
|
OT |
first() |
Stream<IT> |
indexed()
This method only run sequentially, even in parallel stream. |
S |
intersection(java.util.Collection<?> c)
This method only run sequentially, even in parallel stream. |
boolean |
isParallel()
Returns whether this stream, if a terminal operation were to be executed,
would execute in parallel.
|
ImmutableIterator<T> |
iterator()
Returns an iterator for the elements of this stream.
|
java.lang.String |
join(java.lang.CharSequence delimiter) |
java.lang.String |
join(java.lang.CharSequence delimiter,
java.lang.CharSequence prefix,
java.lang.CharSequence suffix) |
OT |
last() |
S |
limit(long maxSize)
Returns a stream consisting of the elements of this stream, truncated
to be no longer than
maxSize in length. |
int |
maxThreadNum()
Return the underlying
maxThreadNum if the stream is parallel, otherwise 1 is returned. |
S |
maxThreadNum(int maxThreadNum)
Returns a parallel stream with the specified
maxThreadNum . |
S |
onClose(java.lang.Runnable closeHandler)
Returns an equivalent stream with an additional close handler.
|
S |
parallel()
Returns an equivalent stream that is parallel.
|
S |
parallel(BaseStream.Splitor splitor)
Returns an equivalent stream that is parallel.
|
S |
parallel(int maxThreadNum)
Returns an equivalent stream that is parallel.
|
S |
parallel(int maxThreadNum,
BaseStream.Splitor splitor)
Returns an equivalent stream that is parallel.
|
S |
peek(C action)
Returns a stream consisting of the elements of this stream, additionally
performing the provided action on each element as elements are consumed
from the resulting stream.
|
Optional<java.util.Map<com.landawn.abacus.util.Percentage,T>> |
percentiles()
All elements will be loaded to memory and sorted if not yet. |
S |
prepend(S stream) |
void |
println() |
S |
remove(long n,
C consumer)
Returns a stream consisting of the remaining elements of this stream
after removing and consuming the first
n elements of the stream. |
S |
removeIf(P predicate) |
S |
removeIf(P predicate,
C consumer) |
S |
removeWhile(P predicate,
C consumer)
Returns a stream consisting of the remaining elements of this stream
after removing and consuming until the specified
predicate return false. |
S |
reversed()
This method only run sequentially, even in parallel stream and all elements will be loaded to memory. |
S |
reverseSorted() |
S |
rotated(int distance)
This method only run sequentially, even in parallel stream and all elements will be loaded to memory. |
S |
sequential()
Returns an equivalent stream that is sequential.
|
S |
shuffled()
This method only run sequentially, even in parallel stream and all elements will be loaded to memory. |
S |
shuffled(java.util.Random rnd)
This method only run sequentially, even in parallel stream and all elements will be loaded to memory. |
S |
skip(long n)
Returns a stream consisting of the remaining elements of this stream
after discarding the first
n elements of the stream. |
Stream<S> |
sliding(int windowSize) |
Stream<S> |
sliding(int windowSize,
int increment)
Stream.of(1, 2, 3, 4, 5, 6, 7, 8).sliding(3, 1).forEach(Stream::println)
output: [1, 2, 3] [2, 3, 4] [3, 4, 5] [4, 5, 6] [5, 6, 7] [6, 7, 8] ============================================================================ Stream.of(1, 2, 3, 4, 5, 6, 7, 8).sliding(3, 3).forEach(Stream::println)
output: [1, 2, 3] [4, 5, 6] [7, 8] ============================================================================ Stream.of(1, 2, 3, 4, 5, 6, 7, 5).sliding(3, 5).forEach(Stream::println)
output: [1, 2, 3] [6, 7, 8] This method only run sequentially, even in parallel stream. |
Stream<PL> |
slidingToList(int windowSize) |
Stream<PL> |
slidingToList(int windowSize,
int increment) |
S |
sorted()
Returns a stream consisting of the elements of this stream in sorted
order.
|
Stream<S> |
split(int size)
Returns Stream of ByteStream with consecutive sub sequences of the elements, each of the same size (the final sequence may be smaller).
|
Stream<S> |
split(P predicate)
Split the stream by the specified predicate.
|
<U> Stream<S> |
split(U seed,
BiPredicate<? super T,? super U> predicate,
Consumer<? super U> seedUpdate)
Split the stream by the specified predicate.
|
Stream<S> |
splitAt(int where)
Split the stream into two pieces at
where |
Stream<S> |
splitBy(P where)
Split the stream into two pieces at
where turns to false |
BaseStream.Splitor |
splitor()
Return the underlying
splitor if the stream is parallel, otherwise the default value splitor.ITERATOR is returned. |
S |
splitor(BaseStream.Splitor splitor)
Returns a parallel stream with the specified
splitor . |
Stream<PL> |
splitToList(int size)
Returns Stream of Stream with consecutive sub sequences of the elements, each of the same size (the final sequence may be smaller).
|
Stream<PL> |
splitToList(P predicate)
Split the stream by the specified predicate.
|
<U> Stream<PL> |
splitToList(U seed,
BiPredicate<? super T,? super U> predicate,
Consumer<? super U> seedUpdate)
Split the stream by the specified predicate.
|
S |
step(long step) |
S |
symmetricDifference(java.util.Collection<T> c)
This method only run sequentially, even in parallel stream. |
S |
takeWhile(P predicate)
Keep the elements until the given predicate returns false.
|
A |
toArray()
Returns an array containing the elements of this stream.
|
java.util.List<T> |
toList() |
<R extends java.util.List<T>> |
toList(Supplier<R> supplier) |
LongMultiset<T> |
toLongMultiset() |
LongMultiset<T> |
toLongMultiset(Supplier<? extends LongMultiset<T>> supplier) |
Multiset<T> |
toMultiset() |
Multiset<T> |
toMultiset(Supplier<? extends Multiset<T>> supplier) |
java.util.Set<T> |
toSet() |
<R extends java.util.Set<T>> |
toSet(Supplier<R> supplier) |
Try<S> |
tried() |
S filter(P predicate)
This is an intermediate operation.
predicate
- a non-interfering,
stateless
predicate to apply to each element to determine if it
should be includedS takeWhile(P predicate)
predicate.text(x)
returns false, any element y behind x: predicate.text(y)
should returns false.
In parallel Streams, the elements after the first element which predicate
returns false may be tested by predicate too.predicate
- S dropWhile(P predicate)
predicate.text(x)
returns true, any element y behind x: predicate.text(y)
should returns true.
In parallel Streams, the elements after the first element which predicate
returns false may be tested by predicate too.predicate
- S remove(long n, C consumer)
n
elements of the stream.
If this stream contains fewer than n
elements then an
empty stream will be returned.n
- consumer
- S removeWhile(P predicate, C consumer)
predicate
return false.
If there is no more elements then an empty stream will be returned.predicate
- consumer
- Stream<S> split(int size)
size
- Stream<PL> splitToList(int size)
size
- Stream<S> split(P predicate)
predicate
- Stream<PL> splitToList(P predicate)
predicate
- <U> Stream<S> split(U seed, BiPredicate<? super T,? super U> predicate, Consumer<? super U> seedUpdate)
// split the number sequence by window 5.
Stream.of(1, 2, 3, 5, 7, 9, 10, 11, 19).splitToList(MutableInt.of(5), (e, b) -> e <= b.intValue(), b -> b.addAndGet(5)).forEach(N::println);
This stream should be sorted by value which is used to verify the border.
seed
- predicate
- seedUpdate
- <U> Stream<PL> splitToList(U seed, BiPredicate<? super T,? super U> predicate, Consumer<? super U> seedUpdate)
// split the number sequence by window 5.
Stream.of(1, 2, 3, 5, 7, 9, 10, 11, 19).splitToList(MutableInt.of(5), (e, b) -> e <= b.intValue(), b -> b.addAndGet(5)).forEach(N::println);
This stream should be sorted by value which is used to verify the border.
seed
- predicate
- seedUpdate
- Stream<S> splitAt(int where)
where
where
- Stream<S> splitBy(P where)
where
turns to false
Stream.of(1, 3, 2, 4, 2, 5).splitBy(i -> i <= 3).forEach(s -> s.println()); // [1, 3, 2], [4, 2, 5]
where
- Stream<S> sliding(int windowSize)
windowSize
- sliding(int, int)
Stream<PL> slidingToList(int windowSize)
windowSize
- sliding(int, int)
Stream<S> sliding(int windowSize, int increment)
Stream.of(1, 2, 3, 4, 5, 6, 7, 8).sliding(3, 1).forEach(Stream::println)
Stream.of(1, 2, 3, 4, 5, 6, 7, 8).sliding(3, 3).forEach(Stream::println)
Stream.of(1, 2, 3, 4, 5, 6, 7, 5).sliding(3, 5).forEach(Stream::println)
windowSize
- increment
- Stream<PL> slidingToList(int windowSize, int increment)
windowSize
- increment
- sliding(int, int)
S intersection(java.util.Collection<?> c)
c
- IntList.intersection(IntList)
S difference(java.util.Collection<?> c)
c
- IntList.difference(IntList)
S symmetricDifference(java.util.Collection<T> c)
c
- IntList.symmetricDifference(IntList)
Optional<java.util.Map<com.landawn.abacus.util.Percentage,T>> percentiles()
S reversed()
S shuffled()
S shuffled(java.util.Random rnd)
S rotated(int distance)
S distinct()
This is a stateful intermediate operation.
S sorted()
S reverseSorted()
S cached()
java.lang.String join(java.lang.CharSequence delimiter)
java.lang.String join(java.lang.CharSequence delimiter, java.lang.CharSequence prefix, java.lang.CharSequence suffix)
S skip(long n)
n
elements of the stream.
If this stream contains fewer than n
elements then an
empty stream will be returned.
This is a stateful intermediate operation.
n
- the number of leading elements to skipjava.lang.IllegalArgumentException
- if n
is negativeS limit(long maxSize)
maxSize
in length.
maxSize
- the number of elements the stream should be limited tojava.lang.IllegalArgumentException
- if maxSize
is negativeS step(long step)
long count()
return mapToLong(e -> 1L).sum();
This is a terminal operation.
S peek(C action)
This is an intermediate operation.
For parallel stream pipelines, the action may be called at whatever time and in whatever thread the element is made available by the upstream operation. If the action modifies shared state, it is responsible for providing the required synchronization.
action
- a
non-interfering action to perform on the elements as
they are consumed from the streamS carry(C action)
peek
action
- peek(Object)
OT first()
OT last()
A toArray()
This is a terminal operation.
java.util.List<T> toList()
java.util.Set<T> toSet()
LongMultiset<T> toLongMultiset()
LongMultiset<T> toLongMultiset(Supplier<? extends LongMultiset<T>> supplier)
ImmutableIterator<T> iterator()
void println()
boolean isParallel()
true
if this stream would execute in parallel if executedS sequential()
S parallel()
parallel(int, Splitor)
S parallel(int maxThreadNum)
maxThreadNum
as the specified one.maxThreadNum
- parallel(int, Splitor)
S parallel(BaseStream.Splitor splitor)
splitor
as the specified one.splitor
- parallel(int, Splitor)
S parallel(int maxThreadNum, BaseStream.Splitor splitor)
maxThreadNum
and splitor
as the specified ones.
Profiler.run(1, 1, 3, "sequential", () -> Stream.of(list).operation(F)...).printResult();
Profiler.run(1, 1, 3, "parallel", () -> Stream.of(list).parallel().operation(F)...).printResult();
Here is a sample performance test with computer: CPU Intel i7-3520M 4-cores 2.9 GHz, JDK 1.8.0_101, Windows 7:
public void test_perf() {
final String[] strs = new String[10_000];
N.fill(strs, N.uuid());
final int m = 1;
final Function mapper = str -> {
long result = 0;
for (int i = 0; i < m; i++) {
result += sum(str.toCharArray()) + 1;
}
return result;
};
final MutableLong sum = MutableLong.of(0);
for (int i = 0, len = strs.length; i < len; i++) {
sum.add(mapper.apply(strs[i]));
}
final int threadNum = 1, loopNum = 100, roundNum = 3;
Profiler.run(threadNum, loopNum, roundNum, "For Loop", () -> {
long result = 0;
for (int i = 0, len = strs.length; i < len; i++) {
result += mapper.apply(strs[i]);
}
assertEquals(sum.longValue(), result);
}).printResult();
Profiler.run(threadNum, loopNum, roundNum, "JDK Sequential",
() -> assertEquals(sum.longValue(), java.util.stream.Stream.of(strs).map(mapper).mapToLong(e -> e).sum())).printResult();
Profiler.run(threadNum, loopNum, roundNum, "JDK Parallel",
() -> assertEquals(sum.longValue(), java.util.stream.Stream.of(strs).parallel().map(mapper).mapToLong(e -> e).sum())).printResult();
Profiler.run(threadNum, loopNum, roundNum, "Abcus Sequential",
() -> assertEquals(sum.longValue(), Stream.of(strs).map(mapper).mapToLong(e -> e).sum().longValue())).printResult();
Profiler.run(threadNum, loopNum, roundNum, "Abcus Parallel",
() -> assertEquals(sum.longValue(), Stream.of(strs).parallel().map(mapper).mapToLong(e -> e).sum().longValue())).printResult();
}
And test result: Unit is milliseconds. N(the number of elements) is 10_000, Q(cost per element of F, the per-element function (usually a lambda), here is mapper
) is calculated by: value of 'For loop' / N(10_000).
m = 1 | m = 10 | m = 50 | m = 100 | m = 500 | m = 1000 | |
---|---|---|---|---|---|---|
Q | 0.00002 | 0.0002 | 0.001 | 0.002 | 0.01 | 0.02 |
For Loop | 0.23 | 2.3 | 11 | 22 | 110 | 219 |
JDK Sequential | 0.28 | 2.3 | 11 | 22 | 114 | 212 |
JDK Parallel | 0.22 | 1.3 | 6 | 12 | 66 | 122 |
Abcus Sequential | 0.3 | 2 | 11 | 22 | 112 | 212 |
Abcus Parallel | 11 | 11 | 11 | 16 | 77 | 128 |
f = (int a, int b) -> a + b;
.
But if we look into the samples in the article and think about it: it just takes less than 1 milliseconds to get the max value in 100k numbers.
There is potential performance issue only if the "get the max value in 100K numbers" call many, many times in your API or single request.
Otherwise, the difference between 0.1 milliseconds to 0.5 milliseconds can be totally ignored.
Usually we meet performance issue only if Q and F is big enough. However, the performance of Lambdas/Streams APIs is closed to for loop when Q and F is big enough.
No matter in which scenario, We don't need and should not concern the performance of Lambdas/Stream APIs.
maxThreadNum
- Default value is the number of cpu-cores. Steps/operations will be executed sequentially if maxThreadNum
is 1.splitor
- The target array is split by ranges for multiple threads if splitor is splitor.ARRAY
and target stream composed by array. It looks like:
for (int i = 0; i < maxThreadNum; i++) {
final int sliceIndex = i;
futureList.add(asyncExecutor.execute(new Runnable() {
public void run() {
int cursor = fromIndex + sliceIndex * sliceSize;
final int to = toIndex - cursor > sliceSize ? cursor + sliceSize : toIndex;
while (cursor < to) {
action.accept(elements[cursor++]);
}
}
}));
}
Otherwise, each thread will get the elements from the target array/iterator in the stream one by one with the target array/iterator synchronized. It looks like:
for (int i = 0; i < maxThreadNum; i++) {
futureList.add(asyncExecutor.execute(new Runnable() {
public void run() {
T next = null;
while (true) {
synchronized (elements) {
if (cursor.intValue() < toIndex) {
next = elements[cursor.getAndIncrement()];
} else {
break;
}
}
action.accept(next);
}
}
}));
}
Using splitor.ARRAY
only when F (the per-element function (usually a lambda)) is very tiny and the cost of synchronization on the target array/iterator is too big to it.
For the F involving IO or taking 'long' to complete, choose splitor.ITERATOR
. Default value is splitor.ITERATOR
.Nth
,
com.landawn.abacus.util.Profiler#run(int, int, int, String, Runnable)
,
Understanding Parallel Stream Performance in Java SE 8,
When to use parallel Streamsint maxThreadNum()
maxThreadNum
if the stream is parallel, otherwise 1
is returned.S maxThreadNum(int maxThreadNum)
maxThreadNum
. Or return
itself, either because the stream was already parallel with same maxThreadNum
, or because
it's a sequential stream.maxThreadNum
- BaseStream.Splitor splitor()
splitor
if the stream is parallel, otherwise the default value splitor.ITERATOR
is returned.S splitor(BaseStream.Splitor splitor)
splitor
. Or return
itself, either because the stream was already parallel with same splitor
, or because
it's a sequential stream.splitor
- S onClose(java.lang.Runnable closeHandler)
close()
method
is called on the stream, and are executed in the order they were
added. All close handlers are run, even if earlier close handlers throw
exceptions. If any close handler throws an exception, the first
exception thrown will be relayed to the caller of close()
, with
any remaining exceptions added to that exception as suppressed exceptions
(unless one of the remaining exceptions is the same exception as the
first exception, since an exception cannot suppress itself.) May
return itself.
This is an intermediate operation.
closeHandler
- A task to execute when the stream is closedvoid close()
close
in interface java.lang.AutoCloseable
AutoCloseable.close()