T
- the type of the stream elementsA
- the type of arrayP
- the type of predicateC
- the type of consumerPL
- the type of PrimitiveList/ListOT
- the type of OptionalIT
- the type of IndexedITER
- the type of IteratorS
- the type of of the stream implementing BaseStream
public interface BaseStream<T,A,P,C,PL,OT,IT,ITER,S extends BaseStream<T,A,P,C,PL,OT,IT,ITER,S>> extends AutoCloseable, Immutable
Stream
,
IntStream
,
LongStream
,
DoubleStream
Modifier and Type | Interface and Description |
---|---|
static class |
BaseStream.Splitor |
Modifier and Type | Method and Description |
---|---|
<SS extends BaseStream> |
__(Function<? super S,SS> transfer) |
<E extends Exception> |
acceptIfNotEmpty(Throwables.Consumer<? super S,E> action)
This is a terminal operation.
|
S |
append(S s) |
S |
appendIfEmpty(Supplier<? extends S> supplier) |
<R,E extends Exception> |
applyIfNotEmpty(Throwables.Function<? super S,R,E> func)
This is a terminal operation.
|
S |
carry(C action)
Same as
peek |
void |
close() |
long |
count() |
S |
difference(Collection<?> c)
This method only run sequentially, even in parallel stream. |
S |
distinct()
Returns a stream consisting of the distinct elements of this stream.
|
S |
dropWhile(P predicate)
Remove the elements until the given predicate returns false.
|
S |
dropWhile(P predicate,
C actionOnDroppedItem)
Remove the elements until the given predicate returns false.
|
S |
filter(P predicate)
Returns a stream consisting of the elements of this stream that match the given predicate.
|
S |
filter(P predicate,
C actionOnDroppedItem)
Returns a stream consisting of the elements of this stream that match the given predicate.
|
OT |
first() |
Stream<IT> |
indexed()
This method only run sequentially, even in parallel stream. |
S |
intersection(Collection<?> c)
This method only run sequentially, even in parallel stream. |
boolean |
isParallel() |
ITER |
iterator()
Deprecated.
? may cause memory/resource leak if forget to close this
Stream |
String |
join(CharSequence delimiter) |
String |
join(CharSequence delimiter,
CharSequence prefix,
CharSequence suffix) |
OT |
last() |
S |
limit(long maxSize) |
S |
onClose(Runnable closeHandler) |
OT |
onlyOne() |
S |
parallel() |
S |
parallel(BaseStream.Splitor splitor) |
S |
parallel(Executor executor) |
S |
parallel(int maxThreadNum) |
S |
parallel(int maxThreadNum,
BaseStream.Splitor splitor)
Returns an equivalent stream that is parallel.
|
S |
parallel(int maxThreadNum,
BaseStream.Splitor splitor,
Executor executor) |
S |
parallel(int maxThreadNum,
Executor executor) |
S |
peek(C action) |
u.Optional<Map<Percentage,T>> |
percentiles()
All elements will be loaded to memory and sorted if not yet. |
S |
prepend(S stream) |
void |
println() |
S |
removeIf(P predicate)
Deprecated.
|
S |
removeIf(P predicate,
C actionOnDroppedItem)
Deprecated.
|
S |
reversed()
This method only run sequentially, even in parallel stream and all elements will be loaded to memory. |
S |
reverseSorted() |
S |
rotated(int distance)
This method only run sequentially, even in parallel stream and all elements will be loaded to memory. |
S |
sequential() |
S |
shuffled()
This method only run sequentially, even in parallel stream and all elements will be loaded to memory. |
S |
shuffled(Random rnd)
This method only run sequentially, even in parallel stream and all elements will be loaded to memory. |
S |
skip(long n) |
S |
skip(long n,
C consumer) |
S |
skipUntil(P predicate) |
Stream<S> |
sliding(int windowSize) |
Stream<S> |
sliding(int windowSize,
int increment)
Stream.of(1, 2, 3, 4, 5, 6, 7, 8).sliding(3, 1).forEach(Stream::println)
output: [1, 2, 3] [2, 3, 4] [3, 4, 5] [4, 5, 6] [5, 6, 7] [6, 7, 8] ============================================================================ Stream.of(1, 2, 3, 4, 5, 6, 7, 8).sliding(3, 3).forEach(Stream::println)
output: [1, 2, 3] [4, 5, 6] [7, 8] ============================================================================ Stream.of(1, 2, 3, 4, 5, 6, 7, 5).sliding(3, 5).forEach(Stream::println)
output: [1, 2, 3] [6, 7, 8] This method only run sequentially, even in parallel stream. |
Stream<PL> |
slidingToList(int windowSize) |
Stream<PL> |
slidingToList(int windowSize,
int increment) |
S |
sorted()
Returns a stream consisting of the elements of this stream in sorted order.
|
Stream<S> |
split(int chunkSize)
Returns Stream of
S with consecutive sub sequences of the elements, each of the same size (the final sequence may be smaller). |
Stream<S> |
split(P predicate)
Split the stream by the specified predicate.
|
Stream<S> |
splitAt(int where)
Split the stream into two pieces at
where . |
Stream<S> |
splitBy(P where)
Split the stream into two pieces at
where turns to false
The first piece will be loaded into memory. |
Stream<PL> |
splitToList(int chunkSize)
Returns Stream of
PL with consecutive sub sequences of the elements, each of the same size (the final sequence may be smaller). |
Stream<PL> |
splitToList(P predicate)
Split the stream by the specified predicate.
|
S |
step(long step) |
S |
symmetricDifference(Collection<T> c)
This method only run sequentially, even in parallel stream. |
S |
takeWhile(P predicate)
Keep the elements until the given predicate returns false.
|
A |
toArray() |
<CC extends Collection<T>> |
toCollection(Supplier<? extends CC> supplier) |
ImmutableList<T> |
toImmutableList() |
ImmutableSet<T> |
toImmutableSet() |
List<T> |
toList() |
LongMultiset<T> |
toLongMultiset() |
LongMultiset<T> |
toLongMultiset(Supplier<? extends LongMultiset<T>> supplier) |
Multiset<T> |
toMultiset() |
Multiset<T> |
toMultiset(Supplier<? extends Multiset<T>> supplier) |
Set<T> |
toSet() |
@ParallelSupported @IntermediateOp S filter(P predicate)
predicate
- @ParallelSupported @IntermediateOp @Beta S filter(P predicate, C actionOnDroppedItem)
predicate
- actionOnDroppedItem
- @ParallelSupported @IntermediateOp S takeWhile(P predicate)
predicate.text(x)
returns false, any element y behind x: predicate.text(y)
should returns false.
In parallel Streams, the elements after the first element which predicate
returns false may be tested by predicate too.predicate
- @ParallelSupported @IntermediateOp S dropWhile(P predicate)
predicate.text(x)
returns true, any element y behind x: predicate.text(y)
should returns true.
In parallel Streams, the elements after the first element which predicate
returns false may be tested by predicate too.predicate
- @ParallelSupported @IntermediateOp @Beta S dropWhile(P predicate, C actionOnDroppedItem)
predicate.text(x)
returns true, any element y behind x: predicate.text(y)
should returns true.
In parallel Streams, the elements after the first element which predicate
returns false may be tested by predicate too.predicate
- actionOnDroppedItem
- @ParallelSupported @IntermediateOp @Beta S skipUntil(P predicate)
@Deprecated @ParallelSupported @IntermediateOp S removeIf(P predicate)
predicate
- @Deprecated @ParallelSupported @IntermediateOp S removeIf(P predicate, C actionOnDroppedItem)
predicate
- actionOnDroppedItem
- @SequentialOnly @IntermediateOp Stream<S> split(int chunkSize)
S
with consecutive sub sequences of the elements, each of the same size (the final sequence may be smaller).chunkSize
- the desired size of each sub sequence (the last may be smaller).@SequentialOnly @IntermediateOp Stream<PL> splitToList(int chunkSize)
PL
with consecutive sub sequences of the elements, each of the same size (the final sequence may be smaller).
chunkSize
- the desired size of each sub sequence (the last may be smaller).@SequentialOnly @IntermediateOp Stream<S> split(P predicate)
predicate
- @SequentialOnly @IntermediateOp Stream<PL> splitToList(P predicate)
predicate
- @SequentialOnly @IntermediateOp Stream<S> splitAt(int where)
where
.
The first piece will be loaded into memory.where
- @SequentialOnly @IntermediateOp Stream<S> splitBy(P where)
where
turns to false
The first piece will be loaded into memory.
Stream.of(1, 3, 2, 4, 2, 5).splitBy(i -> i <= 3).forEach(s -> s.println()); // [1, 3, 2], [4, 2, 5]
where
- @SequentialOnly @IntermediateOp Stream<S> sliding(int windowSize)
windowSize
- sliding(int, int)
@SequentialOnly @IntermediateOp Stream<PL> slidingToList(int windowSize)
windowSize
- sliding(int, int)
@SequentialOnly @IntermediateOp Stream<S> sliding(int windowSize, int increment)
Stream.of(1, 2, 3, 4, 5, 6, 7, 8).sliding(3, 1).forEach(Stream::println)
Stream.of(1, 2, 3, 4, 5, 6, 7, 8).sliding(3, 3).forEach(Stream::println)
Stream.of(1, 2, 3, 4, 5, 6, 7, 5).sliding(3, 5).forEach(Stream::println)
windowSize
- increment
- @SequentialOnly @IntermediateOp Stream<PL> slidingToList(int windowSize, int increment)
windowSize
- increment
- sliding(int, int)
@SequentialOnly @IntermediateOp S intersection(Collection<?> c)
c
- IntList.intersection(IntList)
@SequentialOnly @IntermediateOp S difference(Collection<?> c)
c
- IntList.difference(IntList)
@SequentialOnly @IntermediateOp S symmetricDifference(Collection<T> c)
c
- IntList.symmetricDifference(IntList)
@SequentialOnly @IntermediateOp u.Optional<Map<Percentage,T>> percentiles()
@SequentialOnly @IntermediateOp @TerminalOpTriggered S reversed()
@SequentialOnly @IntermediateOp @TerminalOpTriggered S rotated(int distance)
@SequentialOnly @IntermediateOp @TerminalOpTriggered S shuffled()
@SequentialOnly @IntermediateOp @TerminalOpTriggered S shuffled(Random rnd)
@SequentialOnly @IntermediateOp S distinct()
@ParallelSupported @IntermediateOp @TerminalOpTriggered S sorted()
@ParallelSupported @IntermediateOp @TerminalOpTriggered S reverseSorted()
@SequentialOnly @IntermediateOp Stream<IT> indexed()
@SequentialOnly @IntermediateOp S skip(long n)
n
- @ParallelSupported @IntermediateOp @Beta S skip(long n, C consumer)
n
- consumer
- @SequentialOnly @IntermediateOp S limit(long maxSize)
maxSize
- @SequentialOnly @IntermediateOp S step(long step)
@ParallelSupported @IntermediateOp S peek(C action)
action
- @ParallelSupported @IntermediateOp S carry(C action)
peek
action
- peek(Object)
@SequentialOnly @IntermediateOp S prepend(S stream)
@SequentialOnly @IntermediateOp S append(S s)
@SequentialOnly @IntermediateOp S appendIfEmpty(Supplier<? extends S> supplier)
@TerminalOp <R,E extends Exception> u.Optional<R> applyIfNotEmpty(Throwables.Function<? super S,R,E> func) throws E extends Exception
R
- E
- func
- E
E extends Exception
@TerminalOp <E extends Exception> If.OrElse acceptIfNotEmpty(Throwables.Consumer<? super S,E> action) throws E extends Exception
E
- action
- E
E extends Exception
@SequentialOnly @TerminalOp String join(CharSequence delimiter)
@SequentialOnly @TerminalOp String join(CharSequence delimiter, CharSequence prefix, CharSequence suffix)
@SequentialOnly @TerminalOp long count()
@SequentialOnly @TerminalOp OT first()
@SequentialOnly @TerminalOp OT last()
@SequentialOnly @TerminalOp OT onlyOne() throws DuplicatedResultException
DuplicatedResultException
- if there are more than one element in this stream.@SequentialOnly @TerminalOp A toArray()
@SequentialOnly @TerminalOp List<T> toList()
@SequentialOnly @TerminalOp Set<T> toSet()
@SequentialOnly @TerminalOp ImmutableList<T> toImmutableList()
@SequentialOnly @TerminalOp ImmutableSet<T> toImmutableSet()
@SequentialOnly @TerminalOp <CC extends Collection<T>> CC toCollection(Supplier<? extends CC> supplier)
@SequentialOnly @TerminalOp Multiset<T> toMultiset()
@SequentialOnly @TerminalOp Multiset<T> toMultiset(Supplier<? extends Multiset<T>> supplier)
@SequentialOnly @TerminalOp LongMultiset<T> toLongMultiset()
@SequentialOnly @TerminalOp LongMultiset<T> toLongMultiset(Supplier<? extends LongMultiset<T>> supplier)
@SequentialOnly @TerminalOp @Beta void println()
@Deprecated @SequentialOnly ITER iterator()
Stream
@SequentialOnly @IntermediateOp @Beta <SS extends BaseStream> SS __(Function<? super S,SS> transfer)
@SequentialOnly @IntermediateOp S onClose(Runnable closeHandler)
closeHandler
- @SequentialOnly void close()
close
in interface AutoCloseable
boolean isParallel()
@SequentialOnly @IntermediateOp S sequential()
@SequentialOnly @IntermediateOp S parallel()
@SequentialOnly @IntermediateOp S parallel(int maxThreadNum)
maxThreadNum
- @SequentialOnly @IntermediateOp S parallel(BaseStream.Splitor splitor)
splitor
- @SequentialOnly @IntermediateOp S parallel(int maxThreadNum, BaseStream.Splitor splitor)
maxThreadNum
and splitor
as the specified ones.
Profiler.run(1, 1, 3, "sequential", () -> Stream.of(list).operation(F)...).printResult();
Profiler.run(1, 1, 3, "parallel", () -> Stream.of(list).parallel().operation(F)...).printResult();
Here is a sample performance test with computer: CPU Intel i7-3520M 4-cores 2.9 GHz, JDK 1.8.0_101, Windows 7:
public void test_perf() {
final String[] strs = new String[10_000];
N.fill(strs, N.uuid());
final int m = 1;
final Function mapper = str -> {
long result = 0;
for (int i = 0; i < m; i++) {
result += sum(str.toCharArray()) + 1;
}
return result;
};
final MutableLong sum = MutableLong.of(0);
for (int i = 0, len = strs.length; i < len; i++) {
sum.add(mapper.apply(strs[i]));
}
final int threadNum = 1, loopNum = 100, roundNum = 3;
Profiler.run(threadNum, loopNum, roundNum, "For Loop", () -> {
long result = 0;
for (int i = 0, len = strs.length; i < len; i++) {
result += mapper.apply(strs[i]);
}
assertEquals(sum.longValue(), result);
}).printResult();
Profiler.run(threadNum, loopNum, roundNum, "JDK Sequential",
() -> assertEquals(sum.longValue(), java.util.stream.Stream.of(strs).map(mapper).mapToLong(e -> e).sum())).printResult();
Profiler.run(threadNum, loopNum, roundNum, "JDK Parallel",
() -> assertEquals(sum.longValue(), java.util.stream.Stream.of(strs).parallel().map(mapper).mapToLong(e -> e).sum())).printResult();
Profiler.run(threadNum, loopNum, roundNum, "Abcus Sequential",
() -> assertEquals(sum.longValue(), Stream.of(strs).map(mapper).mapToLong(e -> e).sum().longValue())).printResult();
Profiler.run(threadNum, loopNum, roundNum, "Abcus Parallel",
() -> assertEquals(sum.longValue(), Stream.of(strs).parallel().map(mapper).mapToLong(e -> e).sum().longValue())).printResult();
}
And test result: Unit is milliseconds. N(the number of elements) is 10_000, Q(cost per element of F, the per-element function (usually a lambda), here is mapper
) is calculated by: value of 'For loop' / N(10_000).
m = 1 | m = 10 | m = 50 | m = 100 | m = 500 | m = 1000 | |
---|---|---|---|---|---|---|
Q | 0.00002 | 0.0002 | 0.001 | 0.002 | 0.01 | 0.02 |
For Loop | 0.23 | 2.3 | 11 | 22 | 110 | 219 |
JDK Sequential | 0.28 | 2.3 | 11 | 22 | 114 | 212 |
JDK Parallel | 0.22 | 1.3 | 6 | 12 | 66 | 122 |
Abcus Sequential | 0.3 | 2 | 11 | 22 | 112 | 212 |
Abcus Parallel | 11 | 11 | 11 | 16 | 77 | 128 |
f = (int a, int b) -> a + b;
.
But if we look into the samples in the article and think about it: it just takes less than 1 milliseconds to get the max value in 100k numbers.
There is potential performance issue only if the "get the max value in 100K numbers" call many, many times in your API or single request.
Otherwise, the difference between 0.1 milliseconds to 0.5 milliseconds can be totally ignored.
Usually we meet performance issue only if Q and F is big enough. However, the performance of Lambdas/Streams APIs is closed to for loop when Q and F is big enough.
No matter in which scenario, We don't need and should not concern the performance of Lambdas/Stream APIs.
maxThreadNum
- Default value is the number of cpu-cores. Steps/operations will be executed sequentially if maxThreadNum
is 1.splitor
- The target array is split by ranges for multiple threads if splitor is splitor.ARRAY
and target stream composed by array. It looks like:
for (int i = 0; i < maxThreadNum; i++) {
final int sliceIndex = i;
futureList.add(asyncExecutor.execute(new Runnable() {
public void run() {
int cursor = fromIndex + sliceIndex * sliceSize;
final int to = toIndex - cursor > sliceSize ? cursor + sliceSize : toIndex;
while (cursor < to) {
action.accept(elements[cursor++]);
}
}
}));
}
Otherwise, each thread will get the elements from the target array/iterator in the stream one by one with the target array/iterator synchronized. It looks like:
for (int i = 0; i < maxThreadNum; i++) {
futureList.add(asyncExecutor.execute(new Runnable() {
public void run() {
T next = null;
while (true) {
synchronized (elements) {
if (cursor.intValue() < toIndex) {
next = elements[cursor.getAndIncrement()];
} else {
break;
}
}
action.accept(next);
}
}
}));
}
Using splitor.ARRAY
only when F (the per-element function (usually a lambda)) is very tiny and the cost of synchronization on the target array/iterator is too big to it.
For the F involving IO or taking 'long' to complete, choose splitor.ITERATOR
. Default value is splitor.ITERATOR
.MergeResult
,
com.landawn.abacus.util.Profiler#run(int, int, int, String, Runnable)
,
Understanding Parallel Stream Performance in Java SE 8,
When to use parallel Streams@SequentialOnly @IntermediateOp S parallel(int maxThreadNum, BaseStream.Splitor splitor, Executor executor)
maxThreadNum
- splitor
- executor
- should be able to execute maxThreadNum
* following up operations
in parallel.@SequentialOnly @IntermediateOp S parallel(int maxThreadNum, Executor executor)
maxThreadNum
- executor
- should be able to execute maxThreadNum
* following up operations
in parallel.@SequentialOnly @IntermediateOp S parallel(Executor executor)
executor
- should be able to execute maxThreadNum
* following up operations
in parallel.Copyright © 2020. All rights reserved.