com.landawn.abacus.util.stream

Class ShortStream.ShortStreamEx

java.lang.Object

com.landawn.abacus.util.stream.ShortStream

com.landawn.abacus.util.stream.ShortStream.ShortStreamEx

All Implemented Interfaces:

Immutable, BaseStream<Short,short[],ShortPredicate,ShortConsumer,ShortList,u.OptionalShort,IndexedShort,ShortIterator,ShortStream>, Closeable, AutoCloseable

Enclosing class:

ShortStream

public abstract static class ShortStream.ShortStreamEx
extends ShortStream

Nested Class Summary

Nested classes/interfaces inherited from class com.landawn.abacus.util.stream.ShortStream

ShortStream.ShortStreamEx

Nested classes/interfaces inherited from interface com.landawn.abacus.util.stream.BaseStream

BaseStream.Splitor

Method Summary

Modifier and Type	Method and Description
<SS extends BaseStream> SS	__(Function<? super S,? extends SS> transfer)
void	close()
boolean	isParallel()
String	join(CharSequence delimiter)
S	onEach(C action) Same as peek
S	parallel()
S	parallel(BaseStream.Splitor splitor)
S	parallel(Executor executor)
S	parallel(int maxThreadNum)
S	parallel(int maxThreadNum, BaseStream.Splitor splitor) Returns an equivalent stream that is parallel.
S	parallel(int maxThreadNum, BaseStream.Splitor splitor, Executor executor)
S	parallel(int maxThreadNum, Executor executor)
void	println()
<SS extends BaseStream> SS	psp(Function<? super S,? extends SS> ops) Temporarily switch the stream to sequence for operations ops and then switch back to parallel stream with same maxThreadNum/splitor/asyncExecutor.
S	sequential()
S	shuffled() This method only run sequentially, even in parallel stream and all elements will be loaded to memory.
Stream<S>	sliding(int windowSize)
Stream<PL>	slidingToList(int windowSize)
A	toArray()
ImmutableList<T>	toImmutableList()
ImmutableSet<T>	toImmutableSet()

Methods inherited from class com.landawn.abacus.util.stream.ShortStream

allMatch, anyMatch, append, appendIfEmpty, asIntStream, average, boxed, collapse, collapse, collect, collect, concat, concat, concat, concat, empty, findAny, findFirst, findFirstOrLast, findLast, flatMap, flatMappToObj, flatMapToInt, flatMapToObj, flatten, flatten, flatten, flatten, flattMap, flattMapToObj, forEach, forEachIndexed, from, generate, iterate, iterate, iterate, iterate, iterator, kthLargest, map, mapToInt, mapToObj, max, merge, merge, merge, merge, merge, merge, merge, merge, min, noneMatch, of, of, of, of, of, of, of, of, parallelMerge, parallelMerge, prepend, random, range, range, rangeClosed, rangeClosed, rangeMap, rangeMapToObj, reduce, reduce, repeat, scan, scan, scan, skipUntil, sum, summarize, summarizeAndPercentiles, toMap, toMap, toMap, toMap, toMap, toMap, top, top, toShortList, zip, zip, zip, zip, zip, zip, zip, zip, zip, zip, zip, zip, zip, zip, zipWith, zipWith, zipWith, zipWith

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.landawn.abacus.util.stream.BaseStream

acceptIfNotEmpty, append, append, appendIfEmpty, applyIfNotEmpty, count, difference, distinct, dropWhile, dropWhile, filter, filter, first, indexed, intersection, join, last, limit, onClose, onlyOne, peek, percentiles, prepend, prepend, removeIf, removeIf, reversed, reverseSorted, rotated, shuffled, skip, skip, sliding, slidingToList, sorted, split, split, splitAt, splitAt, splitToList, splitToList, step, symmetricDifference, takeWhile, toCollection, toList, toLongMultiset, toLongMultiset, toMultiset, toMultiset, toSet

Method Detail

onEach

public S onEach(C action)

Description copied from interface: BaseStream

Same as peek

Specified by:

onEach in interface BaseStream<T,A,P,C,PL,OT,IT,ITER,S extends com.landawn.abacus.util.stream.StreamBase<T,A,P,C,PL,OT,IT,ITER,S>>

Returns:

See Also:

BaseStream.peek(Object)

sliding

public Stream<S> sliding(int windowSize)

Specified by:

sliding in interface BaseStream<T,A,P,C,PL,OT,IT,ITER,S extends com.landawn.abacus.util.stream.StreamBase<T,A,P,C,PL,OT,IT,ITER,S>>

Returns:

See Also:

BaseStream.sliding(int, int)

slidingToList

public Stream<PL> slidingToList(int windowSize)

Specified by:

slidingToList in interface BaseStream<T,A,P,C,PL,OT,IT,ITER,S extends com.landawn.abacus.util.stream.StreamBase<T,A,P,C,PL,OT,IT,ITER,S>>

Returns:

See Also:

BaseStream.sliding(int, int)

shuffled

public S shuffled()

Description copied from interface: BaseStream

This method only run sequentially, even in parallel stream and all elements will be loaded to memory.

Specified by:

shuffled in interface BaseStream<T,A,P,C,PL,OT,IT,ITER,S extends com.landawn.abacus.util.stream.StreamBase<T,A,P,C,PL,OT,IT,ITER,S>>

Returns:

toImmutableList

public ImmutableList<T> toImmutableList()

Specified by:

toImmutableList in interface BaseStream<T,A,P,C,PL,OT,IT,ITER,S extends com.landawn.abacus.util.stream.StreamBase<T,A,P,C,PL,OT,IT,ITER,S>>

toImmutableSet

public ImmutableSet<T> toImmutableSet()

Specified by:

toImmutableSet in interface BaseStream<T,A,P,C,PL,OT,IT,ITER,S extends com.landawn.abacus.util.stream.StreamBase<T,A,P,C,PL,OT,IT,ITER,S>>

join

public String join(CharSequence delimiter)

Specified by:

join in interface BaseStream<T,A,P,C,PL,OT,IT,ITER,S extends com.landawn.abacus.util.stream.StreamBase<T,A,P,C,PL,OT,IT,ITER,S>>

println

public void println()

Specified by:

println in interface BaseStream<T,A,P,C,PL,OT,IT,ITER,S extends com.landawn.abacus.util.stream.StreamBase<T,A,P,C,PL,OT,IT,ITER,S>>

__

@SequentialOnly
 @IntermediateOp
 @Beta
public <SS extends BaseStream> SS __(Function<? super S,? extends SS> transfer)

Specified by:

__ in interface BaseStream<T,A,P,C,PL,OT,IT,ITER,S extends com.landawn.abacus.util.stream.StreamBase<T,A,P,C,PL,OT,IT,ITER,S>>

isParallel

public boolean isParallel()

Specified by:

isParallel in interface BaseStream<T,A,P,C,PL,OT,IT,ITER,S extends com.landawn.abacus.util.stream.StreamBase<T,A,P,C,PL,OT,IT,ITER,S>>

sequential

public S sequential()

Specified by:

sequential in interface BaseStream<T,A,P,C,PL,OT,IT,ITER,S extends com.landawn.abacus.util.stream.StreamBase<T,A,P,C,PL,OT,IT,ITER,S>>

parallel

public S parallel()

Specified by:

parallel in interface BaseStream<T,A,P,C,PL,OT,IT,ITER,S extends com.landawn.abacus.util.stream.StreamBase<T,A,P,C,PL,OT,IT,ITER,S>>

parallel

public S parallel(int maxThreadNum)

Specified by:

parallel in interface BaseStream<T,A,P,C,PL,OT,IT,ITER,S extends com.landawn.abacus.util.stream.StreamBase<T,A,P,C,PL,OT,IT,ITER,S>>

Returns:

parallel

public S parallel(BaseStream.Splitor splitor)

Specified by:

parallel in interface BaseStream<T,A,P,C,PL,OT,IT,ITER,S extends com.landawn.abacus.util.stream.StreamBase<T,A,P,C,PL,OT,IT,ITER,S>>

Returns:

parallel

public S parallel(int maxThreadNum,
                  BaseStream.Splitor splitor)

Description copied from interface: BaseStream

Returns an equivalent stream that is parallel. May return itself if the stream was already parallel with the same maxThreadNum and splitor as the specified ones.

When to use parallel Streams?

• First of all, do NOT and should NOT use parallel Streams if you don't have any problem with sequential Streams, because using parallel Streams has extra cost.
• Consider using parallel Streams only when N(the number of elements) * Q(cost per element of F, the per-element function (usually a lambda)) is big enough(e.g. IO involved. Network: DB/web service request..., Reading/Writing file...).
• It's easy to test out the differences of performance by sequential Streams and parallel Streams with Profiler:

 
 Profiler.run(1, 1, 3, "sequential", () -> Stream.of(list).operation(F)...).printResult();
 Profiler.run(1, 1, 3, "parallel", () -> Stream.of(list).parallel().operation(F)...).printResult();
 
 

Here is a sample performance test with computer: CPU Intel i7-3520M 4-cores 2.9 GHz, JDK 1.8.0_101, Windows 7:

 
 public void test_perf() {
     final String[] strs = new String[10_000];
     N.fill(strs, N.uuid());

     final int m = 1;
     final Function mapper = str -> {
         long result = 0;
         for (int i = 0; i < m; i++) {
             result += sum(str.toCharArray()) + 1;
         }
         return result;
     };

     final MutableLong sum = MutableLong.of(0);

     for (int i = 0, len = strs.length; i < len; i++) {
         sum.add(mapper.apply(strs[i]));
     }

     final int threadNum = 1, loopNum = 100, roundNum = 3;

     Profiler.run(threadNum, loopNum, roundNum, "For Loop", () -> {
         long result = 0;
         for (int i = 0, len = strs.length; i < len; i++) {
             result += mapper.apply(strs[i]);
         }
         assertEquals(sum.longValue(), result);
     }).printResult();

     Profiler.run(threadNum, loopNum, roundNum, "JDK Sequential",
             () -> assertEquals(sum.longValue(), java.util.stream.Stream.of(strs).map(mapper).mapToLong(e -> e).sum())).printResult();

     Profiler.run(threadNum, loopNum, roundNum, "JDK Parallel",
             () -> assertEquals(sum.longValue(), java.util.stream.Stream.of(strs).parallel().map(mapper).mapToLong(e -> e).sum())).printResult();

     Profiler.run(threadNum, loopNum, roundNum, "Abcus Sequential",
             () -> assertEquals(sum.longValue(), Stream.of(strs).map(mapper).mapToLong(e -> e).sum().longValue())).printResult();

     Profiler.run(threadNum, loopNum, roundNum, "Abcus Parallel",
             () -> assertEquals(sum.longValue(), Stream.of(strs).parallel().map(mapper).mapToLong(e -> e).sum().longValue())).printResult();
 }
 
 

And test result: Unit is milliseconds. N(the number of elements) is 10_000, Q(cost per element of F, the per-element function (usually a lambda), here is mapper) is calculated by: value of 'For loop' / N(10_000).

	m = 1	m = 10	m = 50	m = 100	m = 500	m = 1000
Q	0.00002	0.0002	0.001	0.002	0.01	0.02
For Loop	0.23	2.3	11	22	110	219
JDK Sequential	0.28	2.3	11	22	114	212
JDK Parallel	0.22	1.3	6	12	66	122
Abcus Sequential	0.3	2	11	22	112	212
Abcus Parallel	11	11	11	16	77	128

Comparison:

• Again, do NOT and should NOT use parallel Streams if you don't have any performance problem with sequential Streams, because using parallel Streams has extra cost.
• Again, consider using parallel Streams only when N(the number of elements) * Q(cost per element of F, the per-element function (usually a lambda)) is big enough.
• The implementation of parallel Streams in Abacus is more than 10 times, slower than parallel Streams in JDK when Q is tiny(here is less than 0.0002 milliseconds by the test):
• The implementation of parallel Streams in JDK 8 still can beat the sequential/for loop when Q is tiny(Here is 0.00002 milliseconds by the test). That's amazing, considering the extra cost brought by parallel computation. It's well done.
• The implementation of parallel Streams in Abacus is pretty simple and straight forward. The extra cost(starting threads/synchronization/queue...) brought by parallel Streams in Abacus is too bigger to tiny Q(Here is less than 0.001 milliseconds by the test). But it starts to be faster than sequential Streams when Q is big enough(Here is 0.001 milliseconds by the test) and starts to catch the parallel Streams in JDK when Q is bigger(Here is 0.01 milliseconds by the test).
• Consider using the parallel Streams in Abacus when Q is big enough, specially when IO involved in F. Because one IO operation(e.g. DB/web service request..., Reading/Writing file...) usually takes 1 to 1000 milliseconds, or even longer. By the parallel Streams APIs in Abacus, it's very simple to specify max thread numbers. Sometimes, it's much faster to execute IO/Network requests with a bit more threads. It's fair to say that the parallel Streams in Abacus is high efficient, may same as or faster than the parallel Streams in JDK when Q is big enough, except F is heavy cpu-used operation. Most of the times, the Q is big enough to consider using parallel Stream is because IO/Network is involved in F.
• JDK 7 is supported by the Streams in Abacus. It's perfect to work with retrolambda on Android
• All primitive types are supported by Stream APIs in Abacus except boolean

A bit more about Lambdas/Stream APIs, you may heard that Lambdas/Stream APIs is 5 time slower than imperative programming. It's true when Q and F is VERY, VERY tiny, like f = (int a, int b) -> a + b;. But if we look into the samples in the article and think about it: it just takes less than 1 milliseconds to get the max value in 100k numbers. There is potential performance issue only if the "get the max value in 100K numbers" call many, many times in your API or single request. Otherwise, the difference between 0.1 milliseconds to 0.5 milliseconds can be totally ignored. Usually we meet performance issue only if Q and F is big enough. However, the performance of Lambdas/Streams APIs is closed to for loop when Q and F is big enough. No matter in which scenario, We don't need and should not concern the performance of Lambdas/Stream APIs.

Although it's is parallel Streams, it doesn't means all the methods are executed in parallel. Because the sequential way is as fast, or even faster than the parallel way for some methods, or is pretty difficult, if not possible, to implement the method by parallel approach. Here are the methods which are executed sequentially even in parallel Streams.

splitXXX/splitAt/splitBy/slidingXXX/collapse, distinct, reverse, rotate, shuffle, indexed, cached, top, kthLargest, count, toArray, toList, toList, toSet, toMultiset, toLongMultiset, intersection(Collection c), difference(Collection c), symmetricDifference(Collection c), forEach(identity, accumulator, predicate), findFirstOrLast, findFirstAndLast

Specified by:

parallel in interface BaseStream<T,A,P,C,PL,OT,IT,ITER,S extends com.landawn.abacus.util.stream.StreamBase<T,A,P,C,PL,OT,IT,ITER,S>>

Parameters:

maxThreadNum - Default value is the number of cpu-cores. Steps/operations will be executed sequentially if maxThreadNum is 1.

splitor - The target array is split by ranges for multiple threads if splitor is splitor.ARRAY and target stream composed by array. It looks like:

 for (int i = 0; i < maxThreadNum; i++) {
     final int sliceIndex = i;

     futureList.add(asyncExecutor.execute(new Runnable() {
         public void run() {
             int cursor = fromIndex + sliceIndex * sliceSize;
             final int to = toIndex - cursor > sliceSize ? cursor + sliceSize : toIndex;
             while (cursor < to) {
                 action.accept(elements[cursor++]);
             }
        }
    }));
 }
 

Otherwise, each thread will get the elements from the target array/iterator in the stream one by one with the target array/iterator synchronized. It looks like:

 for (int i = 0; i < maxThreadNum; i++) {
     futureList.add(asyncExecutor.execute(new Runnable() {
         public void run() {
             T next = null;

             while (true) {
                 synchronized (elements) {
                     if (cursor.intValue() < toIndex) {
                         next = elements[cursor.getAndIncrement()];
                     } else {
                         break;
                     }
                 }

                 action.accept(next);
             }
         }
     }));
 }
 

Using splitor.ARRAY only when F (the per-element function (usually a lambda)) is very tiny and the cost of synchronization on the target array/iterator is too big to it. For the F involving IO or taking 'long' to complete, choose splitor.ITERATOR. Default value is splitor.ITERATOR.

Returns:

See Also:

MergeResult, com.landawn.abacus.util.Profiler#run(int, int, int, String, Runnable), Understanding Parallel Stream Performance in Java SE 8, When to use parallel Streams

parallel

public S parallel(int maxThreadNum,
                  BaseStream.Splitor splitor,
                  Executor executor)

Specified by:

parallel in interface BaseStream<T,A,P,C,PL,OT,IT,ITER,S extends com.landawn.abacus.util.stream.StreamBase<T,A,P,C,PL,OT,IT,ITER,S>>

executor - should be able to execute maxThreadNum * following up operations in parallel.

Returns:

parallel

public S parallel(int maxThreadNum,
                  Executor executor)

Specified by:

parallel in interface BaseStream<T,A,P,C,PL,OT,IT,ITER,S extends com.landawn.abacus.util.stream.StreamBase<T,A,P,C,PL,OT,IT,ITER,S>>

executor - should be able to execute maxThreadNum * following up operations in parallel.

Returns:

parallel

public S parallel(Executor executor)

Specified by:

parallel in interface BaseStream<T,A,P,C,PL,OT,IT,ITER,S extends com.landawn.abacus.util.stream.StreamBase<T,A,P,C,PL,OT,IT,ITER,S>>

Parameters:

executor - should be able to execute maxThreadNum * following up operations in parallel.

Returns:

psp

@SequentialOnly
 @IntermediateOp
 @Beta
public <SS extends BaseStream> SS psp(Function<? super S,? extends SS> ops)

Description copied from interface: BaseStream

Temporarily switch the stream to sequence for operations ops and then switch back to parallel stream with same maxThreadNum/splitor/asyncExecutor.
parallelStream().sequence().map(ops).parallel(sameMaxThreadNum, sameSplitor, sameAsyncExecutor)

Specified by:

psp in interface BaseStream<T,A,P,C,PL,OT,IT,ITER,S extends com.landawn.abacus.util.stream.StreamBase<T,A,P,C,PL,OT,IT,ITER,S>>

Returns:

toArray

@SequentialOnly
 @TerminalOp
public A toArray()

Specified by:

toArray in interface BaseStream<T,A,P,C,PL,OT,IT,ITER,S extends com.landawn.abacus.util.stream.StreamBase<T,A,P,C,PL,OT,IT,ITER,S>>

close

public void close()

Specified by:

close in interface BaseStream<T,A,P,C,PL,OT,IT,ITER,S extends com.landawn.abacus.util.stream.StreamBase<T,A,P,C,PL,OT,IT,ITER,S>>

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable