Package com.landawn.abacus.util.stream
Class ByteStream.ByteStreamEx
java.lang.Object
com.landawn.abacus.util.stream.ByteStream
com.landawn.abacus.util.stream.ByteStream.ByteStreamEx
- All Implemented Interfaces:
Immutable
,BaseStream<Byte,
,byte[], BytePredicate, ByteConsumer, ByteList, u.OptionalByte, IndexedByte, ByteIterator, ByteStream> Closeable
,AutoCloseable
- Enclosing class:
- ByteStream
-
Nested Class Summary
Nested classes/interfaces inherited from class com.landawn.abacus.util.stream.ByteStream
ByteStream.ByteStreamEx
Nested classes/interfaces inherited from interface com.landawn.abacus.util.stream.BaseStream
BaseStream.ParallelSettings, BaseStream.Splitor
-
Method Summary
Modifier and TypeMethodDescription<SS extends BaseStream>
SS__
(Function<? super ByteStream, ? extends SS> transfer) Deprecated.void
close()
It will be called by terminal operations in final.elementAt
(long position) boolean
join
(CharSequence delimiter) parallel()
Consider usingsps(Function)
if only next operation need to be parallelized.parallel
(int maxThreadNum) Consider usingsps(int, Function)
if only next operation need to be parallelized.parallel
(int maxThreadNum, BaseStream.Splitor splitor) Returns an equivalent stream that is parallel.parallel
(int maxThreadNum, BaseStream.Splitor splitor, Executor executor) stream.parallel(maxThreadNum, splitor, executor).map(f).filter(p)...; // Replace above line of code with "sps" if only "f" need to be parallelized.// Replace above line of code with "sps" if only "f" need to be parallelized.stream.parallel(parallelSettings).map(f).filter(p)...; // Replace above line of code with "sps" if only "f" need to be parallelized.parallel
(BaseStream.Splitor splitor) stream.parallel(executor).map(f).filter(p)...; // Replace above line of code with "sps" if only "f" need to be parallelized.peek
(ByteConsumer action) void
println()
<SS extends BaseStream>
SSpsp
(Function<? super ByteStream, ? extends SS> ops) Temporarily switch the stream to sequence stream for operationops
and then switch back to parallel stream with samemaxThreadNum/splitor/asyncExecutor
.rateLimited
(double permitsPerSecond) shuffled()
This method only runs sequentially, even in parallel stream and all elements will be loaded to memory.sliding
(int windowSize) slidingToList
(int windowSize) <SS extends BaseStream>
SSsps
(int maxThreadNum, Function<? super ByteStream, ? extends SS> ops) Temporarily switch the stream to parallel stream for operationops
and then switch back to sequence stream.<SS extends BaseStream>
SSsps
(Function<? super ByteStream, ? extends SS> ops) Temporarily switch the stream to parallel stream for operationops
and then switch back to sequence stream.throwIfEmpty
(Supplier<? extends RuntimeException> exceptionSupplier) byte[]
toArray()
<SS extends BaseStream>
SStransform
(Function<? super ByteStream, ? extends SS> transfer) Methods inherited from class com.landawn.abacus.util.stream.ByteStream
allMatch, anyMatch, append, appendIfEmpty, asIntStream, average, boxed, collapse, collapse, collapse, collect, collect, concat, concat, concat, concat, concat, concatIterators, defer, delay, empty, findAny, findFirst, findFirstOrAny, findFirstOrLast, findLast, flatmap, flatMap, flatMapToInt, flatmapToObj, flatMapToObj, flatten, flatten, flatten, flatten, flattMapToObj, forEach, forEachIndexed, generate, groupTo, groupTo, iterate, iterate, iterate, iterate, iterator, kthLargest, map, mapPartial, mapToInt, mapToObj, max, merge, merge, merge, merge, merge, merge, merge, mergeWith, min, noneMatch, of, of, of, of, of, of, of, of, of, of, ofNullable, parallelMerge, parallelMerge, prepend, random, range, range, rangeClosed, rangeClosed, rangeMap, rangeMapToObj, rateLimited, reduce, reduce, repeat, scan, scan, scan, skipUntil, sum, summarize, summarizeAndPercentiles, toByteList, toMap, toMap, toMap, toMap, zip, zip, zip, zip, zip, zip, zip, zip, zip, zip, zip, zip, zip, zip, zipWith, zipWith, zipWith, zipWith
Methods inherited from class java.lang.Object
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
Methods inherited from interface com.landawn.abacus.util.stream.BaseStream
acceptIfNotEmpty, append, append, appendIfEmpty, applyIfNotEmpty, count, cycled, cycled, difference, distinct, dropWhile, dropWhile, filter, filter, first, indexed, intersection, join, last, limit, onClose, onEach, onlyOne, percentiles, prepend, prepend, reversed, reverseSorted, rotated, shuffled, skip, skip, sliding, slidingToList, sorted, split, split, splitAt, splitAt, splitToList, splitToList, step, symmetricDifference, takeWhile, toCollection, toList, toLongMultiset, toLongMultiset, toMultiset, toMultiset, toSet
-
Method Details
-
elementAt
- Specified by:
elementAt
in interfaceBaseStream<T,
A, P, C, PL, OT, IT, ITER extends Iterator<T>, S extends com.landawn.abacus.util.stream.StreamBase<T, A, P, C, PL, OT, IT, ITER, S>> - Parameters:
position
-- Returns:
-
rateLimited
- Specified by:
rateLimited
in interfaceBaseStream<T,
A, P, C, PL, OT, IT, ITER extends Iterator<T>, S extends com.landawn.abacus.util.stream.StreamBase<T, A, P, C, PL, OT, IT, ITER, S>> - Parameters:
permitsPerSecond
-- Returns:
- See Also:
-
peek
- Specified by:
peek
in interfaceBaseStream<T,
A, P, C, PL, OT, IT, ITER extends Iterator<T>, S extends com.landawn.abacus.util.stream.StreamBase<T, A, P, C, PL, OT, IT, ITER, S>> - Parameters:
action
-- Returns:
-
sliding
- Specified by:
sliding
in interfaceBaseStream<T,
A, P, C, PL, OT, IT, ITER extends Iterator<T>, S extends com.landawn.abacus.util.stream.StreamBase<T, A, P, C, PL, OT, IT, ITER, S>> - Parameters:
windowSize
-- Returns:
- See Also:
-
slidingToList
- Specified by:
slidingToList
in interfaceBaseStream<T,
A, P, C, PL, OT, IT, ITER extends Iterator<T>, S extends com.landawn.abacus.util.stream.StreamBase<T, A, P, C, PL, OT, IT, ITER, S>> - Parameters:
windowSize
-- Returns:
- See Also:
-
shuffled
Description copied from interface:BaseStream
This method only runs sequentially, even in parallel stream and all elements will be loaded to memory.- Specified by:
shuffled
in interfaceBaseStream<T,
A, P, C, PL, OT, IT, ITER extends Iterator<T>, S extends com.landawn.abacus.util.stream.StreamBase<T, A, P, C, PL, OT, IT, ITER, S>> - Returns:
-
throwIfEmpty
@SequentialOnly @IntermediateOp public ByteStream throwIfEmpty(Supplier<? extends RuntimeException> exceptionSupplier) - Specified by:
throwIfEmpty
in interfaceBaseStream<T,
A, P, C, PL, OT, IT, ITER extends Iterator<T>, S extends com.landawn.abacus.util.stream.StreamBase<T, A, P, C, PL, OT, IT, ITER, S>> - Parameters:
exceptionSupplier
-- Returns:
-
toImmutableList
- Specified by:
toImmutableList
in interfaceBaseStream<T,
A, P, C, PL, OT, IT, ITER extends Iterator<T>, S extends com.landawn.abacus.util.stream.StreamBase<T, A, P, C, PL, OT, IT, ITER, S>> - Returns:
-
toImmutableSet
- Specified by:
toImmutableSet
in interfaceBaseStream<T,
A, P, C, PL, OT, IT, ITER extends Iterator<T>, S extends com.landawn.abacus.util.stream.StreamBase<T, A, P, C, PL, OT, IT, ITER, S>> - Returns:
-
join
- Specified by:
join
in interfaceBaseStream<T,
A, P, C, PL, OT, IT, ITER extends Iterator<T>, S extends com.landawn.abacus.util.stream.StreamBase<T, A, P, C, PL, OT, IT, ITER, S>> - Parameters:
delimiter
-- Returns:
-
println
public void println()- Specified by:
println
in interfaceBaseStream<T,
A, P, C, PL, OT, IT, ITER extends Iterator<T>, S extends com.landawn.abacus.util.stream.StreamBase<T, A, P, C, PL, OT, IT, ITER, S>>
-
transform
- Specified by:
transform
in interfaceBaseStream<T,
A, P, C, PL, OT, IT, ITER extends Iterator<T>, S extends com.landawn.abacus.util.stream.StreamBase<T, A, P, C, PL, OT, IT, ITER, S>> - Type Parameters:
SS
-- Parameters:
transfer
-- Returns:
-
__
@Deprecated public <SS extends BaseStream> SS __(Function<? super ByteStream, ? extends SS> transfer) Deprecated.replaced byBaseStream.transform(Function)
- Specified by:
__
in interfaceBaseStream<T,
A, P, C, PL, OT, IT, ITER extends Iterator<T>, S extends com.landawn.abacus.util.stream.StreamBase<T, A, P, C, PL, OT, IT, ITER, S>> - Type Parameters:
SS
-- Parameters:
transfer
-- Returns:
-
isParallel
public boolean isParallel()- Specified by:
isParallel
in interfaceBaseStream<T,
A, P, C, PL, OT, IT, ITER extends Iterator<T>, S extends com.landawn.abacus.util.stream.StreamBase<T, A, P, C, PL, OT, IT, ITER, S>> - Returns:
-
sequential
- Specified by:
sequential
in interfaceBaseStream<T,
A, P, C, PL, OT, IT, ITER extends Iterator<T>, S extends com.landawn.abacus.util.stream.StreamBase<T, A, P, C, PL, OT, IT, ITER, S>> - Returns:
-
parallel
Description copied from interface:BaseStream
Consider usingsps(Function)
if only next operation need to be parallelized. For example:stream.parallel().map(f).filter(p)...; // Replace above line of code with "sps" if only "f" need to be parallelized. And "p" is fast enough to be executed in sequential Stream. stream.sps(s -> s.map(f)).filter(p)...; // Or switch the stream back sequential stream if don't use "sps". stream.parallel().map(f).sequential().filter(p)...;
In most scenarios, there could be only one operation need be parallelized in the stream. Sosps(Function)
is recommended in most of scenarios.- Specified by:
parallel
in interfaceBaseStream<T,
A, P, C, PL, OT, IT, ITER extends Iterator<T>, S extends com.landawn.abacus.util.stream.StreamBase<T, A, P, C, PL, OT, IT, ITER, S>> - Returns:
- See Also:
-
parallel
Description copied from interface:BaseStream
Consider usingsps(int, Function)
if only next operation need to be parallelized. For example:stream.parallel(maxThreadNum).map(f).filter(p)...; // Replace above line of code with "sps" if only "f" need to be parallelized. And "p" is fast enough to be executed in sequential Stream. stream.sps(maxThreadNum, s -> s.map(f)).filter(p)...; // Or switch the stream back sequential stream if don't use "sps". stream.parallel(maxThreadNum).map(f).sequential().filter(p)...;
In most scenarios, there could be only one operation need be parallelized in the stream. Sosps(int, Function)
is recommended in most of scenarios.- Specified by:
parallel
in interfaceBaseStream<T,
A, P, C, PL, OT, IT, ITER extends Iterator<T>, S extends com.landawn.abacus.util.stream.StreamBase<T, A, P, C, PL, OT, IT, ITER, S>> - Parameters:
maxThreadNum
-- Returns:
- See Also:
-
#maxThreadNumPerOperation()
-
parallel
- Specified by:
parallel
in interfaceBaseStream<T,
A, P, C, PL, OT, IT, ITER extends Iterator<T>, S extends com.landawn.abacus.util.stream.StreamBase<T, A, P, C, PL, OT, IT, ITER, S>> - Parameters:
splitor
-- Returns:
- See Also:
-
parallel
Description copied from interface:BaseStream
Returns an equivalent stream that is parallel. May return itself if the stream was already parallel with the samemaxThreadNum
andsplitor
as the specified ones.
When to use parallel Streams?- First of all, do NOT and should NOT use parallel Streams if you don't have any problem with sequential Streams, because using parallel Streams has extra cost.
- Consider using parallel Streams only when N(the number of elements) * Q(cost per element of F, the per-element function (usually a lambda)) is big enough(e.g. IO involved. Network: DB/web service request..., Reading/Writing file...).
- It's easy to test out the differences of performance by sequential Streams and parallel Streams with Profiler:
Profiler.run(1, 1, 3, "sequential", () -> Stream.of(list).operation(F)...).printResult(); Profiler.run(1, 1, 3, "parallel", () -> Stream.of(list).parallel().operation(F)...).printResult();
public void test_perf() { final String[] strs = new String[10_000]; N.fill(strs, Strings.uuid()); final int m = 10; final Function<String, Long> mapper = str -> { long result = 0; for (int i = 0; i < m; i++) { result += N.sum(str.toCharArray()) + 1; } return result; }; final MutableLong sum = MutableLong.of(0); for (int i = 0, len = strs.length; i < len; i++) { sum.add(mapper.apply(strs[i])); } final int threadNum = 1, loopNum = 100, roundNum = 3; Profiler.run(threadNum, loopNum, roundNum, "For Loop", () -> { long result = 0; for (int i = 0, len = strs.length; i < len; i++) { result += mapper.apply(strs[i]); } assertEquals(sum.longValue(), result); }).printResult(); Profiler.run(threadNum, loopNum, roundNum, "JDK Sequential", () -> assertEquals(sum.longValue(), java.util.stream.Stream.of(strs).map(mapper).mapToLong(e -> e).sum())).printResult(); Profiler.run(threadNum, loopNum, roundNum, "JDK Parallel", () -> assertEquals(sum.longValue(), java.util.stream.Stream.of(strs).parallel().map(mapper).mapToLong(e -> e).sum())).printResult(); Profiler.run(threadNum, loopNum, roundNum, "Abcus Sequential", () -> assertEquals(sum.longValue(), Stream.of(strs).map(mapper).mapToLong(e -> e).sum())) .printResult(); Profiler.run(threadNum, loopNum, roundNum, "Abcus Parallel", () -> assertEquals(sum.longValue(), Stream.of(strs).parallel().map(mapper).mapToLong(e -> e).sum())).printResult(); Profiler.run(threadNum, loopNum, roundNum, "Abcus Parallel by chunck", () -> assertEquals(sum.longValue(), Stream.of(strs).splitToList(100).parallel().map(it -> N.sumLong(it, e -> mapper.apply(e))).mapToLong(e -> e).sum())).printResult(); }
mapper
) is calculated by: value of 'For loop' / N(10_000).m = 1 m = 10 m = 50 m = 100 m = 500 m = 1000 Q 0.00002 0.0002 0.001 0.002 0.01 0.02 For Loop 0.23 2.3 11 22 110 219 JDK Sequential 0.28 2.3 11 22 114 212 JDK Parallel 0.22 1.3 6 12 66 122 Abcus Sequential 0.3 2 11 22 112 212 Abcus Parallel 11 11 11 16 77 128 - Again, do NOT and should NOT use parallel Streams if you don't have any performance problem with sequential Streams, because using parallel Streams has extra cost.
- Again, consider using parallel Streams only when N(the number of elements) * Q(cost per element of F, the per-element function (usually a lambda)) is big enough.
- The implementation of parallel Streams in Abacus is more than 10 times, slower than parallel Streams in JDK when Q is tiny(here is less than 0.0002 milliseconds by the test):
- The implementation of parallel Streams in JDK 8 still can beat the sequential/for loop when Q is tiny(Here is 0.00002 milliseconds by the test). That's amazing, considering the extra cost brought by parallel computation. It's well done.
- The implementation of parallel Streams in Abacus is pretty simple and straight forward. The extra cost(starting threads/synchronization/queue...) brought by parallel Streams in Abacus is too bigger to tiny Q(Here is less than 0.001 milliseconds by the test). But it starts to be faster than sequential Streams when Q is big enough(Here is 0.001 milliseconds by the test) and starts to catch the parallel Streams in JDK when Q is bigger(Here is 0.01 milliseconds by the test).
- Consider using the parallel Streams in Abacus when Q is big enough, specially when IO involved in F. Because one IO operation(e.g. DB/web service request..., Reading/Writing file...) usually takes 1 to 1000 milliseconds, or even longer. By the parallel Streams APIs in Abacus, it's very simple to specify max thread numbers. Sometimes, it's much faster to execute IO/Network requests with a bit more threads. It's fair to say that the parallel Streams in Abacus is high efficient, may same as or faster than the parallel Streams in JDK when Q is big enough, except F is heavy cpu-used operation. Most of the times, the Q is big enough to consider using parallel Stream is because IO/Network is involved in F.
- JDK 7 is supported by the Streams in Abacus. It's perfect to work with retrolambda on Android
- All primitive types are supported by Stream APIs in Abacus except boolean
A bit more about Lambdas/Stream APIs, you may heard that Lambdas/Stream APIs is 5 time slower than imperative programming. It's true when Q and F is VERY, VERY tiny, likef = (int a, int b) -> a + b;
. But if we look into the samples in the article and think about it: it just takes less than 1 milliseconds to get the max value in 100k numbers. There is potential performance issue only if the "get the max value in 100K numbers" call many, many times in your API or single request. Otherwise, the difference between 0.1 milliseconds to 0.5 milliseconds can be totally ignored. Usually we meet performance issue only if Q and F is big enough. However, the performance of Lambdas/Streams APIs is closed to for loop when Q and F is big enough. No matter in which scenario, We don't need and should not concern the performance of Lambdas/Stream APIs.
Although it's is parallel Streams, it doesn't means all the methods are executed in parallel. Because the sequential way is as fast, or even faster than the parallel way for some methods, or is pretty difficult, if not possible, to implement the method by parallel approach. Here are the methods which are executed sequentially even in parallel Streams.
splitXXX/splitAt/splitBy/slidingXXX/collapse, distinct, reverse, rotate, shuffle, indexed, cached, top, kthLargest, count, toArray, toList, toList, toSet, toMultiset, toLongMultiset, intersection(Collection c), difference(Collection c), symmetricDifference(Collection c), forEach(identity, accumulator, predicate), findFirstOrLast, findFirstAndLast- Specified by:
parallel
in interfaceBaseStream<T,
A, P, C, PL, OT, IT, ITER extends Iterator<T>, S extends com.landawn.abacus.util.stream.StreamBase<T, A, P, C, PL, OT, IT, ITER, S>> - Parameters:
maxThreadNum
-splitor
-- Returns:
- See Also:
-
#maxThreadNumPerOperation()
-
parallel
Description copied from interface:BaseStream
// Replace above line of code with "sps" if only "f" need to be parallelized. And "p" is fast enough to be executed in sequential Stream. stream.sps(SP.create(maxThreadNum, executor), s -> s.map(f)).filter(p)...; // Or switch the stream back sequential stream if don't use "sps". stream.parallel(maxThreadNum, executor).map(f).sequential().filter(p)...;- Specified by:
parallel
in interfaceBaseStream<T,
A, P, C, PL, OT, IT, ITER extends Iterator<T>, S extends com.landawn.abacus.util.stream.StreamBase<T, A, P, C, PL, OT, IT, ITER, S>> - Parameters:
maxThreadNum
-executor
-- Returns:
- See Also:
-
parallel
Description copied from interface:BaseStream
stream.parallel(executor).map(f).filter(p)...; // Replace above line of code with "sps" if only "f" need to be parallelized. And "p" is fast enough to be executed in sequential Stream. stream.sps(SP.create(executor), s -> s.map(f)).filter(p)...; // Or switch the stream back sequential stream if don't use "sps". stream.parallel(executor).map(f).sequential().filter(p)...;
- Specified by:
parallel
in interfaceBaseStream<T,
A, P, C, PL, OT, IT, ITER extends Iterator<T>, S extends com.landawn.abacus.util.stream.StreamBase<T, A, P, C, PL, OT, IT, ITER, S>> - Parameters:
executor
-- Returns:
- See Also:
-
parallel
Description copied from interface:BaseStream
stream.parallel(maxThreadNum, splitor, executor).map(f).filter(p)...; // Replace above line of code with "sps" if only "f" need to be parallelized. And "p" is fast enough to be executed in sequential Stream. stream.sps(SP.create(maxThreadNum, splitor, executor), s -> s.map(f)).filter(p)...; // Or switch the stream back sequential stream if don't use "sps". stream.parallel(maxThreadNum, splitor, executor).map(f).sequential().filter(p)...;
- Specified by:
parallel
in interfaceBaseStream<T,
A, P, C, PL, OT, IT, ITER extends Iterator<T>, S extends com.landawn.abacus.util.stream.StreamBase<T, A, P, C, PL, OT, IT, ITER, S>> - Parameters:
maxThreadNum
-splitor
-executor
-- Returns:
- See Also:
-
parallel
Description copied from interface:BaseStream
stream.parallel(parallelSettings).map(f).filter(p)...; // Replace above line of code with "sps" if only "f" need to be parallelized. And "p" is fast enough to be executed in sequential Stream. stream.sps(SP.create(parallelSettings), s -> s.map(f)).filter(p)...; // Or switch the stream back sequential stream if don't use "sps". stream.parallel(parallelSettings).map(f).sequential().filter(p)...;
- Specified by:
parallel
in interfaceBaseStream<T,
A, P, C, PL, OT, IT, ITER extends Iterator<T>, S extends com.landawn.abacus.util.stream.StreamBase<T, A, P, C, PL, OT, IT, ITER, S>> - Parameters:
ps
-- Returns:
- See Also:
-
sps
Description copied from interface:BaseStream
Temporarily switch the stream to parallel stream for operationops
and then switch back to sequence stream.
stream().parallel().ops(map/filter/...).sequence()
- Specified by:
sps
in interfaceBaseStream<T,
A, P, C, PL, OT, IT, ITER extends Iterator<T>, S extends com.landawn.abacus.util.stream.StreamBase<T, A, P, C, PL, OT, IT, ITER, S>> - Type Parameters:
SS
-- Parameters:
ops
-- Returns:
-
sps
public <SS extends BaseStream> SS sps(int maxThreadNum, Function<? super ByteStream, ? extends SS> ops) Description copied from interface:BaseStream
Temporarily switch the stream to parallel stream for operationops
and then switch back to sequence stream.
stream().parallel(maxThreadNum).ops(map/filter/...).sequence()
- Specified by:
sps
in interfaceBaseStream<T,
A, P, C, PL, OT, IT, ITER extends Iterator<T>, S extends com.landawn.abacus.util.stream.StreamBase<T, A, P, C, PL, OT, IT, ITER, S>> - Type Parameters:
SS
-- Parameters:
maxThreadNum
-ops
-- Returns:
-
psp
Description copied from interface:BaseStream
Temporarily switch the stream to sequence stream for operationops
and then switch back to parallel stream with samemaxThreadNum/splitor/asyncExecutor
.
stream().sequence().ops(map/filter/...).parallel(sameMaxThreadNum, sameSplitor, sameAsyncExecutor)
- Specified by:
psp
in interfaceBaseStream<T,
A, P, C, PL, OT, IT, ITER extends Iterator<T>, S extends com.landawn.abacus.util.stream.StreamBase<T, A, P, C, PL, OT, IT, ITER, S>> - Type Parameters:
SS
-- Parameters:
ops
-- Returns:
-
toArray
public byte[] toArray()- Specified by:
toArray
in interfaceBaseStream<T,
A, P, C, PL, OT, IT, ITER extends Iterator<T>, S extends com.landawn.abacus.util.stream.StreamBase<T, A, P, C, PL, OT, IT, ITER, S>> - Returns:
-
close
public void close()Description copied from interface:BaseStream
It will be called by terminal operations in final.- Specified by:
close
in interfaceAutoCloseable
- Specified by:
close
in interfaceBaseStream<T,
A, P, C, PL, OT, IT, ITER extends Iterator<T>, S extends com.landawn.abacus.util.stream.StreamBase<T, A, P, C, PL, OT, IT, ITER, S>> - Specified by:
close
in interfaceCloseable
-
BaseStream.transform(Function)