Package org.apache.commons.compress.archivers.zip

Class ParallelScatterZipCreator

java.lang.Object

org.apache.commons.compress.archivers.zip.ParallelScatterZipCreator

public class ParallelScatterZipCreator
extends Object

Creates a zip in parallel by using multiple threadlocal ScatterZipOutputStream instances.

Note that until 1.18, this class generally made no guarantees about the order of things written to the output file. Things that needed to come in a specific order (manifests, directories) had to be handled by the client of this class, usually by writing these things to the ZipArchiveOutputStream before calling writeTo on this class.

The client can supply an ExecutorService, but for reasons of memory model consistency, this will be shut down by this class prior to completion.

Since:

1.10

Constructor Summary

Constructors

Constructor	Description
ParallelScatterZipCreator()	Create a ParallelScatterZipCreator with default threads, which is set to the number of available processors, as defined by Runtime.availableProcessors()
ParallelScatterZipCreator(ExecutorService executorService)	Create a ParallelScatterZipCreator
ParallelScatterZipCreator(ExecutorService executorService, ScatterGatherBackingStoreSupplier backingStoreSupplier)	Create a ParallelScatterZipCreator
ParallelScatterZipCreator(ExecutorService executorService, ScatterGatherBackingStoreSupplier backingStoreSupplier, int compressionLevel)	Create a ParallelScatterZipCreator

Method Summary

Modifier and Type	Method	Description
void	addArchiveEntry(ZipArchiveEntryRequestSupplier zipArchiveEntryRequestSupplier)	Adds an archive entry to this archive.
void	addArchiveEntry(ZipArchiveEntry zipArchiveEntry, InputStreamSupplier source)	Adds an archive entry to this archive.
Callable<ScatterZipOutputStream>	createCallable(ZipArchiveEntryRequestSupplier zipArchiveEntryRequestSupplier)	Create a callable that will compress archive entry supplied by ZipArchiveEntryRequestSupplier.
Callable<ScatterZipOutputStream>	createCallable(ZipArchiveEntry zipArchiveEntry, InputStreamSupplier source)	Create a callable that will compress the given archive entry.
ScatterStatistics	getStatisticsMessage()	Returns a message describing the overall statistics of the compression run
void	submit(Callable<? extends Object> callable)	Submit a callable for compression.
void	submitStreamAwareCallable(Callable<? extends ScatterZipOutputStream> callable)	Submit a callable for compression.
void	writeTo(ZipArchiveOutputStream targetStream)	Write the contents this to the target ZipArchiveOutputStream.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ParallelScatterZipCreator

public ParallelScatterZipCreator()

Create a ParallelScatterZipCreator with default threads, which is set to the number of available processors, as defined by Runtime.availableProcessors()

ParallelScatterZipCreator

public ParallelScatterZipCreator(ExecutorService executorService)

Create a ParallelScatterZipCreator

Parameters:

executorService - The executorService to use for parallel scheduling. For technical reasons, this will be shut down by this class.

ParallelScatterZipCreator

public ParallelScatterZipCreator(ExecutorService executorService,
                                 ScatterGatherBackingStoreSupplier backingStoreSupplier)

Create a ParallelScatterZipCreator

Parameters:

executorService - The executorService to use. For technical reasons, this will be shut down by this class.

backingStoreSupplier - The supplier of backing store which shall be used

ParallelScatterZipCreator

public ParallelScatterZipCreator(ExecutorService executorService,
                                 ScatterGatherBackingStoreSupplier backingStoreSupplier,
                                 int compressionLevel)
                          throws IllegalArgumentException

Create a ParallelScatterZipCreator

Parameters:

executorService - The executorService to use. For technical reasons, this will be shut down by this class.

backingStoreSupplier - The supplier of backing store which shall be used

compressionLevel - The compression level used in compression, this value should be -1(default level) or between 0~9.

Throws:

IllegalArgumentException - if the compression level is illegal

Since:

1.21

Method Detail

addArchiveEntry

public void addArchiveEntry(ZipArchiveEntry zipArchiveEntry,
                            InputStreamSupplier source)

Adds an archive entry to this archive.

This method is expected to be called from a single client thread

Parameters:

zipArchiveEntry - The entry to add.

source - The source input stream supplier

addArchiveEntry

public void addArchiveEntry(ZipArchiveEntryRequestSupplier zipArchiveEntryRequestSupplier)

Adds an archive entry to this archive.

This method is expected to be called from a single client thread

Parameters:

zipArchiveEntryRequestSupplier - Should supply the entry to be added.

Since:

1.13

submit

public final void submit(Callable<? extends Object> callable)

Submit a callable for compression.

Parameters:

callable - The callable to run, created by createCallable, possibly wrapped by caller.

See Also:

for details of if/when to use this.

submitStreamAwareCallable

public final void submitStreamAwareCallable(Callable<? extends ScatterZipOutputStream> callable)

Submit a callable for compression.

Parameters:

callable - The callable to run, created by createCallable, possibly wrapped by caller.

Since:

1.19

See Also:

for details of if/when to use this.

createCallable

public final Callable<ScatterZipOutputStream> createCallable(ZipArchiveEntry zipArchiveEntry,
                                                             InputStreamSupplier source)

Create a callable that will compress the given archive entry.

This method is expected to be called from a single client thread.

Consider using addArchiveEntry, which wraps this method and submitStreamAwareCallable. The most common use case for using createCallable and submitStreamAwareCallable from a client is if you want to wrap the callable in something that can be prioritized by the supplied ExecutorService, for instance to process large or slow files first. Since the creation of the ExecutorService is handled by the client, all of this is up to the client.

Parameters:

zipArchiveEntry - The entry to add.

source - The source input stream supplier

Returns:

A callable that should subsequently passed to #submitStreamAwareCallable, possibly in a wrapped/adapted from. The value of this callable is not used, but any exceptions happening inside the compression will be propagated through the callable.

createCallable

public final Callable<ScatterZipOutputStream> createCallable(ZipArchiveEntryRequestSupplier zipArchiveEntryRequestSupplier)

Create a callable that will compress archive entry supplied by ZipArchiveEntryRequestSupplier.

This method is expected to be called from a single client thread.

The same as createCallable(ZipArchiveEntry, InputStreamSupplier), but the archive entry to be added is supplied by a ZipArchiveEntryRequestSupplier.

Parameters:

zipArchiveEntryRequestSupplier - Should supply the entry to be added.

Returns:

A callable that should subsequently passed to #submitStreamAwareCallable, possibly in a wrapped/adapted from. The value of this callable is not used, but any exceptions happening inside the compression will be propagated through the callable.

Since:

1.13

See Also:

createCallable(ZipArchiveEntry, InputStreamSupplier)

writeTo

public void writeTo(ZipArchiveOutputStream targetStream)
             throws IOException,
                    InterruptedException,
                    ExecutionException

Write the contents this to the target ZipArchiveOutputStream.

It may be beneficial to write things like directories and manifest files to the targetStream before calling this method.

Calling this method will shut down the ExecutorService used by this class. If any of the Callables submitted to this instance throws an exception, the archive can not be created properly and this method will throw an exception.

Parameters:

targetStream - The ZipArchiveOutputStream to receive the contents of the scatter streams

Throws:

IOException - If writing fails

InterruptedException - If we get interrupted

ExecutionException - If something happens in the parallel execution

getStatisticsMessage

public ScatterStatistics getStatisticsMessage()

Returns a message describing the overall statistics of the compression run

Returns:

A string