Package org.broadinstitute.hellbender.utils.nio

Class NioFileCopierWithProgressMeter

java.lang.Object

org.broadinstitute.hellbender.utils.nio.NioFileCopierWithProgressMeter

public class NioFileCopierWithProgressMeter
extends Object

Class to copy a file using java.nio. Operates using paths. INSTANCES OF THIS CLASS ARE NOT THREAD-SAFE! Created by jonn on 8/27/18.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static interface	NioFileCopierWithProgressMeter.ChecksumCalculator	An interface that defines a method to use to calculate a checksum on an InputStream.
static enum	NioFileCopierWithProgressMeter.Verbosity	An enum to allow for verbosity of logging progress of an NioFileCopierWithProgressMeter.

Field Summary

Fields

Modifier and Type	Field	Description
protected static final int	BUFFER_SIZE_BYTES
protected String	checksum
protected static final int	COPY_SPEED_HISTORY_SIZE
protected final byte[]	copyBuffer
protected boolean	copyComplete
protected static final long	DAY_IN_MS
protected static final double	DEFAULT_PROGRESS_DISPLAY_PERCENT_INCREMENT
protected final Path	dest
protected final Queue<Double>	downloadBytesPerMilliSecond
protected String	expectedChecksum
protected boolean	formatTimeRemainingAsTimestamp
protected static final long	HOUR_IN_MS
protected static final int	KB_TO_BYTES
protected long	lastProgressTime_ns
protected double	lastProgressValue
protected MessageDigest	messageDigest
protected static final long	MINUTE_IN_MS
protected static final int	MS_TO_SEC
protected static final int	NANOS_TO_MILLIS
protected static final int	NANOS_TO_SECONDS
protected static final boolean	OVERWRITE_EXISTING_DEFAULT
protected boolean	overwriteExisting
protected long	progressBytesRead
protected double	progressPercentDisplayIncrement
protected static final long	SECOND_IN_MS
protected final Path	source
protected long	srcFileSize
protected int	srcFileSizeNumDigits
protected long	totalBytesRead
protected NioFileCopierWithProgressMeter.Verbosity	verbosity
protected static final NioFileCopierWithProgressMeter.Verbosity	VERBOSITY_DEFAULT

Constructor Summary

Constructors

Modifier	Constructor	Description
protected	NioFileCopierWithProgressMeter(Path source, Path dest, boolean overwriteExisting, NioFileCopierWithProgressMeter.Verbosity verbosity)	NioFileCopierWithProgressMeter uses a factory pattern.

Method Summary

Modifier and Type	Method	Description
protected void	calculateChecksumFromMessageDigest()
static NioFileCopierWithProgressMeter	create(Path source, Path dest)	Create an NioFileCopierWithProgressMeter.
static NioFileCopierWithProgressMeter	create(Path source, Path dest, boolean overwriteExisting)	Create an NioFileCopierWithProgressMeter.
static NioFileCopierWithProgressMeter	create(Path source, Path dest, boolean overwriteExisting, NioFileCopierWithProgressMeter.Verbosity verbosity)	Create an NioFileCopierWithProgressMeter.
protected void	determineProgessDisplayIncrement(long fileSize)
protected void	doCopy()
protected String	formatMillisecondsTime(long time_ms)
Path	getDest()
protected Duration	getRemainingDuration(long totalBytesRead, double bytesPerMillisecond)
Path	getSource()
NioFileCopierWithProgressMeter.Verbosity	getVerbosity()
NioFileCopierWithProgressMeterResults	initiateCopy()	Initiate the copy from source to dest.
boolean	isCopyComplete()
boolean	isOverwriteExisting()
protected boolean	isSilent()
protected void	logProgress(double progressValue, long totalBytesRead, double bytesPerMillisecond)
protected void	logProgressSimple(double progressValue, long totalBytesRead, double bytesPerMillisecond)
protected void	logProgressVerbose(double progressValue, long totalBytesRead, double bytesPerMillisecond)
NioFileCopierWithProgressMeter	setChecksumAlgorithmAndExpectedChecksum(String algorithm, String expectedChecksum)	Sets the messageDigest and expected checksum to be used by this NioFileCopierWithProgressMeter to validate the copied file.
NioFileCopierWithProgressMeter	setFormatTimeRemainingAsTimestamp()	Sets the logger to log the time remaining in timestamp format ala 'DD:HH:MM:ss.SS'.
NioFileCopierWithProgressMeter	setFormatTimeRemainingAsWords()	Sets the logger to log the time remaining in word format ala 'D days, H hours, M minutes, s seconds'.
NioFileCopierWithProgressMeter	setOverwriteExisting(boolean overwriteExisting)	Sets whether this NioFileCopierWithProgressMeter will overwrite dest when copying.
NioFileCopierWithProgressMeter	setVerbosity(NioFileCopierWithProgressMeter.Verbosity verbosity)	Sets the progress meter verbosity of this NioFileCopierWithProgressMeter.
protected void	updateCopyProgress(int bytesRead)
protected void	updateMessageDigest(byte[] copyBuffer, int startIndex, int endIndex)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

BUFFER_SIZE_BYTES

protected static final int BUFFER_SIZE_BYTES

See Also:

• Constant Field Values

DEFAULT_PROGRESS_DISPLAY_PERCENT_INCREMENT

protected static final double DEFAULT_PROGRESS_DISPLAY_PERCENT_INCREMENT

See Also:

• Constant Field Values

SECOND_IN_MS

protected static final long SECOND_IN_MS

See Also:

• Constant Field Values

MINUTE_IN_MS

protected static final long MINUTE_IN_MS

See Also:

• Constant Field Values

HOUR_IN_MS

protected static final long HOUR_IN_MS

See Also:

• Constant Field Values

DAY_IN_MS

protected static final long DAY_IN_MS

See Also:

• Constant Field Values

KB_TO_BYTES

protected static final int KB_TO_BYTES

See Also:

• Constant Field Values

MS_TO_SEC

protected static final int MS_TO_SEC

See Also:

• Constant Field Values

NANOS_TO_MILLIS

protected static final int NANOS_TO_MILLIS

See Also:

• Constant Field Values

NANOS_TO_SECONDS

protected static final int NANOS_TO_SECONDS

See Also:

• Constant Field Values

COPY_SPEED_HISTORY_SIZE

protected static final int COPY_SPEED_HISTORY_SIZE

See Also:

• Constant Field Values

OVERWRITE_EXISTING_DEFAULT

protected static final boolean OVERWRITE_EXISTING_DEFAULT

See Also:

• Constant Field Values

VERBOSITY_DEFAULT

protected static final NioFileCopierWithProgressMeter.Verbosity VERBOSITY_DEFAULT

source

protected final Path source

dest

protected final Path dest

srcFileSize

protected long srcFileSize

srcFileSizeNumDigits

protected int srcFileSizeNumDigits

checksum

protected String checksum

messageDigest

protected MessageDigest messageDigest

expectedChecksum

protected String expectedChecksum

overwriteExisting

protected boolean overwriteExisting

verbosity

protected NioFileCopierWithProgressMeter.Verbosity verbosity

formatTimeRemainingAsTimestamp

protected boolean formatTimeRemainingAsTimestamp

copyBuffer

protected final byte[] copyBuffer

progressPercentDisplayIncrement

protected double progressPercentDisplayIncrement

downloadBytesPerMilliSecond

protected final Queue<Double> downloadBytesPerMilliSecond

copyComplete

protected boolean copyComplete

totalBytesRead

protected long totalBytesRead

progressBytesRead

protected long progressBytesRead

lastProgressValue

protected double lastProgressValue

lastProgressTime_ns

protected long lastProgressTime_ns

Constructor Details

NioFileCopierWithProgressMeter

protected NioFileCopierWithProgressMeter(Path source,
 Path dest,
 boolean overwriteExisting,
 NioFileCopierWithProgressMeter.Verbosity verbosity)

NioFileCopierWithProgressMeter uses a factory pattern. This internal constructor is to be used by the class itself.

Parameters:

source - The Path to the source file for the copy.

dest - The Path to the destination file for the copy.

overwriteExisting - If true will overwrite an existing file in the location specified by dest.

verbosity - NioFileCopierWithProgressMeter.Verbosity of the progress progress log over the duration of the copy.

Method Details

create

public static NioFileCopierWithProgressMeter create(Path source,
 Path dest)

Create an NioFileCopierWithProgressMeter. By default the resulting NioFileCopierWithProgressMeter will not overwrite the destination if anything already exists there.

Parameters:

source - The Path to the source file for the copy.

dest - The Path to the destination file for the copy.

Returns:

An NioFileCopierWithProgressMeter initialized to copy the file located at source to the location specified by dest.

create

public static NioFileCopierWithProgressMeter create(Path source,
 Path dest,
 boolean overwriteExisting)

Create an NioFileCopierWithProgressMeter. Will periodically display progress of copying files.

Parameters:

source - The Path to the source file for the copy.

dest - The Path to the destination file for the copy.

overwriteExisting - If true will overwrite an existing file in the location specified by dest.

Returns:

An NioFileCopierWithProgressMeter initialized to copy the file located at source to the location specified by dest.

create

public static NioFileCopierWithProgressMeter create(Path source,
 Path dest,
 boolean overwriteExisting,
 NioFileCopierWithProgressMeter.Verbosity verbosity)

Create an NioFileCopierWithProgressMeter.

Parameters:

source - The Path to the source file for the copy.

dest - The Path to the destination file for the copy.

overwriteExisting - If true will overwrite an existing file in the location specified by dest.

verbosity - NioFileCopierWithProgressMeter.Verbosity of the progress progress log over the duration of the copy.

Returns:

An NioFileCopierWithProgressMeter initialized to copy the file located at source to the location specified by dest.

getSource

public Path getSource()

Returns:

A copy of the Path used as the source for this NioFileCopierWithProgressMeter.

getDest

public Path getDest()

Returns:

A copy of the Path used as the destination for this NioFileCopierWithProgressMeter.

isCopyComplete

public boolean isCopyComplete()

Returns:

true iff the source has been copied to the dest in this NioFileCopierWithProgressMeter. false otherwise.

isOverwriteExisting

public boolean isOverwriteExisting()

Returns:

true iff this NioFileCopierWithProgressMeter will overwrite dest when copying. false otherwise.

setOverwriteExisting

public NioFileCopierWithProgressMeter setOverwriteExisting(boolean overwriteExisting)

Sets whether this NioFileCopierWithProgressMeter will overwrite dest when copying.

setChecksumAlgorithmAndExpectedChecksum

public NioFileCopierWithProgressMeter setChecksumAlgorithmAndExpectedChecksum(String algorithm,
 String expectedChecksum)

Sets the messageDigest and expected checksum to be used by this NioFileCopierWithProgressMeter to validate the copied file. NOTE: Setting these values will automatically cause the file to be checked for integrity upon copy completion.

Parameters:

algorithm - String specifying the checksum algorithm to be used to calculate the checksum of the copied file.

expectedChecksum - Expected value of the checksum calculated by the given messageDigest for the copied file.

setFormatTimeRemainingAsTimestamp

public NioFileCopierWithProgressMeter setFormatTimeRemainingAsTimestamp()

Sets the logger to log the time remaining in timestamp format ala 'DD:HH:MM:ss.SS'.

setFormatTimeRemainingAsWords

public NioFileCopierWithProgressMeter setFormatTimeRemainingAsWords()

Sets the logger to log the time remaining in word format ala 'D days, H hours, M minutes, s seconds'.

getVerbosity

public NioFileCopierWithProgressMeter.Verbosity getVerbosity()

Returns:

The NioFileCopierWithProgressMeter.Verbosity at which this NioFileCopierWithProgressMeter will log copy progress.

setVerbosity

public NioFileCopierWithProgressMeter setVerbosity(NioFileCopierWithProgressMeter.Verbosity verbosity)

Sets the progress meter verbosity of this NioFileCopierWithProgressMeter.

updateMessageDigest

protected void updateMessageDigest(byte[] copyBuffer,
 int startIndex,
 int endIndex)

calculateChecksumFromMessageDigest

protected void calculateChecksumFromMessageDigest()

isSilent

protected boolean isSilent()

formatMillisecondsTime

protected String formatMillisecondsTime(long time_ms)

logProgress

protected void logProgress(double progressValue,
 long totalBytesRead,
 double bytesPerMillisecond)

getRemainingDuration

protected Duration getRemainingDuration(long totalBytesRead,
 double bytesPerMillisecond)

logProgressSimple

protected void logProgressSimple(double progressValue,
 long totalBytesRead,
 double bytesPerMillisecond)

logProgressVerbose

protected void logProgressVerbose(double progressValue,
 long totalBytesRead,
 double bytesPerMillisecond)

updateCopyProgress

protected void updateCopyProgress(int bytesRead)

determineProgessDisplayIncrement

protected void determineProgessDisplayIncrement(long fileSize)

doCopy

protected void doCopy()

initiateCopy

public NioFileCopierWithProgressMeterResults initiateCopy()

Initiate the copy from source to dest.