org.deeplearning4j.optimize.solvers.accumulation

Class EncodingHandler

java.lang.Object

org.deeplearning4j.optimize.solvers.accumulation.EncodingHandler

All Implemented Interfaces:

Serializable, MessageHandler

public class EncodingHandler
extends Object
implements MessageHandler

This MessageHandler implementation is suited for debugging mostly, but still can be used in production environment if you really want that. Basic idea: updates are encoded before sharing. This handler is used as basement for distributed handler though. PLEASE NOTE: This handler does NOT provide any network connectivity. *

Author:

[email protected]

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
protected GradientsAccumulator	accumulator
protected Map<Long,ThresholdAlgorithm>	allThreadThresholdAlgorithms
protected AtomicInteger	atomicBoundary
protected ThreadLocal<AtomicBoolean>	bitmapMode
protected Double	boundary
protected NDArrayCompressor	compressor
protected ThreadLocal<org.nd4j.shade.guava.util.concurrent.AtomicDouble>	currentThreshold
protected boolean	encodingDebugMode
protected static ThreadLocal<DecimalFormat>	formatter
protected static ThreadLocal<DecimalFormat>	formatter2
protected ResidualPostProcessor	initialResidualPostProcessor
protected ThresholdAlgorithm	initialThresholdAlgorithm
protected ThreadLocal<AtomicLong>	iterations
protected ThreadLocal<AtomicBoolean>	lastIterWasDense
protected ThreadLocal<org.nd4j.shade.guava.util.concurrent.AtomicDouble>	lastSparsityRatio
protected ThreadLocal<AtomicLong>	lastStep
protected ThreadLocal<org.nd4j.shade.guava.util.concurrent.AtomicDouble>	lastThreshold
protected AtomicLong	lastThresholdLogTime
protected ThreadLocal<ResidualPostProcessor>	residualPostProcessor
static long	THRESHOLD_LOG_FREQ_MS
protected ThreadLocal<ThresholdAlgorithm>	thresholdAlgorithm

Constructor Summary

Constructors

Constructor and Description
EncodingHandler(ThresholdAlgorithm thresholdAlgorithm, ResidualPostProcessor residualPostProcessor, Double boundary, boolean encodingDebugMode)

Method Summary

Modifier and Type	Method and Description
void	applyPostProcessor(int iteration, int epoch, Double lastThreshold, INDArray residuals)
boolean	broadcastUpdates(INDArray updates, int iterationNumber, int epochNumber) This method does broadcast of given update message across network
INDArray	decodeUpdates(INDArray message) Deprecated.
INDArray	encodeUpdates(int iteration, int epoch, INDArray updates)
protected static String	format(double d)
ThresholdAlgorithm	getAverageThresholdAlgorithm() This should ONLY be called once all training threads have completed
void	initialize(GradientsAccumulator accumulator) This method does initial configuration of given MessageHandler instance
protected void	logThresholdIfReq(boolean denseUpdates, int iter, int epoch)
protected void	residualDebugOutputIfRequired(INDArray residual)
protected void	sendMessage(INDArray message, int iterationNumber, int epochNumber) This method does loops encoded data back to updates queue

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

THRESHOLD_LOG_FREQ_MS

public static final long THRESHOLD_LOG_FREQ_MS

See Also:

Constant Field Values

accumulator

protected transient GradientsAccumulator accumulator

initialThresholdAlgorithm

protected ThresholdAlgorithm initialThresholdAlgorithm

initialResidualPostProcessor

protected ResidualPostProcessor initialResidualPostProcessor

boundary

protected Double boundary

encodingDebugMode

protected boolean encodingDebugMode

compressor

protected NDArrayCompressor compressor

atomicBoundary

protected AtomicInteger atomicBoundary

thresholdAlgorithm

protected ThreadLocal<ThresholdAlgorithm> thresholdAlgorithm

allThreadThresholdAlgorithms

protected Map<Long,ThresholdAlgorithm> allThreadThresholdAlgorithms

residualPostProcessor

protected ThreadLocal<ResidualPostProcessor> residualPostProcessor

iterations

protected ThreadLocal<AtomicLong> iterations

lastStep

protected ThreadLocal<AtomicLong> lastStep

lastThreshold

protected ThreadLocal<org.nd4j.shade.guava.util.concurrent.AtomicDouble> lastThreshold

lastSparsityRatio

protected ThreadLocal<org.nd4j.shade.guava.util.concurrent.AtomicDouble> lastSparsityRatio

currentThreshold

protected ThreadLocal<org.nd4j.shade.guava.util.concurrent.AtomicDouble> currentThreshold

bitmapMode

protected ThreadLocal<AtomicBoolean> bitmapMode

lastIterWasDense

protected ThreadLocal<AtomicBoolean> lastIterWasDense

lastThresholdLogTime

protected final AtomicLong lastThresholdLogTime

formatter

protected static ThreadLocal<DecimalFormat> formatter

formatter2

protected static ThreadLocal<DecimalFormat> formatter2

Constructor Detail

EncodingHandler

public EncodingHandler(ThresholdAlgorithm thresholdAlgorithm,
                       ResidualPostProcessor residualPostProcessor,
                       Double boundary,
                       boolean encodingDebugMode)

Method Detail

initialize

public void initialize(@NonNull
                       GradientsAccumulator accumulator)

Description copied from interface: MessageHandler

This method does initial configuration of given MessageHandler instance

Specified by:

initialize in interface MessageHandler

encodeUpdates

public INDArray encodeUpdates(int iteration,
                              int epoch,
                              INDArray updates)

applyPostProcessor

public void applyPostProcessor(int iteration,
                               int epoch,
                               Double lastThreshold,
                               INDArray residuals)

decodeUpdates

@Deprecated
public INDArray decodeUpdates(INDArray message)

Deprecated.

sendMessage

protected void sendMessage(INDArray message,
                           int iterationNumber,
                           int epochNumber)

This method does loops encoded data back to updates queue

Parameters:

message -

broadcastUpdates

public boolean broadcastUpdates(INDArray updates,
                                int iterationNumber,
                                int epochNumber)

Description copied from interface: MessageHandler

This method does broadcast of given update message across network

Specified by:

broadcastUpdates in interface MessageHandler

Returns:

TRUE if something was sent, FALSE otherwise

logThresholdIfReq

protected void logThresholdIfReq(boolean denseUpdates,
                                 int iter,
                                 int epoch)

residualDebugOutputIfRequired

protected void residualDebugOutputIfRequired(INDArray residual)

format

protected static String format(double d)

getAverageThresholdAlgorithm

public ThresholdAlgorithm getAverageThresholdAlgorithm()

This should ONLY be called once all training threads have completed

Returns: