org.deeplearning4j.optimize.solvers.accumulation.encoding

Interface ResidualPostProcessor

All Superinterfaces:

Cloneable, Serializable

All Known Implementing Classes:

NoOpResidualPostProcessor, ResidualClippingPostProcessor

public interface ResidualPostProcessor
extends Serializable, Cloneable

ResidualPostProcessor: is (as the name suggests) is used to post process the residual vector for DL4J's gradient sharing implementation. The motivation for post processing the residual vector is to avoid it getting too large: a large residual can take many steps to communicate, which may lead to stale gradient issues. Thus most ResidualPostProcessor implementations will simply decay or clip the residual vector to keep values from getting too large relative to the current threshold.

Author:

Alex Black

Method Summary

Modifier and Type	Method and Description
ResidualPostProcessor	clone()
void	processResidual(int iteration, int epoch, double lastThreshold, INDArray residualVector)

Method Detail

processResidual

void processResidual(int iteration,
                     int epoch,
                     double lastThreshold,
                     INDArray residualVector)

Parameters:

iteration - Current iteration

epoch - Current epoch

lastThreshold - Last threshold that was used

residualVector - The current residual vector. Should be modified in-place

clone

ResidualPostProcessor clone()