ai.djl.training.optimizer

Class Optimizer.OptimizerBuilder<T extends Optimizer.OptimizerBuilder>

java.lang.Object

ai.djl.training.optimizer.Optimizer.OptimizerBuilder<T>

Direct Known Subclasses:

Adadelta.Builder, Adagrad.Builder, Adam.Builder, Nag.Builder, RmsProp.Builder, Sgd.Builder

Enclosing class:

Optimizer

public abstract static class Optimizer.OptimizerBuilder<T extends Optimizer.OptimizerBuilder>
extends java.lang.Object

The Builder to construct an Optimizer.

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	OptimizerBuilder()

Method Summary

Modifier and Type	Method and Description
T	optBeginNumUpdate(int beginNumUpdate) Sets the initial value of the number of updates.
T	optClipGrad(float clipGrad) Sets the value of the \(clipGrad\).
T	optWeightDecays(float weightDecays) Sets the value of weight decay.
protected abstract T	self()
T	setRescaleGrad(float rescaleGrad) Sets the value used to rescale the gradient.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

OptimizerBuilder

protected OptimizerBuilder()

Method Detail

setRescaleGrad

public T setRescaleGrad(float rescaleGrad)

Sets the value used to rescale the gradient. This is used to alleviate the effect of batching on the loss. Usually, the value is set to \( 1/batch_size \). Defaults to 1.

Parameters:

rescaleGrad - the value used to rescale the gradient

Returns:

this Builder

optWeightDecays

public T optWeightDecays(float weightDecays)

Sets the value of weight decay. Weight decay augments the objective function with a regularization term that penalizes large weights.

Parameters:

weightDecays - the value of weight decay to be set

Returns:

this Builder

optClipGrad

public T optClipGrad(float clipGrad)

Sets the value of the \(clipGrad\). Clips the gradient to the range of \([-clipGrad, clipGrad]\). If \(clipGrad \lt 0\), gradient clipping is turned off.

\(grad = max(min(grad, clipGrad), -clipGrad)\)

Parameters:

clipGrad - the value of \(clipGrad\)

Returns:

this Builder

optBeginNumUpdate

public T optBeginNumUpdate(int beginNumUpdate)

Sets the initial value of the number of updates.

Parameters:

beginNumUpdate - the initial value of the number of updates

Returns:

this Builder

self

protected abstract T self()