ai.djl.training.optimizer

Class Sgd

java.lang.Object

ai.djl.training.optimizer.Optimizer

ai.djl.training.optimizer.Sgd

public class Sgd
extends Optimizer

Sgd is a Stochastic Gradient Descent (SGD) optimizer.

If momentum is not set, it updates weights using the following update function:
\( weight = weight - learning_rate * (gradient + wd * weight) \).

If momentum is set, it updates weights using the following update function:
\( state = momentum * state + learning_rate * gradient \)
\( weight -= state \)
Momentum update has better convergence rates on neural networks.

See Also:

The D2L chapter on SGD

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	Sgd.Builder The Builder to construct an Sgd object.

Nested classes/interfaces inherited from class ai.djl.training.optimizer.Optimizer

Optimizer.OptimizerBuilder<T extends Optimizer.OptimizerBuilder>

Field Summary

Fields inherited from class ai.djl.training.optimizer.Optimizer

clipGrad, rescaleGrad

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	Sgd(Sgd.Builder builder) Creates a new instance of Sgd.

Method Summary

Modifier and Type	Method and Description
void	update(java.lang.String parameterId, NDArray weight, NDArray grad) Updates the parameters according to the gradients.

Methods inherited from class ai.djl.training.optimizer.Optimizer

adadelta, adagrad, adam, getWeightDecay, nag, rmsprop, sgd, updateCount, withDefaultState

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Sgd

protected Sgd(Sgd.Builder builder)

Creates a new instance of Sgd.

Parameters:

builder - the builder to create a new instance of Sgd

Method Detail

update

public void update(java.lang.String parameterId,
                   NDArray weight,
                   NDArray grad)

Updates the parameters according to the gradients.

Specified by:

update in class Optimizer

Parameters:

parameterId - the parameter to be updated

weight - the weights of the parameter

grad - the gradients