Package ai.djl.training.optimizer
Class Sgd
- java.lang.Object
-
- ai.djl.training.optimizer.Optimizer
-
- ai.djl.training.optimizer.Sgd
-
public class Sgd extends Optimizer
Sgd
is a Stochastic Gradient Descent (SGD) optimizer.If momentum is not set, it updates weights using the following update function:
\( weight = weight - learning_rate * (gradient + wd * weight) \).If momentum is set, it updates weights using the following update function:
\( state = momentum * state + learning_rate * gradient \)
\( weight -= state \)
Momentum update has better convergence rates on neural networks.- See Also:
- The D2L chapter on SGD
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static class
Sgd.Builder
The Builder to construct anSgd
object.-
Nested classes/interfaces inherited from class ai.djl.training.optimizer.Optimizer
Optimizer.OptimizerBuilder<T extends Optimizer.OptimizerBuilder>
-
-
Field Summary
-
Fields inherited from class ai.djl.training.optimizer.Optimizer
clipGrad, rescaleGrad
-
-
Constructor Summary
Constructors Modifier Constructor Description protected
Sgd(Sgd.Builder builder)
Creates a new instance ofSgd
.
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description void
update(java.lang.String parameterId, NDArray weight, NDArray grad)
Updates the parameters according to the gradients.-
Methods inherited from class ai.djl.training.optimizer.Optimizer
adadelta, adagrad, adam, getWeightDecay, nag, rmsprop, sgd, updateCount, withDefaultState
-
-
-
-
Constructor Detail
-
Sgd
protected Sgd(Sgd.Builder builder)
Creates a new instance ofSgd
.- Parameters:
builder
- the builder to create a new instance ofSgd
-
-