org.platanios.tensorflow.api.ops.training.optimizers.schedules
Composes the provided other schedule with this schedule and returns the resulting schedule.
Composes the provided other schedule with this schedule and returns the resulting schedule.
Applies the scheduling method to value, the current iteration in the optimization loop is step and returns the
result.
Applies the scheduling method to value, the current iteration in the optimization loop is step and returns the
result.
Value to change based on this schedule.
Option containing current iteration in the optimization loop, if one has been provided.
Potentially modified value.
IllegalArgumentException If the scheduling method requires a value for step but the provided option is
empty.
Composes this schedule with the provided, other schedule and returns the resulting schedule.
Composes this schedule with the provided, other schedule and returns the resulting schedule.
Warm-up learning rate scaling factor.
Number of warm-up steps.
Learning rate schedule that implements a warm-up scheme, similar to the one proposed in [Attention is All You Need (Section 5.3)](https://arxiv.org/pdf/1706.03762.pdf).
For the first
warmUpStepssteps the learning rate is multiplied by:exp(log(warmUpFactor) / step) ^ (warmUpSteps - step).