Interface | Description |
---|---|
ISchedule |
ISchedule: a general purpose interface for getting values according to some schedule.
|
Class | Description |
---|---|
CycleSchedule |
Cycle schedule
Based on 1cycle schedule as proposed in https://arxiv.org/abs/1803.09820
Starts at initial learning rate, then linearly increases learning rate until max learning rate is reached,
at that point the learning rate is decreased back to initial learning rate.
|
ExponentialSchedule |
An exponential schedule, with 2 parameters: initial value, and gamma.
value(i) = initialValue * gamma^i where i is the iteration or epoch (depending on the setting) |
FixedSchedule | |
InverseSchedule |
Inverse schedule, with 3 parameters: initial value, gamma and power.
value(i) = initialValue * (1 + gamma * iter)^(-power) where i is the iteration or epoch (depending on the setting) |
MapSchedule |
MapSchedule is a schedule based on specific values in a
Map<Integer,Double> .For example, if the map contains the following: (0,1.0), (10,0.5), (20, 0.2) then iteration/epoch 0 to 9 inclusive will have value 1.0, 10 to 19 will have 0.5, and 20+ will have value 0.2. Note that the map MUST have a key for position 0 - this is the initial value. |
MapSchedule.Builder |
DynamicCustomOpsBuilder for conveniently constructing map schedules
|
PolySchedule |
Polynomial decay schedule, with 3 parameters: initial value, maxIter, power.
Note that the value will be 0 after maxIter, otherwise is given by: value(i) = initialValue * (1 + i/maxIter)^(-power) where i is the iteration or epoch (depending on the setting) |
RampSchedule |
A "Wrapper" schedule that ramps up from
1/numIter * baseLR to baseLR over numIter iterations. |
SigmoidSchedule |
Sigmoid decay schedule, with 3 parameters: initial value, gamma and stepSize.
value(i) = initialValue * 1.0 / (1 + exp(-gamma * (iter - stepSize))) where i is the iteration or epoch (depending on the setting) |
StepSchedule |
Step decay schedule, with 3 parameters: initial value, gamma and step.
value(i) = initialValue * gamma^( floor(iter/step) ) where i is the iteration or epoch (depending on the setting) |
Enum | Description |
---|---|
ScheduleType |
Copyright © 2020. All rights reserved.