org.nd4j.linalg.schedule

Class CycleSchedule

java.lang.Object

org.nd4j.linalg.schedule.CycleSchedule

All Implemented Interfaces:

Serializable, Cloneable, ISchedule

public class CycleSchedule
extends Object
implements ISchedule

Cycle schedule Based on 1cycle schedule as proposed in https://arxiv.org/abs/1803.09820 Starts at initial learning rate, then linearly increases learning rate until max learning rate is reached, at that point the learning rate is decreased back to initial learning rate. When cycleLength - annealingLength is reached, the annealing period starts, and the learning rate starts decaying below the initial learning rate. The Learning rate curve Looks something like this: +-----------------------------------------+ | XX | | XX XX | | XX XX | | XX XX | | XX XX | | XX XX | | XX XX | | XX XX | | XX XX | | XX XX | | XX XX | | XX XX | | XX XX | | XX XXX | | XXX | | XXX | | | +-----------------------------------------+

Author:

Paul Dubs

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
CycleSchedule(ScheduleType scheduleType, double initialLearningRate, double maxLearningRate, int cycleLength, int annealingLength, double annealingDecay)
CycleSchedule(ScheduleType scheduleType, double maxLearningRate, int cycleLength)

Method Summary

Modifier and Type	Method and Description
ISchedule	clone()
double	valueAt(int iteration, int epoch)

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CycleSchedule

public CycleSchedule(ScheduleType scheduleType,
                     double initialLearningRate,
                     double maxLearningRate,
                     int cycleLength,
                     int annealingLength,
                     double annealingDecay)

CycleSchedule

public CycleSchedule(ScheduleType scheduleType,
                     double maxLearningRate,
                     int cycleLength)

Method Detail

valueAt

public double valueAt(int iteration,
                      int epoch)

Specified by:

valueAt in interface ISchedule

Parameters:

iteration - Current iteration number. Starts at 0

epoch - Current epoch number. Starts at 0

Returns:

Value at the current iteration/epoch for this schedule

clone

public ISchedule clone()

Specified by:

clone in interface ISchedule

Overrides:

clone in class Object