org.nd4j.linalg.learning

Class RmsProp

java.lang.Object

org.nd4j.linalg.learning.RmsProp

All Implemented Interfaces:

Serializable, GradientUpdater

public class RmsProp
extends Object
implements GradientUpdater

RMS Prop updates:

http://www.cs.toronto.edu/~tijmen/csc321/slides/lecture_slides_lec6.pdf http://cs231n.github.io/neural-networks-3/#ada

Author:

Adam Gibson

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	RmsProp.RmsPropAggregator

Constructor Summary

Constructors

Constructor and Description
RmsProp(double learningRate, double rmsDecay)

Method Summary

Modifier and Type	Method and Description
GradientUpdaterAggregator	getAggregator(boolean addThis) Get a GradientUpdaterAggregator.
INDArray	getGradient(INDArray gradient, int iteration) Modify the gradient to be an update
void	setStateViewArray(INDArray viewArray, int[] gradientShape, char gradientOrder, boolean initialize) For the internal updater state (if any): set this to use the provided array.
int	stateSizeForInputSize(int inputSize) For a give input size (length) array, how big is the internal state? Typically 0, 1 or 2x the input size, depending on the type of updater
void	update(Object... args) update(learningRate,momentum)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

RmsProp

public RmsProp(double learningRate,
               double rmsDecay)

Method Detail

stateSizeForInputSize

public int stateSizeForInputSize(int inputSize)

Description copied from interface: GradientUpdater

For a give input size (length) array, how big is the internal state? Typically 0, 1 or 2x the input size, depending on the type of updater

Specified by:

stateSizeForInputSize in interface GradientUpdater

Parameters:

inputSize - Length of the input array

Returns:

Number of elements in the internal state

setStateViewArray

public void setStateViewArray(INDArray viewArray,
                              int[] gradientShape,
                              char gradientOrder,
                              boolean initialize)

Description copied from interface: GradientUpdater

For the internal updater state (if any): set this to use the provided array. Used during initialization, and when restoring the updater state (after serialization, for example)

Specified by:

setStateViewArray in interface GradientUpdater

Parameters:

viewArray - Array (that is a view of a larger array) to use for the state.

initialize - If true: the updater must initialize the view array. If false: no change to view array contents

update

public void update(Object... args)

Description copied from interface: GradientUpdater

update(learningRate,momentum)

Specified by:

update in interface GradientUpdater

getGradient

public INDArray getGradient(INDArray gradient,
                            int iteration)

Description copied from interface: GradientUpdater

Modify the gradient to be an update

Specified by:

getGradient in interface GradientUpdater

Parameters:

gradient - the gradient to modify

Returns:

the modified gradient

getAggregator

public GradientUpdaterAggregator getAggregator(boolean addThis)

Description copied from interface: GradientUpdater

Get a GradientUpdaterAggregator. The aggregator is used (typically in distributed learning scenarios) to combine separate GradientUpdater instances for different networks (usually by averaging).

Specified by:

getAggregator in interface GradientUpdater

Parameters:

addThis - If true: return a GradientUpdaterAggregator with the GradientUpdater already added. If false: return an empty (uninitialized) GradientUpdaterAggregator