org.deeplearning4j.nn.conf.layers.variational

Class VariationalAutoencoder

java.lang.Object

org.deeplearning4j.nn.conf.layers.Layer

org.deeplearning4j.nn.conf.layers.BaseLayer

org.deeplearning4j.nn.conf.layers.FeedForwardLayer

org.deeplearning4j.nn.conf.layers.BasePretrainNetwork

org.deeplearning4j.nn.conf.layers.variational.VariationalAutoencoder

All Implemented Interfaces:

Serializable, Cloneable

public class VariationalAutoencoder
extends BasePretrainNetwork

Variational Autoencoder layer

See: Kingma & Welling, 2013: Auto-Encoding Variational Bayes - https://arxiv.org/abs/1312.6114

This implementation allows multiple encoder and decoder layers, the number and sizes of which can be set independently.

A note on scores during pretraining: This implementation minimizes the negative of the variational lower bound objective as described in Kingma & Welling; the mathematics in that paper is based on maximization of the variational lower bound instead. Thus, scores reported during pretraining in DL4J are the negative of the variational lower bound equation in the paper. The backpropagation and learning procedure is otherwise as described there.

Author:

Alex Black

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	VariationalAutoencoder.Builder

Field Summary

Fields inherited from class org.deeplearning4j.nn.conf.layers.BasePretrainNetwork

lossFunction, visibleBiasInit

Fields inherited from class org.deeplearning4j.nn.conf.layers.FeedForwardLayer

nIn, nOut

Fields inherited from class org.deeplearning4j.nn.conf.layers.BaseLayer

activationFn, adamMeanDecay, adamVarDecay, biasInit, biasLearningRate, dist, epsilon, gradientNormalization, gradientNormalizationThreshold, iUpdater, l1, l1Bias, l2, l2Bias, learningRate, learningRateSchedule, momentum, momentumSchedule, rho, rmsDecay, updater, weightInit

Fields inherited from class org.deeplearning4j.nn.conf.layers.Layer

dropOut, layerName

Method Summary

Modifier and Type	Method and Description
double	getL1ByParam(String paramName) Get the L1 coefficient for the given parameter.
double	getL2ByParam(String paramName) Get the L2 coefficient for the given parameter.
double	getLearningRateByParam(String paramName) Get the (initial) learning rate coefficient for the given parameter.
LayerMemoryReport	getMemoryReport(InputType inputType) This is a report of the estimated memory consumption for the given layer
ParamInitializer	initializer()
Layer	instantiate(NeuralNetConfiguration conf, Collection<IterationListener> iterationListeners, int layerIndex, org.nd4j.linalg.api.ndarray.INDArray layerParamsView, boolean initializeParams)
boolean	isPretrainParam(String paramName) Is the specified parameter a layerwise pretraining only parameter? For example, visible bias params in an autoencoder (or, decoder params in a variational autoencoder) aren't used during supervised backprop. Layers (like DenseLayer, etc) with no pretrainable parameters will return false for all (valid) inputs.

Methods inherited from class org.deeplearning4j.nn.conf.layers.FeedForwardLayer

getOutputType, getPreProcessorForInputType, setNIn

Methods inherited from class org.deeplearning4j.nn.conf.layers.BaseLayer

clone, getIUpdaterByParam, getUpdaterByParam, resetLayerDefaultConfig

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

instantiate

public Layer instantiate(NeuralNetConfiguration conf,
                         Collection<IterationListener> iterationListeners,
                         int layerIndex,
                         org.nd4j.linalg.api.ndarray.INDArray layerParamsView,
                         boolean initializeParams)

Specified by:

instantiate in class Layer

initializer

public ParamInitializer initializer()

Specified by:

initializer in class Layer

getLearningRateByParam

public double getLearningRateByParam(String paramName)

Description copied from class: Layer

Get the (initial) learning rate coefficient for the given parameter. Different parameters may be configured to have different learning rates, though commonly all parameters will have the same learning rate

Overrides:

getLearningRateByParam in class BasePretrainNetwork

Parameters:

paramName - Parameter name

Returns:

Initial learning rate value for that parameter

getL1ByParam

public double getL1ByParam(String paramName)

Description copied from class: Layer

Get the L1 coefficient for the given parameter. Different parameters may have different L1 values, even for a single .l1(x) configuration. For example, biases generally aren't L1 regularized, even if weights are

Overrides:

getL1ByParam in class BasePretrainNetwork

Parameters:

paramName - Parameter name

Returns:

L1 value for that parameter

getL2ByParam

public double getL2ByParam(String paramName)

Description copied from class: Layer

Get the L2 coefficient for the given parameter. Different parameters may have different L2 values, even for a single .l2(x) configuration. For example, biases generally aren't L1 regularized, even if weights are

Overrides:

getL2ByParam in class BasePretrainNetwork

Parameters:

paramName - Parameter name

Returns:

L2 value for that parameter

isPretrainParam

public boolean isPretrainParam(String paramName)

Description copied from class: Layer

Is the specified parameter a layerwise pretraining only parameter?
For example, visible bias params in an autoencoder (or, decoder params in a variational autoencoder) aren't used during supervised backprop.
Layers (like DenseLayer, etc) with no pretrainable parameters will return false for all (valid) inputs.

Overrides:

isPretrainParam in class BasePretrainNetwork

Parameters:

paramName - Parameter name/key

Returns:

True if the parameter is for layerwise pretraining only, false otherwise

getMemoryReport

public LayerMemoryReport getMemoryReport(InputType inputType)

Description copied from class: Layer

This is a report of the estimated memory consumption for the given layer

Specified by:

getMemoryReport in class Layer

Parameters:

inputType - Input type to the layer. Memory consumption is often a function of the input type

Returns:

Memory report for the layer