org.deeplearning4j.nn.conf.layers.variational

Class VariationalAutoencoder

java.lang.Object

org.deeplearning4j.nn.conf.layers.Layer

org.deeplearning4j.nn.conf.layers.BaseLayer

org.deeplearning4j.nn.conf.layers.FeedForwardLayer

org.deeplearning4j.nn.conf.layers.BasePretrainNetwork

org.deeplearning4j.nn.conf.layers.variational.VariationalAutoencoder

All Implemented Interfaces:

Serializable, Cloneable, TrainingConfig

public class VariationalAutoencoder
extends BasePretrainNetwork

Variational Autoencoder layer

See: Kingma & Welling, 2013: Auto-Encoding Variational Bayes - https://arxiv.org/abs/1312.6114

This implementation allows multiple encoder and decoder layers, the number and sizes of which can be set independently.

A note on scores during pretraining: This implementation minimizes the negative of the variational lower bound objective as described in Kingma & Welling; the mathematics in that paper is based on maximization of the variational lower bound instead. Thus, scores reported during pretraining in DL4J are the negative of the variational lower bound equation in the paper. The backpropagation and learning procedure is otherwise as described there.

Author:

Alex Black

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	VariationalAutoencoder.Builder

Field Summary

Fields inherited from class org.deeplearning4j.nn.conf.layers.BasePretrainNetwork

lossFunction, visibleBiasInit

Fields inherited from class org.deeplearning4j.nn.conf.layers.FeedForwardLayer

nIn, nOut, timeDistributedFormat

Fields inherited from class org.deeplearning4j.nn.conf.layers.BaseLayer

activationFn, biasInit, biasUpdater, gainInit, gradientNormalization, gradientNormalizationThreshold, iUpdater, regularization, regularizationBias, weightInitFn, weightNoise

Fields inherited from class org.deeplearning4j.nn.conf.layers.Layer

constraints, iDropout, layerName

Method Summary

Modifier and Type	Method and Description
LayerMemoryReport	getMemoryReport(InputType inputType) This is a report of the estimated memory consumption for the given layer
ParamInitializer	initializer()
Layer	instantiate(NeuralNetConfiguration conf, Collection<TrainingListener> trainingListeners, int layerIndex, INDArray layerParamsView, boolean initializeParams, DataType networkDataType)
boolean	isPretrainParam(String paramName) Is the specified parameter a layerwise pretraining only parameter? For example, visible bias params in an autoencoder (or, decoder params in a variational autoencoder) aren't used during supervised backprop. Layers (like DenseLayer, etc) with no pretrainable parameters will return false for all (valid) inputs.

Methods inherited from class org.deeplearning4j.nn.conf.layers.FeedForwardLayer

getOutputType, getPreProcessorForInputType, setNIn

Methods inherited from class org.deeplearning4j.nn.conf.layers.BaseLayer

clone, getGradientNormalization, getRegularizationByParam, getUpdaterByParam, resetLayerDefaultConfig

Methods inherited from class org.deeplearning4j.nn.conf.layers.Layer

initializeConstraints, setDataType

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.deeplearning4j.nn.api.TrainingConfig

getGradientNormalizationThreshold, getLayerName

Method Detail

instantiate

public Layer instantiate(NeuralNetConfiguration conf,
                         Collection<TrainingListener> trainingListeners,
                         int layerIndex,
                         INDArray layerParamsView,
                         boolean initializeParams,
                         DataType networkDataType)

Specified by:

instantiate in class Layer

initializer

public ParamInitializer initializer()

Specified by:

initializer in class Layer

Returns:

The parameter initializer for this model

isPretrainParam

public boolean isPretrainParam(String paramName)

Description copied from class: Layer

Is the specified parameter a layerwise pretraining only parameter?
For example, visible bias params in an autoencoder (or, decoder params in a variational autoencoder) aren't used during supervised backprop.
Layers (like DenseLayer, etc) with no pretrainable parameters will return false for all (valid) inputs.

Specified by:

isPretrainParam in interface TrainingConfig

Overrides:

isPretrainParam in class BasePretrainNetwork

Parameters:

paramName - Parameter name/key

Returns:

True if the parameter is for layerwise pretraining only, false otherwise

getMemoryReport

public LayerMemoryReport getMemoryReport(InputType inputType)

Description copied from class: Layer

This is a report of the estimated memory consumption for the given layer

Specified by:

getMemoryReport in class Layer

Parameters:

inputType - Input type to the layer. Memory consumption is often a function of the input type

Returns:

Memory report for the layer