org.deeplearning4j.nn.conf.layers.variational

Class VariationalAutoencoder.Builder

java.lang.Object

org.deeplearning4j.nn.conf.layers.Layer.Builder<T>

org.deeplearning4j.nn.conf.layers.BaseLayer.Builder<T>

org.deeplearning4j.nn.conf.layers.FeedForwardLayer.Builder<T>

org.deeplearning4j.nn.conf.layers.BasePretrainNetwork.Builder<VariationalAutoencoder.Builder>

org.deeplearning4j.nn.conf.layers.variational.VariationalAutoencoder.Builder

Enclosing class:

VariationalAutoencoder

public static class VariationalAutoencoder.Builder
extends BasePretrainNetwork.Builder<VariationalAutoencoder.Builder>

Field Summary

Fields inherited from class org.deeplearning4j.nn.conf.layers.BasePretrainNetwork.Builder

lossFunction, preTrainIterations, visibleBiasInit

Fields inherited from class org.deeplearning4j.nn.conf.layers.FeedForwardLayer.Builder

nIn, nOut

Fields inherited from class org.deeplearning4j.nn.conf.layers.BaseLayer.Builder

activationFn, adamMeanDecay, adamVarDecay, biasInit, biasLearningRate, dist, epsilon, gradientNormalization, gradientNormalizationThreshold, iupdater, l1, l1Bias, l2, l2Bias, learningRate, learningRatePolicy, learningRateSchedule, momentum, momentumAfter, rho, rmsDecay, updater, weightInit

Fields inherited from class org.deeplearning4j.nn.conf.layers.Layer.Builder

dropOut, layerName

Constructor Summary

Constructors

Constructor and Description
Builder()

Method Summary

Modifier and Type	Method and Description
VariationalAutoencoder	build()
VariationalAutoencoder.Builder	decoderLayerSizes(int... decoderLayerSizes) Size of the decoder layers, in units.
VariationalAutoencoder.Builder	encoderLayerSizes(int... encoderLayerSizes) Size of the encoder layers, in units.
VariationalAutoencoder.Builder	lossFunction(org.nd4j.linalg.activations.Activation outputActivationFn, org.nd4j.linalg.lossfunctions.LossFunctions.LossFunction lossFunction) Configure the VAE to use the specified loss function for the reconstruction, instead of a ReconstructionDistribution.
VariationalAutoencoder.Builder	lossFunction(org.nd4j.linalg.activations.IActivation outputActivationFn, org.nd4j.linalg.lossfunctions.ILossFunction lossFunction) Configure the VAE to use the specified loss function for the reconstruction, instead of a ReconstructionDistribution.
VariationalAutoencoder.Builder	lossFunction(org.nd4j.linalg.activations.IActivation outputActivationFn, org.nd4j.linalg.lossfunctions.LossFunctions.LossFunction lossFunction) Configure the VAE to use the specified loss function for the reconstruction, instead of a ReconstructionDistribution.
VariationalAutoencoder.Builder	nOut(int nOut) Set the size of the VAE state Z.
VariationalAutoencoder.Builder	numSamples(int numSamples) Set the number of samples per data point (from VAE state Z) used when doing pretraining.
VariationalAutoencoder.Builder	pzxActivationFn(org.nd4j.linalg.activations.IActivation activationFunction) Activation function for the input to P(z|data). Care should be taken with this, as some activation functions (relu, etc) are not suitable due to being bounded in range [0,infinity).
VariationalAutoencoder.Builder	pzxActivationFunction(org.nd4j.linalg.activations.Activation activation) Activation function for the input to P(z|data). Care should be taken with this, as some activation functions (relu, etc) are not suitable due to being bounded in range [0,infinity).
VariationalAutoencoder.Builder	pzxActivationFunction(String activationFunction) Deprecated. Use pzxActivationFunction(Activation)
VariationalAutoencoder.Builder	reconstructionDistribution(ReconstructionDistribution distribution) The reconstruction distribution for the data given the hidden state - i.e., P(data|Z). This should be selected carefully based on the type of data being modelled.

Methods inherited from class org.deeplearning4j.nn.conf.layers.BasePretrainNetwork.Builder

lossFunction, preTrainIterations, visibleBiasInit

Methods inherited from class org.deeplearning4j.nn.conf.layers.FeedForwardLayer.Builder

nIn

Methods inherited from class org.deeplearning4j.nn.conf.layers.BaseLayer.Builder

activation, activation, activation, adamMeanDecay, adamVarDecay, biasInit, biasLearningRate, dist, epsilon, gradientNormalization, gradientNormalizationThreshold, l1, l1Bias, l2, l2Bias, learningRate, learningRateDecayPolicy, learningRateSchedule, momentum, momentumAfter, rho, rmsDecay, updater, updater, weightInit

Methods inherited from class org.deeplearning4j.nn.conf.layers.Layer.Builder

dropOut, name

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Builder

public Builder()

Method Detail

encoderLayerSizes

public VariationalAutoencoder.Builder encoderLayerSizes(int... encoderLayerSizes)

Size of the encoder layers, in units. Each encoder layer is functionally equivalent to a DenseLayer. Typically the number and size of the decoder layers (set via decoderLayerSizes(int...) is similar to the encoder layers.

Parameters:

encoderLayerSizes - Size of each encoder layer in the variational autoencoder

decoderLayerSizes

public VariationalAutoencoder.Builder decoderLayerSizes(int... decoderLayerSizes)

Size of the decoder layers, in units. Each decoder layer is functionally equivalent to a DenseLayer. Typically the number and size of the decoder layers is similar to the encoder layers (set via encoderLayerSizes(int...).

Parameters:

decoderLayerSizes - Size of each deccoder layer in the variational autoencoder

reconstructionDistribution

public VariationalAutoencoder.Builder reconstructionDistribution(ReconstructionDistribution distribution)

The reconstruction distribution for the data given the hidden state - i.e., P(data|Z).
This should be selected carefully based on the type of data being modelled. For example:
- GaussianReconstructionDistribution + {identity or tanh} for real-valued (Gaussian) data
- BernoulliReconstructionDistribution + sigmoid for binary-valued (0 or 1) data

Parameters:

distribution - Reconstruction distribution

lossFunction

public VariationalAutoencoder.Builder lossFunction(org.nd4j.linalg.activations.IActivation outputActivationFn,
                                                   org.nd4j.linalg.lossfunctions.LossFunctions.LossFunction lossFunction)

Configure the VAE to use the specified loss function for the reconstruction, instead of a ReconstructionDistribution. Note that this is NOT following the standard VAE design (as per Kingma & Welling), which assumes a probabilistic output - i.e., some p(x|z). It is however a valid network configuration, allowing for optimization of more traditional objectives such as mean squared error.
Note: clearly, setting the loss function here will override any previously set recontruction distribution

Parameters:

outputActivationFn - Activation function for the output/reconstruction

lossFunction - Loss function to use

lossFunction

public VariationalAutoencoder.Builder lossFunction(org.nd4j.linalg.activations.Activation outputActivationFn,
                                                   org.nd4j.linalg.lossfunctions.LossFunctions.LossFunction lossFunction)

Configure the VAE to use the specified loss function for the reconstruction, instead of a ReconstructionDistribution. Note that this is NOT following the standard VAE design (as per Kingma & Welling), which assumes a probabilistic output - i.e., some p(x|z). It is however a valid network configuration, allowing for optimization of more traditional objectives such as mean squared error.
Note: clearly, setting the loss function here will override any previously set recontruction distribution

Parameters:

outputActivationFn - Activation function for the output/reconstruction

lossFunction - Loss function to use

lossFunction

public VariationalAutoencoder.Builder lossFunction(org.nd4j.linalg.activations.IActivation outputActivationFn,
                                                   org.nd4j.linalg.lossfunctions.ILossFunction lossFunction)

Configure the VAE to use the specified loss function for the reconstruction, instead of a ReconstructionDistribution. Note that this is NOT following the standard VAE design (as per Kingma & Welling), which assumes a probabilistic output - i.e., some p(x|z). It is however a valid network configuration, allowing for optimization of more traditional objectives such as mean squared error.
Note: clearly, setting the loss function here will override any previously set recontruction distribution

Parameters:

outputActivationFn - Activation function for the output/reconstruction

lossFunction - Loss function to use

pzxActivationFn

public VariationalAutoencoder.Builder pzxActivationFn(org.nd4j.linalg.activations.IActivation activationFunction)

Activation function for the input to P(z|data).
Care should be taken with this, as some activation functions (relu, etc) are not suitable due to being bounded in range [0,infinity).

Parameters:

activationFunction - Activation function for p(z|x)

pzxActivationFunction

@Deprecated
public VariationalAutoencoder.Builder pzxActivationFunction(String activationFunction)

Deprecated. Use pzxActivationFunction(Activation)

pzxActivationFunction

public VariationalAutoencoder.Builder pzxActivationFunction(org.nd4j.linalg.activations.Activation activation)

Activation function for the input to P(z|data).
Care should be taken with this, as some activation functions (relu, etc) are not suitable due to being bounded in range [0,infinity).

Parameters:

activation - Activation function for p(z|x)

nOut

public VariationalAutoencoder.Builder nOut(int nOut)

Set the size of the VAE state Z. This is the output size during standard forward pass, and the size of the distribution P(Z|data) during pretraining.

Overrides:

nOut in class FeedForwardLayer.Builder<VariationalAutoencoder.Builder>

Parameters:

nOut - Size of P(Z|data) and output size

numSamples

public VariationalAutoencoder.Builder numSamples(int numSamples)

Set the number of samples per data point (from VAE state Z) used when doing pretraining. Default value: 1.

This is parameter L from Kingma and Welling: "In our experiments we found that the number of samples L per datapoint can be set to 1 as long as the minibatch size M was large enough, e.g. M = 100."

Parameters:

numSamples - Number of samples per data point for pretraining

build

public VariationalAutoencoder build()

Specified by:

build in class Layer.Builder<VariationalAutoencoder.Builder>