org.deeplearning4j.nn.conf.layers.variational

Class BernoulliReconstructionDistribution

java.lang.Object

org.deeplearning4j.nn.conf.layers.variational.BernoulliReconstructionDistribution

All Implemented Interfaces:

Serializable, ReconstructionDistribution

public class BernoulliReconstructionDistribution
extends Object
implements ReconstructionDistribution

Bernoulli reconstruction distribution for variational autoencoder.
Outputs are modelled by a Bernoulli distribution - i.e., the Bernoulli distribution should be used for binary data (all values 0 or 1); the VAE models the probability of the output being 0 or 1.
Consequently, the sigmoid activation function should be used to bound activations to the range of 0 to 1. Activation functions that do not produce outputs in the range of 0 to 1 (including relu, tanh, and many others) should be avoided.

Author:

Alex Black

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
BernoulliReconstructionDistribution() Create a BernoulliReconstructionDistribution with the default Sigmoid activation function
BernoulliReconstructionDistribution(Activation activationFn)
BernoulliReconstructionDistribution(IActivation activationFn)

Method Summary

Modifier and Type	Method and Description
int	distributionInputSize(int dataSize) Get the number of distribution parameters for the given input data size.
INDArray	exampleNegLogProbability(INDArray x, INDArray preOutDistributionParams) Calculate the negative log probability for each example individually
INDArray	generateAtMean(INDArray preOutDistributionParams) Generate a sample from P(x|z), where x = E[P(x|z)] i.e., return the mean value for the distribution
INDArray	generateRandom(INDArray preOutDistributionParams) Randomly sample from P(x|z) using the specified distribution parameters
INDArray	gradient(INDArray x, INDArray preOutDistributionParams) Calculate the gradient of the negative log probability with respect to the preOutDistributionParams
boolean	hasLossFunction() Does this reconstruction distribution has a standard neural network loss function (such as mean squared error, which is deterministic) or is it a standard VAE with a probabilistic reconstruction distribution?
double	negLogProbability(INDArray x, INDArray preOutDistributionParams, boolean average) Calculate the negative log probability (summed or averaged over each example in the minibatch)
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

BernoulliReconstructionDistribution

public BernoulliReconstructionDistribution()

Create a BernoulliReconstructionDistribution with the default Sigmoid activation function

BernoulliReconstructionDistribution

public BernoulliReconstructionDistribution(Activation activationFn)

Parameters:

activationFn - Activation function. Sigmoid generally; must be bounded in range 0 to 1

BernoulliReconstructionDistribution

public BernoulliReconstructionDistribution(IActivation activationFn)

Parameters:

activationFn - Activation function. Sigmoid generally; must be bounded in range 0 to 1

Method Detail

hasLossFunction

public boolean hasLossFunction()

Description copied from interface: ReconstructionDistribution

Does this reconstruction distribution has a standard neural network loss function (such as mean squared error, which is deterministic) or is it a standard VAE with a probabilistic reconstruction distribution?

Specified by:

hasLossFunction in interface ReconstructionDistribution

Returns:

true if the reconstruction distribution has a loss function only (and no probabilistic reconstruction distribution)

distributionInputSize

public int distributionInputSize(int dataSize)

Description copied from interface: ReconstructionDistribution

Get the number of distribution parameters for the given input data size. For example, a Gaussian distribution has 2 parameters value (mean and log(variance)) for each data value, whereas a Bernoulli distribution has only 1 parameter value (probability) for each data value.

Specified by:

distributionInputSize in interface ReconstructionDistribution

Parameters:

dataSize - Size of the data. i.e., nIn value

Returns:

Number of distribution parameters for the given reconstruction distribution

negLogProbability

public double negLogProbability(INDArray x,
                                INDArray preOutDistributionParams,
                                boolean average)

Description copied from interface: ReconstructionDistribution

Calculate the negative log probability (summed or averaged over each example in the minibatch)

Specified by:

negLogProbability in interface ReconstructionDistribution

Parameters:

x - Data to be modelled (reconstructions)

preOutDistributionParams - Distribution parameters used by this reconstruction distribution (for example, mean and log variance values for Gaussian)

average - Whether the log probability should be averaged over the minibatch, or simply summed.

Returns:

Average or sum of negative log probability of the reconstruction given the distribution parameters

exampleNegLogProbability

public INDArray exampleNegLogProbability(INDArray x,
                                         INDArray preOutDistributionParams)

Description copied from interface: ReconstructionDistribution

Calculate the negative log probability for each example individually

Specified by:

exampleNegLogProbability in interface ReconstructionDistribution

Parameters:

x - Data to be modelled (reconstructions)

preOutDistributionParams - Distribution parameters used by this reconstruction distribution (for example, mean and log variance values for Gaussian) - before applying activation function

Returns:

Negative log probability of the reconstruction given the distribution parameters, for each example individually. Column vector, shape [numExamples, 1]

gradient

public INDArray gradient(INDArray x,
                         INDArray preOutDistributionParams)

Description copied from interface: ReconstructionDistribution

Calculate the gradient of the negative log probability with respect to the preOutDistributionParams

Specified by:

gradient in interface ReconstructionDistribution

Parameters:

x - Data

preOutDistributionParams - Distribution parameters used by this reconstruction distribution (for example, mean and log variance values for Gaussian) - before applying activation function

Returns:

Gradient with respect to the preOutDistributionParams

generateRandom

public INDArray generateRandom(INDArray preOutDistributionParams)

Description copied from interface: ReconstructionDistribution

Randomly sample from P(x|z) using the specified distribution parameters

Specified by:

generateRandom in interface ReconstructionDistribution

Parameters:

preOutDistributionParams - Distribution parameters used by this reconstruction distribution (for example, mean and log variance values for Gaussian) - before applying activation function

Returns:

A random sample of x given the distribution parameters

generateAtMean

public INDArray generateAtMean(INDArray preOutDistributionParams)

Description copied from interface: ReconstructionDistribution

Generate a sample from P(x|z), where x = E[P(x|z)] i.e., return the mean value for the distribution

Specified by:

generateAtMean in interface ReconstructionDistribution

Parameters:

preOutDistributionParams - Distribution parameters used by this reconstruction distribution (for example, mean and log variance values for Gaussian) - before applying activation function

Returns:

A deterministic sample of x (mean/expected value) given the distribution parameters

toString

public String toString()

Overrides:

toString in class Object