org.deeplearning4j.nn.conf.layers.variational

Class ExponentialReconstructionDistribution

java.lang.Object

org.deeplearning4j.nn.conf.layers.variational.ExponentialReconstructionDistribution

All Implemented Interfaces:

Serializable, ReconstructionDistribution

public class ExponentialReconstructionDistribution
extends Object
implements ReconstructionDistribution

Exponential reconstruction distribution.
Supports data in range [0,infinity)

Parameterization used here: network models distribution parameter gamma, where gamma = log(lambda), with gamma \in (-inf, inf)

This means that an input from the decoder of gamma = 0 gives lambda = 1 which corresponds to a mean value for the expontial distribution of 1/lambda = 1

Regarding the choice of activation function: the parameterization above supports gamma in the range (-infinity,infinity) therefore a symmetric activation function such as "identity" or perhaps "tanh" is preferred.

Author:

Alex Black

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
ExponentialReconstructionDistribution()
ExponentialReconstructionDistribution(Activation activation)
ExponentialReconstructionDistribution(IActivation activationFn)
ExponentialReconstructionDistribution(String activationFn) Deprecated. Use ExponentialReconstructionDistribution(Activation)

Method Summary

Modifier and Type	Method and Description
int	distributionInputSize(int dataSize) Get the number of distribution parameters for the given input data size.
INDArray	exampleNegLogProbability(INDArray x, INDArray preOutDistributionParams) Calculate the negative log probability for each example individually
INDArray	generateAtMean(INDArray preOutDistributionParams) Generate a sample from P(x|z), where x = E[P(x|z)] i.e., return the mean value for the distribution
INDArray	generateRandom(INDArray preOutDistributionParams) Randomly sample from P(x|z) using the specified distribution parameters
INDArray	gradient(INDArray x, INDArray preOutDistributionParams) Calculate the gradient of the negative log probability with respect to the preOutDistributionParams
boolean	hasLossFunction() Does this reconstruction distribution has a standard neural network loss function (such as mean squared error, which is deterministic) or is it a standard VAE with a probabilistic reconstruction distribution?
double	negLogProbability(INDArray x, INDArray preOutDistributionParams, boolean average) Calculate the negative log probability (summed or averaged over each example in the minibatch)
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

ExponentialReconstructionDistribution

public ExponentialReconstructionDistribution()

ExponentialReconstructionDistribution

@Deprecated
public ExponentialReconstructionDistribution(String activationFn)

Deprecated. Use ExponentialReconstructionDistribution(Activation)

ExponentialReconstructionDistribution

public ExponentialReconstructionDistribution(Activation activation)

ExponentialReconstructionDistribution

public ExponentialReconstructionDistribution(IActivation activationFn)

Method Detail

hasLossFunction

public boolean hasLossFunction()

Description copied from interface: ReconstructionDistribution

Does this reconstruction distribution has a standard neural network loss function (such as mean squared error, which is deterministic) or is it a standard VAE with a probabilistic reconstruction distribution?

Specified by:

hasLossFunction in interface ReconstructionDistribution

Returns:

true if the reconstruction distribution has a loss function only (and no probabilistic reconstruction distribution)

distributionInputSize

public int distributionInputSize(int dataSize)

Description copied from interface: ReconstructionDistribution

Get the number of distribution parameters for the given input data size. For example, a Gaussian distribution has 2 parameters value (mean and log(variance)) for each data value, whereas a Bernoulli distribution has only 1 parameter value (probability) for each data value.

Specified by:

distributionInputSize in interface ReconstructionDistribution

Parameters:

dataSize - Size of the data. i.e., nIn value

Returns:

Number of distribution parameters for the given reconstruction distribution

negLogProbability

public double negLogProbability(INDArray x,
                                INDArray preOutDistributionParams,
                                boolean average)

Description copied from interface: ReconstructionDistribution

Calculate the negative log probability (summed or averaged over each example in the minibatch)

Specified by:

negLogProbability in interface ReconstructionDistribution

Parameters:

x - Data to be modelled (reconstructions)

preOutDistributionParams - Distribution parameters used by this reconstruction distribution (for example, mean and log variance values for Gaussian)

average - Whether the log probability should be averaged over the minibatch, or simply summed.

Returns:

Average or sum of negative log probability of the reconstruction given the distribution parameters

exampleNegLogProbability

public INDArray exampleNegLogProbability(INDArray x,
                                         INDArray preOutDistributionParams)

Description copied from interface: ReconstructionDistribution

Calculate the negative log probability for each example individually

Specified by:

exampleNegLogProbability in interface ReconstructionDistribution

Parameters:

x - Data to be modelled (reconstructions)

preOutDistributionParams - Distribution parameters used by this reconstruction distribution (for example, mean and log variance values for Gaussian) - before applying activation function

Returns:

Negative log probability of the reconstruction given the distribution parameters, for each example individually. Column vector, shape [numExamples, 1]

gradient

public INDArray gradient(INDArray x,
                         INDArray preOutDistributionParams)

Description copied from interface: ReconstructionDistribution

Calculate the gradient of the negative log probability with respect to the preOutDistributionParams

Specified by:

gradient in interface ReconstructionDistribution

Parameters:

x - Data

preOutDistributionParams - Distribution parameters used by this reconstruction distribution (for example, mean and log variance values for Gaussian) - before applying activation function

Returns:

Gradient with respect to the preOutDistributionParams

generateRandom

public INDArray generateRandom(INDArray preOutDistributionParams)

Description copied from interface: ReconstructionDistribution

Randomly sample from P(x|z) using the specified distribution parameters

Specified by:

generateRandom in interface ReconstructionDistribution

Parameters:

preOutDistributionParams - Distribution parameters used by this reconstruction distribution (for example, mean and log variance values for Gaussian) - before applying activation function

Returns:

A random sample of x given the distribution parameters

generateAtMean

public INDArray generateAtMean(INDArray preOutDistributionParams)

Description copied from interface: ReconstructionDistribution

Generate a sample from P(x|z), where x = E[P(x|z)] i.e., return the mean value for the distribution

Specified by:

generateAtMean in interface ReconstructionDistribution

Parameters:

preOutDistributionParams - Distribution parameters used by this reconstruction distribution (for example, mean and log variance values for Gaussian) - before applying activation function

Returns:

A deterministic sample of x (mean/expected value) given the distribution parameters

toString

public String toString()

Overrides:

toString in class Object