org.deeplearning4j.nn.layers.feedforward.rbm

Class RBM

java.lang.Object

org.deeplearning4j.nn.layers.BaseLayer<LayerConfT>

org.deeplearning4j.nn.layers.BasePretrainNetwork<RBM>

org.deeplearning4j.nn.layers.feedforward.rbm.RBM

All Implemented Interfaces:

Serializable, Cloneable, Layer, Model

public class RBM
extends BasePretrainNetwork<RBM>

Restricted Boltzmann Machine. Markov chain with gibbs sampling. Supports the following visible units: binary gaussian softmax linear Supports the following hidden units: rectified binary gaussian softmax linear Based on Hinton et al.'s work Great reference: http://www.iro.umontreal.ca/~lisa/publications2/index.php/publications/show/239

Author:

Adam Gibson

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from interface org.deeplearning4j.nn.api.Layer

Layer.TrainingMode, Layer.Type

Field Summary

Fields

Modifier and Type	Field and Description
protected org.nd4j.linalg.api.ndarray.INDArray	hiddenSigma Deprecated.
protected org.nd4j.linalg.api.ndarray.INDArray	sigma Deprecated.

Fields inherited from class org.deeplearning4j.nn.layers.BasePretrainNetwork

trainingListeners

Fields inherited from class org.deeplearning4j.nn.layers.BaseLayer

conf, dropoutApplied, dropoutMask, gradient, gradientsFlattened, gradientViews, index, input, iterationListeners, maskArray, maskState, optimizer, params, paramsFlattened, score, solver

Constructor Summary

Constructors

Constructor and Description
RBM(NeuralNetConfiguration conf)
RBM(NeuralNetConfiguration conf, org.nd4j.linalg.api.ndarray.INDArray input)

Method Summary

Modifier and Type	Method and Description
org.nd4j.linalg.api.ndarray.INDArray	activate(boolean training) Reconstructs the visible INPUT.
Pair<Gradient,org.nd4j.linalg.api.ndarray.INDArray>	backpropGradient(org.nd4j.linalg.api.ndarray.INDArray epsilon) Calculate the gradient relative to the error in the next layer
void	computeGradientAndScore() Update the score
void	contrastiveDivergence() Deprecated. No longer used; use fit methods in MultiLayerNetwork
Pair<Pair<org.nd4j.linalg.api.ndarray.INDArray,org.nd4j.linalg.api.ndarray.INDArray>,Pair<org.nd4j.linalg.api.ndarray.INDArray,org.nd4j.linalg.api.ndarray.INDArray>>	gibbhVh(org.nd4j.linalg.api.ndarray.INDArray h) Gibbs sampling step: hidden ---> visible ---> hidden
boolean	isPretrainLayer() Returns true if the layer can be trained in an unsupervised/pretrain manner (VAE, RBMs etc)
void	iterate(org.nd4j.linalg.api.ndarray.INDArray input) Deprecated. No longer used
org.nd4j.linalg.api.ndarray.INDArray	preOutput(org.nd4j.linalg.api.ndarray.INDArray v, boolean training) Raw activations
org.nd4j.linalg.api.ndarray.INDArray	propDown(org.nd4j.linalg.api.ndarray.INDArray h) Calculates the activation of the hidden: activation(h * W + vbias)
org.nd4j.linalg.api.ndarray.INDArray	propUp(org.nd4j.linalg.api.ndarray.INDArray v) Calculates the activation of the visible : sigmoid(v * W + hbias)
org.nd4j.linalg.api.ndarray.INDArray	propUp(org.nd4j.linalg.api.ndarray.INDArray v, boolean training) Calculates the activation of the visible : sigmoid(v * W + hbias)
org.nd4j.linalg.api.ndarray.INDArray	propUpDerivative(org.nd4j.linalg.api.ndarray.INDArray z)
Pair<org.nd4j.linalg.api.ndarray.INDArray,org.nd4j.linalg.api.ndarray.INDArray>	sampleHiddenGivenVisible(org.nd4j.linalg.api.ndarray.INDArray v) Binomial sampling of the hidden values given visible
Pair<org.nd4j.linalg.api.ndarray.INDArray,org.nd4j.linalg.api.ndarray.INDArray>	sampleVisibleGivenHidden(org.nd4j.linalg.api.ndarray.INDArray h) Guess the visible values given the hidden
Layer	transpose() Deprecated.

Methods inherited from class org.deeplearning4j.nn.layers.BasePretrainNetwork

calcL1, calcL2, createGradient, getCorruptedInput, numParams, numParams, params, paramTable, setListeners, setListeners, setParams, setScoreWithZ

Methods inherited from class org.deeplearning4j.nn.layers.BaseLayer

accumulateScore, activate, activate, activate, activate, activate, activationMean, applyDropOutIfNecessary, applyLearningRateScoreDecay, applyMask, batchSize, calcGradient, clear, clone, conf, createGradient, derivativeActivation, error, feedForwardMaskArray, fit, fit, getIndex, getInput, getInputMiniBatchSize, getListeners, getMaskArray, getOptimizer, getParam, gradient, gradientAndScore, initParams, input, layerConf, layerNameAndIndex, merge, paramTable, preOutput, preOutput, preOutput, score, setBackpropGradientsViewArray, setConf, setIndex, setInput, setInputMiniBatchSize, setMaskArray, setParam, setParams, setParamsViewArray, setParamTable, toString, type, update, update, validateInput

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

sigma

@Deprecated
protected org.nd4j.linalg.api.ndarray.INDArray sigma

Deprecated.

hiddenSigma

@Deprecated
protected org.nd4j.linalg.api.ndarray.INDArray hiddenSigma

Deprecated.

Constructor Detail

RBM

public RBM(NeuralNetConfiguration conf)

RBM

public RBM(NeuralNetConfiguration conf,
           org.nd4j.linalg.api.ndarray.INDArray input)

Method Detail

contrastiveDivergence

@Deprecated
public void contrastiveDivergence()

Deprecated. No longer used; use fit methods in MultiLayerNetwork

Contrastive divergence revolves around the idea of approximating the log likelihood around x1(input) with repeated sampling. Given is an energy based model: the higher k is (the more we sample the model) the more we lower the energy (increase the likelihood of the model)

and lower the likelihood (increase the energy) of the hidden samples.

Other insights: CD - k involves keeping the first k samples of a gibbs sampling of the model.

computeGradientAndScore

public void computeGradientAndScore()

Description copied from interface: Model

Update the score

Specified by:

computeGradientAndScore in interface Model

Overrides:

computeGradientAndScore in class BaseLayer<RBM>

gibbhVh

public Pair<Pair<org.nd4j.linalg.api.ndarray.INDArray,org.nd4j.linalg.api.ndarray.INDArray>,Pair<org.nd4j.linalg.api.ndarray.INDArray,org.nd4j.linalg.api.ndarray.INDArray>> gibbhVh(org.nd4j.linalg.api.ndarray.INDArray h)

Gibbs sampling step: hidden ---> visible ---> hidden

Parameters:

h - the hidden input

Returns:

the expected values and samples of both the visible samples given the hidden and the new hidden input and expected values

sampleHiddenGivenVisible

public Pair<org.nd4j.linalg.api.ndarray.INDArray,org.nd4j.linalg.api.ndarray.INDArray> sampleHiddenGivenVisible(org.nd4j.linalg.api.ndarray.INDArray v)

Binomial sampling of the hidden values given visible

Specified by:

sampleHiddenGivenVisible in class BasePretrainNetwork<RBM>

Parameters:

v - the visible values

Returns:

a binomial distribution containing the expected values and the samples

sampleVisibleGivenHidden

public Pair<org.nd4j.linalg.api.ndarray.INDArray,org.nd4j.linalg.api.ndarray.INDArray> sampleVisibleGivenHidden(org.nd4j.linalg.api.ndarray.INDArray h)

Guess the visible values given the hidden

Specified by:

sampleVisibleGivenHidden in class BasePretrainNetwork<RBM>

Parameters:

h - the hidden units

Returns:

a visible mean and sample relative to the hidden states passed in

preOutput

public org.nd4j.linalg.api.ndarray.INDArray preOutput(org.nd4j.linalg.api.ndarray.INDArray v,
                                                      boolean training)

Description copied from interface: Layer

Raw activations

Specified by:

preOutput in interface Layer

Overrides:

preOutput in class BaseLayer<RBM>

Parameters:

v - the input to transform

Returns:

the raw activation for this layer

propUp

public org.nd4j.linalg.api.ndarray.INDArray propUp(org.nd4j.linalg.api.ndarray.INDArray v)

Calculates the activation of the visible : sigmoid(v * W + hbias)

Parameters:

v - the visible layer

Returns:

the approximated activations of the visible layer

propUp

public org.nd4j.linalg.api.ndarray.INDArray propUp(org.nd4j.linalg.api.ndarray.INDArray v,
                                                   boolean training)

Calculates the activation of the visible : sigmoid(v * W + hbias)

Parameters:

v - the visible layer

Returns:

the approximated activations of the visible layer

propUpDerivative

public org.nd4j.linalg.api.ndarray.INDArray propUpDerivative(org.nd4j.linalg.api.ndarray.INDArray z)

propDown

public org.nd4j.linalg.api.ndarray.INDArray propDown(org.nd4j.linalg.api.ndarray.INDArray h)

Calculates the activation of the hidden: activation(h * W + vbias)

Parameters:

h - the hidden layer

Returns:

the approximated output of the hidden layer

activate

public org.nd4j.linalg.api.ndarray.INDArray activate(boolean training)

Reconstructs the visible INPUT. A reconstruction is a propdown of the reconstructed hidden input.

Specified by:

activate in interface Layer

Overrides:

activate in class BaseLayer<RBM>

Parameters:

training - true or false

Returns:

the reconstruction of the visible input

backpropGradient

public Pair<Gradient,org.nd4j.linalg.api.ndarray.INDArray> backpropGradient(org.nd4j.linalg.api.ndarray.INDArray epsilon)

Description copied from interface: Layer

Calculate the gradient relative to the error in the next layer

Specified by:

backpropGradient in interface Layer

Overrides:

backpropGradient in class BasePretrainNetwork<RBM>

Parameters:

epsilon - w^(L+1)*delta^(L+1). Or, equiv: dC/da, i.e., (dC/dz)*(dz/da) = dC/da, where C is cost function a=sigma(z) is activation.

Returns:

Pair where Gradient is gradient for this layer, INDArray is epsilon needed by next layer, but before element-wise multiply by sigmaPrime(z). So for standard feed-forward layer, if this layer is L, then return.getSecond() == (w^(L)*(delta^(L))^T)^T

iterate

@Deprecated
public void iterate(org.nd4j.linalg.api.ndarray.INDArray input)

Deprecated. No longer used

Description copied from class: BaseLayer

iterate one iteration of the network

Specified by:

iterate in interface Model

Overrides:

iterate in class BaseLayer<RBM>

Parameters:

input - the input to iterate on

transpose

@Deprecated
public Layer transpose()

Deprecated.

Description copied from interface: Layer

Return a transposed copy of the weights/bias (this means reverse the number of inputs and outputs on the weights)

Specified by:

transpose in interface Layer

Overrides:

transpose in class BaseLayer<RBM>

Returns:

the transposed layer

isPretrainLayer

public boolean isPretrainLayer()

Description copied from interface: Layer

Returns true if the layer can be trained in an unsupervised/pretrain manner (VAE, RBMs etc)

Returns:

true if the layer can be pretrained (using fit(INDArray), false otherwise