Package org.nd4j.linalg.activations.impl

Class Summary

Class	Description
ActivationCube	f(x) = x^3
ActivationELU
ActivationGELU
ActivationHardSigmoid	f(x) = min(1, max(0, 0.2*x + 0.5))
ActivationHardTanH	⎧ 1, if x > 1 f(x) = ⎨ -1, if x < -1 ⎩ x, otherwise
ActivationIdentity	f(x) = x
ActivationLReLU	Leaky RELU f(x) = max(0, x) + alpha * min(0, x) alpha defaults to 0.01
ActivationMish	https://arxiv.org/ftp/arxiv/papers/1908/1908.08681.pdf
ActivationPReLU
ActivationRationalTanh
ActivationRectifiedTanh
ActivationReLU	f(x) = max(0, x)
ActivationReLU6
ActivationRReLU
ActivationSELU	https://arxiv.org/pdf/1706.02515.pdf
ActivationSigmoid	f(x) = 1 / (1 + exp(-x))
ActivationSoftmax	f_i(x) = exp(x_i - shift) / sum_j exp(x_j - shift) where shift = max_i(x_i)
ActivationSoftPlus	f(x) = log(1+e^x)
ActivationSoftSign	f_i(x) = x_i / (1+|x_i|)
ActivationSwish	f(x) = x * sigmoid(x)
ActivationTanH	f(x) = (exp(x) - exp(-x)) / (exp(x) + exp(-x))
ActivationThresholdedReLU	Thresholded RELU f(x) = x for x > theta, f(x) = 0 otherwise.