Activation (Deep Java Library 0.10.0 API specification)

java.lang.Object
- ai.djl.nn.Activation

```
public final class Activation
extends java.lang.Object
```
Utility class that provides activation functions and blocks.
Many networks make use of the Linear block and other similar linear transformations. However, any number of linear transformations that are composed will only result in a different linear transformation ($$f(x) = W_2(W_1x) = (W_2W_1)x = W_{combined}x$). In order to represent non-linear data, non-linear functions called activation functions are interspersed between the linear transformations. This allows the network to represent non-linear functions of increasing complexity.
See wikipedia for more details.

Method Summary

All Methods Static Methods Concrete Methods
Modifier and Type	Method and Description
`static NDArray`	`elu(NDArray array, float alpha)` Applies ELU activation on the input `NDArray`.
`static NDList`	`elu(NDList arrays, float alpha)` Applies ELU(Exponential Linear Unit) activation on the input singleton `NDList`.
`static Block`	`eluBlock(float alpha)` Creates a `LambdaBlock` that applies the `ELU` activation function in its forward function.
`static NDArray`	`gelu(NDArray array)` Applies GELU(Gaussian Error Linear Unit) activation on the input `NDArray`.
`static NDList`	`gelu(NDList arrays)` Applies GELU(Gaussian Error Linear Unit) activation on the input singleton `NDList`.
`static Block`	`geluBlock()` Creates a `LambdaBlock` that applies the `GELU` activation function in its forward function.
`static NDArray`	`leakyRelu(NDArray array, float alpha)` Applies Leaky ReLU activation on the input `NDArray`.
`static NDList`	`leakyRelu(NDList arrays, float alpha)` Applies Leaky ReLU activation on the input singleton `NDList`.
`static Block`	`leakyReluBlock(float alpha)` Creates a `LambdaBlock` that applies the `LeakyReLU` activation function in its forward function.
`static NDArray`	`mish(NDArray array)` Applies Mish activation on the input `NDArray`.
`static NDList`	`mish(NDList arrays)` Applies Mish activation on the input singleton `NDList`.
`static Block`	`mishBlock()` Creates a `LambdaBlock` that applies the `Mish` activation function in its forward function.
`static Block`	`preluBlock()` Returns a `Prelu` block.
`static NDArray`	`relu(NDArray array)` Applies ReLU activation on the input `NDArray`.
`static NDList`	`relu(NDList arrays)` Applies ReLU activation on the input singleton `NDList`.
`static Block`	`reluBlock()` Creates a `LambdaBlock` that applies the `ReLU` activation function in its forward function.
`static NDArray`	`selu(NDArray array)` Applies Scaled ELU activation on the input `NDArray`.
`static NDList`	`selu(NDList arrays)` Applies Scaled ELU activation on the input singleton `NDList`.
`static Block`	`seluBlock()` Creates a `LambdaBlock` that applies the `SELU` activation function in its forward function.
`static NDArray`	`sigmoid(NDArray array)` Applies Sigmoid activation on the input `NDArray`.
`static NDList`	`sigmoid(NDList arrays)` Applies Sigmoid activation on the input singleton `NDList`.
`static Block`	`sigmoidBlock()` Creates a `LambdaBlock` that applies the `Sigmoid` activation function in its forward function.
`static NDArray`	`softPlus(NDArray array)` Applies softPlus activation on the input `NDArray`.
`static NDList`	`softPlus(NDList arrays)` Applies softPlus activation on the input singleton `NDList`.
`static Block`	`softPlusBlock()` Creates a `LambdaBlock` that applies the `SoftReLU` activation function in its forward function.
`static NDArray`	`swish(NDArray array, float beta)` Applies Swish activation on the input `NDArray`.
`static NDList`	`swish(NDList arrays, float beta)` Applies SWish activation on the input singleton `NDList`.
`static Block`	`swishBlock(float beta)` Creates a `LambdaBlock` that applies the `Swish` activation function in its forward function.
`static NDArray`	`tanh(NDArray array)` Applies Tanh activation on the input `NDArray`.
`static NDList`	`tanh(NDList arrays)` Applies Tanh activation on the input singleton `NDList`.
`static Block`	`tanhBlock()` Creates a `LambdaBlock` that applies the `Tanh` activation function in its forward function.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Method Detail
  - relu
```
public static NDArray relu(NDArray array)
```
    Applies ReLU activation on the input NDArray.
    ReLU is defined by: $ y = max(0, x) $
    
    Parameters:
    
    array - the input NDArray
    
    Returns:
    
    the NDArray after applying ReLU activation
  - relu
```
public static NDList relu(NDList arrays)
```
    Applies ReLU activation on the input singleton NDList.
    ReLU is defined by: $ y = max(0, x) $
    
    Parameters:
    
    arrays - the input singleton NDList
    
    Returns:
    
    the singleton NDList after applying ReLU activation
  - sigmoid
```
public static NDArray sigmoid(NDArray array)
```
    Applies Sigmoid activation on the input NDArray.
    Sigmoid is defined by: $ y = 1 / (1 + e^{-x}) $
    
    Parameters:
    
    array - the input NDArray
    
    Returns:
    
    the NDArray after applying Sigmoid activation
  - sigmoid
```
public static NDList sigmoid(NDList arrays)
```
    Applies Sigmoid activation on the input singleton NDList.
    Sigmoid is defined by: $ y = 1 / (1 + e^{-x}) $
    
    Parameters:
    
    arrays - the input singleton NDList
    
    Returns:
    
    the singleton NDList after applying Sigmoid activation
  - tanh
```
public static NDArray tanh(NDArray array)
```
    Applies Tanh activation on the input NDArray.
    Tanh is defined by: $ y = (e^x - e^{-x}) / (e^x + e^{-x}) $
    
    Parameters:
    
    array - the input NDArray
    
    Returns:
    
    the NDArray after applying Tanh activation
  - tanh
```
public static NDList tanh(NDList arrays)
```
    Applies Tanh activation on the input singleton NDList.
    Tanh is defined by: $ y = (e^x - e^{-x}) / (e^x + e^{-x}) $
    
    Parameters:
    
    arrays - the input singleton NDList
    
    Returns:
    
    the singleton NDList after applying tanh activation
  - softPlus
```
public static NDArray softPlus(NDArray array)
```
    Applies softPlus activation on the input NDArray.
    softPlus is defined by: $ y = log(1 + e^x) $
    
    Parameters:
    
    array - the input NDArray
    
    Returns:
    
    the NDArray after applying soft ReLU activation
  - softPlus
```
public static NDList softPlus(NDList arrays)
```
    Applies softPlus activation on the input singleton NDList.
    softPlus is defined by: $ y = log(1 + e^x) $
    
    Parameters:
    
    arrays - the input singleton NDList
    
    Returns:
    
    the singleton NDList after applying soft ReLU activation
  - leakyRelu
```
public static NDArray leakyRelu(NDArray array,
                                float alpha)
```
    Applies Leaky ReLU activation on the input NDArray.
    Leaky ReLU is defined by: $ y = x \gt 0 ? x : alpha * x $
    
    Parameters:
    
    array - the input NDArray
    
    alpha - the slope for the activation
    
    Returns:
    
    the NDArray after applying Leaky ReLU activation
  - leakyRelu
```
public static NDList leakyRelu(NDList arrays,
                               float alpha)
```
    Applies Leaky ReLU activation on the input singleton NDList.
    Leaky ReLU is defined by: $ y = x \gt 0 ? x : alpha * x $
    
    Parameters:
    
    arrays - the input singleton NDList
    
    alpha - the slope for the activation
    
    Returns:
    
    the singleton NDList after applying Leaky ReLU activation
  - elu
```
public static NDArray elu(NDArray array,
                          float alpha)
```
    Applies ELU activation on the input NDArray.
    ELU is defined by: $ y = x \gt 0 ? x : alpha * (e^x - 1) $
    
    Parameters:
    
    array - the input NDArray
    
    alpha - the slope for the activation
    
    Returns:
    
    the NDArray after applying ELU activation
  - elu
```
public static NDList elu(NDList arrays,
                         float alpha)
```
    Applies ELU(Exponential Linear Unit) activation on the input singleton NDList.
    ELU is defined by: $ y = x \gt 0 ? x : alpha * (e^x - 1) $
    
    Parameters:
    
    arrays - the input singleton NDList
    
    alpha - the slope for the activation
    
    Returns:
    
    the singleton NDList after applying ELU activation
  - selu
```
public static NDArray selu(NDArray array)
```
    Applies Scaled ELU activation on the input NDArray.
    Scaled ELU is defined by: $ y = lambda * (x \gt 0 ? x : alpha * (e^x - 1))$ where $lambda = 1.0507009873554804934193349852946$ and $alpha = 1.6732632423543772848170429916717$
    
    Parameters:
    
    array - the input NDArray
    
    Returns:
    
    the NDArray after applying Scale ELU activation
  - selu
```
public static NDList selu(NDList arrays)
```
    Applies Scaled ELU activation on the input singleton NDList.
    Scaled ELU is defined by: $ y = lambda * (x \gt 0 ? x : alpha * (e^x - 1))$ where $lambda = 1.0507009873554804934193349852946$ and $alpha = 1.6732632423543772848170429916717 $
    
    Parameters:
    
    arrays - the input singleton NDList
    
    Returns:
    
    the singleton NDList after applying Scaled ELU activation
  - gelu
```
public static NDArray gelu(NDArray array)
```
    Applies GELU(Gaussian Error Linear Unit) activation on the input NDArray.
    
    Parameters:
    
    array - the input NDArray
    
    Returns:
    
    the NDArray after applying GELU activation
  - gelu
```
public static NDList gelu(NDList arrays)
```
    Applies GELU(Gaussian Error Linear Unit) activation on the input singleton NDList.
    
    Parameters:
    
    arrays - the input singleton NDList
    
    Returns:
    
    the singleton NDList after applying GELU activation
  - swish
```
public static NDArray swish(NDArray array,
                            float beta)
```
    Applies Swish activation on the input NDArray.
    Swish is defined as $y = x * sigmoid(beta * x)$
    
    Parameters:
    
    array - the input NDArray
    
    beta - a hyper-parameter
    
    Returns:
    
    the NDArray after applying Swish activation
  - swish
```
public static NDList swish(NDList arrays,
                           float beta)
```
    Applies SWish activation on the input singleton NDList.
    Swish is defined as $y = x * sigmoid(beta * x)$
    
    Parameters:
    
    arrays - the input singleton NDList
    
    beta - a hyper-parameter
    
    Returns:
    
    the singleton NDList after applying Swish activation
  - mish
```
public static NDArray mish(NDArray array)
```
    Applies Mish activation on the input NDArray.
    Mish is defined as $y = x * tanh(ln(1 + e^x)$ defined by Diganta Misra in his paper Mish: A Self Regularized Non-Monotonic Neural Activation Function
    
    Parameters:
    
    array - the input NDArray
    
    Returns:
    
    the NDArray after applying Mish activation
  - mish
```
public static NDList mish(NDList arrays)
```
    Applies Mish activation on the input singleton NDList.
    Mish is defined as $y = x * tanh(ln(1 + e^x)$ defined by Diganta Misra in his paper Mish: A Self Regularized Non-Monotonic Neural Activation Function
    
    Parameters:
    
    arrays - the input singleton NDList
    
    Returns:
    
    the singleton NDList after applying Mish activation
  - reluBlock
```
public static Block reluBlock()
```
    Creates a LambdaBlock that applies the ReLU activation function in its forward function.
    
    Returns:
    
    the LambdaBlock that applies the ReLU activation function
  - sigmoidBlock
```
public static Block sigmoidBlock()
```
    Creates a LambdaBlock that applies the Sigmoid activation function in its forward function.
    
    Returns:
    
    the LambdaBlock that applies the Sigmoid activation function
  - tanhBlock
```
public static Block tanhBlock()
```
    Creates a LambdaBlock that applies the Tanh activation function in its forward function.
    
    Returns:
    
    the LambdaBlock that applies the Tanh activation function
  - softPlusBlock
```
public static Block softPlusBlock()
```
    Creates a LambdaBlock that applies the SoftReLU activation function in its forward function.
    
    Returns:
    
    the LambdaBlock that applies the SoftReLU activation function
  - leakyReluBlock
```
public static Block leakyReluBlock(float alpha)
```
    Creates a LambdaBlock that applies the LeakyReLU activation function in its forward function.
    
    Parameters:
    
    alpha - the slope for the activation
    
    Returns:
    
    the LambdaBlock that applies the LeakyReLU activation function
  - eluBlock
```
public static Block eluBlock(float alpha)
```
    Creates a LambdaBlock that applies the ELU activation function in its forward function.
    
    Parameters:
    
    alpha - the slope for the activation
    
    Returns:
    
    the LambdaBlock that applies the ELU activation function
  - seluBlock
```
public static Block seluBlock()
```
    Creates a LambdaBlock that applies the SELU activation function in its forward function.
    
    Returns:
    
    the LambdaBlock that applies the SELU activation function
  - geluBlock
```
public static Block geluBlock()
```
    Creates a LambdaBlock that applies the GELU activation function in its forward function.
    
    Returns:
    
    the LambdaBlock that applies the GELU activation function
  - swishBlock
```
public static Block swishBlock(float beta)
```
    Creates a LambdaBlock that applies the Swish activation function in its forward function.
    
    Parameters:
    
    beta - a hyper-parameter
    
    Returns:
    
    the LambdaBlock that applies the Swish activation function
  - mishBlock
```
public static Block mishBlock()
```
    Creates a LambdaBlock that applies the Mish activation function in its forward function.
    
    Returns:
    
    the LambdaBlock that applies the Mish activation function
  - preluBlock
```
public static Block preluBlock()
```
    Returns a Prelu block.
    
    Returns:
    
    a Prelu block

Class Activation

Method Summary

Methods inherited from class java.lang.Object

Method Detail

relu

relu

sigmoid

sigmoid

tanh

tanh

softPlus

softPlus

leakyRelu

leakyRelu

elu

elu

selu

selu

gelu

gelu

swish

swish

mish

mish

reluBlock

sigmoidBlock

tanhBlock

softPlusBlock

leakyReluBlock

eluBlock

seluBlock

geluBlock

swishBlock

mishBlock

preluBlock