org.nd4j.linalg.api.ops.impl.scalar

Class LeakyReLU

java.lang.Object

org.nd4j.autodiff.functions.DifferentialFunction

org.nd4j.linalg.api.ops.BaseOp

org.nd4j.linalg.api.ops.BaseScalarOp

org.nd4j.linalg.api.ops.impl.scalar.LeakyReLU

All Implemented Interfaces:

Op, ScalarOp

public class LeakyReLU
extends BaseScalarOp

Leaky Rectified linear unit. Default alpha=0.01, cutoff=0
Out(x) = alpha*x if x<0
Out(x) = x if x >= 0
Leaky ReLU may avoid zero gradient "dying ReLU" problem by having non-zero gradient below 0.
See for example https://arxiv.org/abs/1505.00853 for a comparison of ReLU variants.

Author:

Alex Black

Nested Class Summary

Nested classes/interfaces inherited from interface org.nd4j.linalg.api.ops.Op

Op.Type

Field Summary

Fields

Modifier and Type	Field and Description
static double	DEFAULT_ALPHA

Fields inherited from class org.nd4j.linalg.api.ops.BaseOp

dimensionz, extraArgz, x, xVertexId, y, yVertexId, z, zVertexId

Fields inherited from class org.nd4j.autodiff.functions.DifferentialFunction

dimensions, extraArgs, inPlace, sameDiff, scalarValue

Constructor Summary

Constructors

Constructor and Description
LeakyReLU()
LeakyReLU(INDArray x)
LeakyReLU(INDArray x, double alpha)
LeakyReLU(INDArray x, INDArray z)
LeakyReLU(INDArray x, INDArray z, double alpha)
LeakyReLU(SameDiff sameDiff, SDVariable i_v, boolean inPlace, double alpha)
LeakyReLU(SameDiff sameDiff, SDVariable i_v, double alpha)
LeakyReLU(SameDiff sameDiff, SDVariable i_v, Object[] extraArgs, double alpha)

Method Summary

Modifier and Type	Method and Description
List<SDVariable>	doDiff(List<SDVariable> i_v) The actual implementation for automatic differentiation.
void	initFromTensorFlow(NodeDef nodeDef, SameDiff initWith, Map<String,AttrValue> attributesForNode, GraphDef graph) Initialize the function from the given NodeDef
String	onnxName() The opName of this function in onnx
String	opName() The name of the op
int	opNum() The number of the op (mainly for old legacy XYZ ops like Op)
String	tensorflowName() The opName of this function tensorflow

Methods inherited from class org.nd4j.linalg.api.ops.BaseScalarOp

calculateOutputDataTypes, calculateOutputShape, calculateOutputShape, getDimension, getOpType, opType, scalar, setDimension, setScalar, setScalar, validateDataTypes, z

Methods inherited from class org.nd4j.linalg.api.ops.BaseOp

clearArrays, defineDimensions, dimensions, equals, extraArgs, extraArgsBuff, extraArgsDataBuff, getFinalResult, getInputArgument, getNumOutputs, getOpType, hashCode, initFromOnnx, outputVariables, setX, setY, setZ, toCustomOp, toString, x, y

Methods inherited from class org.nd4j.autodiff.functions.DifferentialFunction

arg, arg, argNames, args, attributeAdaptersForFunction, configFieldName, diff, dup, getValue, isConfigProperties, larg, mappingsForFunction, onnxNames, outputs, outputVariable, outputVariables, outputVariablesNames, propertiesForFunction, rarg, replaceArg, setInstanceId, setPropertiesForFunction, setValueFor, tensorflowNames

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.nd4j.linalg.api.ops.ScalarOp

dimensions

Methods inherited from interface org.nd4j.linalg.api.ops.Op

clearArrays, extraArgs, extraArgsBuff, extraArgsDataBuff, setExtraArgs, setX, setY, setZ, toCustomOp, x, y

Field Detail

DEFAULT_ALPHA

public static final double DEFAULT_ALPHA

See Also:

Constant Field Values

Constructor Detail

LeakyReLU

public LeakyReLU(SameDiff sameDiff,
                 SDVariable i_v,
                 boolean inPlace,
                 double alpha)

LeakyReLU

public LeakyReLU(SameDiff sameDiff,
                 SDVariable i_v,
                 double alpha)

LeakyReLU

public LeakyReLU(SameDiff sameDiff,
                 SDVariable i_v,
                 Object[] extraArgs,
                 double alpha)

LeakyReLU

public LeakyReLU()

LeakyReLU

public LeakyReLU(INDArray x,
                 double alpha)

LeakyReLU

public LeakyReLU(INDArray x,
                 INDArray z,
                 double alpha)

LeakyReLU

public LeakyReLU(INDArray x,
                 INDArray z)

LeakyReLU

public LeakyReLU(INDArray x)

Method Detail

opNum

public int opNum()

Description copied from class: DifferentialFunction

The number of the op (mainly for old legacy XYZ ops like Op)

Specified by:

opNum in interface Op

Overrides:

opNum in class DifferentialFunction

Returns:

opName

public String opName()

Description copied from class: DifferentialFunction

The name of the op

Specified by:

opName in interface Op

Overrides:

opName in class DifferentialFunction

Returns:

the opName of this operation

onnxName

public String onnxName()

Description copied from class: DifferentialFunction

The opName of this function in onnx

Overrides:

onnxName in class BaseOp

Returns:

tensorflowName

public String tensorflowName()

Description copied from class: DifferentialFunction

The opName of this function tensorflow

Overrides:

tensorflowName in class BaseOp

Returns:

doDiff

public List<SDVariable> doDiff(List<SDVariable> i_v)

Description copied from class: DifferentialFunction

The actual implementation for automatic differentiation.

Specified by:

doDiff in class DifferentialFunction

Returns:

initFromTensorFlow

public void initFromTensorFlow(NodeDef nodeDef,
                               SameDiff initWith,
                               Map<String,AttrValue> attributesForNode,
                               GraphDef graph)

Description copied from class: DifferentialFunction

Initialize the function from the given NodeDef

Overrides:

initFromTensorFlow in class BaseOp