Class/Object

com.intel.analytics.zoo.pipeline.api.keras.layers

LayerNorm

Related Docs: object LayerNorm | package layers

Permalink

class LayerNorm[T] extends KerasLayer[Tensor[T], Tensor[T], T] with Net

Normalization layer used in Bert. u = AutoGrad.mean(x, sizes.size - 1, true) t = x - u s = AutoGrad.mean(AutoGrad.square(x - u), sizes.size -1, true) y = (x - u) / AutoGrad.sqrt(s + e) y * weight + bias

T

Numeric type. Only support float/double now

Linear Supertypes
Net, KerasLayer[Tensor[T], Tensor[T], T], Container[Tensor[T], Tensor[T], T], AbstractModule[Tensor[T], Tensor[T], T], InferShape, Serializable, Serializable, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. LayerNorm
  2. Net
  3. KerasLayer
  4. Container
  5. AbstractModule
  6. InferShape
  7. Serializable
  8. Serializable
  9. AnyRef
  10. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new LayerNorm(nOutput: Int = 768, eps: Double = 1e-5, inputShape: Shape = null)(implicit arg0: ClassTag[T], ev: TensorNumeric[T])

    Permalink

    nOutput

    The size of output dimension.

    eps

    Optional. Small value to avoid divide zero.

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  4. def accGradParameters(input: Tensor[T], gradOutput: Tensor[T]): Unit

    Permalink
    Definition Classes
    KerasLayer → AbstractModule
  5. def apply(name: String): Option[AbstractModule[Activity, Activity, T]]

    Permalink
    Definition Classes
    Container → AbstractModule
  6. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any
  7. def backward(input: Tensor[T], gradOutput: Tensor[T]): Tensor[T]

    Permalink
    Definition Classes
    AbstractModule
  8. var backwardTime: Long

    Permalink
    Attributes
    protected
    Definition Classes
    AbstractModule
  9. def build(calcInputShape: Shape): Shape

    Permalink
    Definition Classes
    KerasLayer → InferShape
  10. def canEqual(other: Any): Boolean

    Permalink
    Definition Classes
    Container → AbstractModule
  11. final def checkEngineType(): LayerNorm.this.type

    Permalink
    Definition Classes
    Container → AbstractModule
  12. def clearState(): LayerNorm.this.type

    Permalink
    Definition Classes
    Container → AbstractModule
  13. final def clone(deepCopy: Boolean): AbstractModule[Tensor[T], Tensor[T], T]

    Permalink
    Definition Classes
    AbstractModule
  14. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  15. final def cloneModule(): LayerNorm.this.type

    Permalink
    Definition Classes
    AbstractModule
  16. def computeOutputShape(inputShape: Shape): Shape

    Permalink
    Definition Classes
    LayerNorm → KerasLayer → InferShape
  17. def doBuild(inputShape: Shape): AbstractModule[Tensor[T], Tensor[T], T]

    Permalink
    Definition Classes
    LayerNorm → KerasLayer
  18. val eps: Double

    Permalink

    Optional.

    Optional. Small value to avoid divide zero.

  19. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  20. def equals(other: Any): Boolean

    Permalink
    Definition Classes
    Container → AbstractModule → AnyRef → Any
  21. final def evaluate(): LayerNorm.this.type

    Permalink
    Definition Classes
    Container → AbstractModule
  22. final def evaluate(dataSet: LocalDataSet[MiniBatch[T]], vMethods: Array[_ <: ValidationMethod[T]]): Array[(ValidationResult, ValidationMethod[T])]

    Permalink
    Definition Classes
    AbstractModule
  23. final def evaluate(dataset: RDD[MiniBatch[T]], vMethods: Array[_ <: ValidationMethod[T]]): Array[(ValidationResult, ValidationMethod[T])]

    Permalink
    Definition Classes
    AbstractModule
  24. final def evaluate(dataset: RDD[Sample[T]], vMethods: Array[_ <: ValidationMethod[T]], batchSize: Option[Int]): Array[(ValidationResult, ValidationMethod[T])]

    Permalink
    Definition Classes
    AbstractModule
  25. final def evaluateImage(imageFrame: ImageFrame, vMethods: Array[_ <: ValidationMethod[T]], batchSize: Option[Int]): Array[(ValidationResult, ValidationMethod[T])]

    Permalink
    Definition Classes
    AbstractModule
  26. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  27. def findModules(moduleType: String): ArrayBuffer[AbstractModule[_, _, T]]

    Permalink
    Definition Classes
    Container
  28. final def forward(input: Tensor[T]): Tensor[T]

    Permalink
    Definition Classes
    AbstractModule
  29. var forwardTime: Long

    Permalink
    Attributes
    protected
    Definition Classes
    AbstractModule
  30. def freeze(names: String*): LayerNorm.this.type

    Permalink
    Definition Classes
    Container → AbstractModule
  31. def from[T](vars: Variable[T]*)(implicit arg0: ClassTag[T], ev: TensorNumeric[T]): Variable[T]

    Permalink

    Build graph: some other modules point to current module

    Build graph: some other modules point to current module

    vars

    upstream variables

    returns

    Variable containing current module

    Definition Classes
    Net
  32. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any
  33. def getExtraParameter(): Array[Tensor[T]]

    Permalink
    Definition Classes
    Container → AbstractModule
  34. final def getInputShape(): Shape

    Permalink
    Definition Classes
    InferShape
  35. final def getName(): String

    Permalink
    Definition Classes
    AbstractModule
  36. final def getNumericType(): TensorDataType

    Permalink
    Definition Classes
    AbstractModule
  37. final def getOutputShape(): Shape

    Permalink
    Definition Classes
    InferShape
  38. def getParametersTable(): Table

    Permalink
    Definition Classes
    Container → AbstractModule
  39. final def getPrintName(): String

    Permalink
    Attributes
    protected
    Definition Classes
    AbstractModule
  40. final def getScaleB(): Double

    Permalink
    Definition Classes
    AbstractModule
  41. final def getScaleW(): Double

    Permalink
    Definition Classes
    AbstractModule
  42. def getTimes(): Array[(AbstractModule[_ <: Activity, _ <: Activity, T], Long, Long)]

    Permalink
    Definition Classes
    Container → AbstractModule
  43. final def getTimesGroupByModuleType(): Array[(String, Long, Long)]

    Permalink
    Definition Classes
    AbstractModule
  44. final def getWeightsBias(): Array[Tensor[T]]

    Permalink
    Definition Classes
    AbstractModule
  45. var gradInput: Tensor[T]

    Permalink
    Definition Classes
    AbstractModule
  46. final def hasName: Boolean

    Permalink
    Definition Classes
    AbstractModule
  47. def hashCode(): Int

    Permalink
    Definition Classes
    Container → AbstractModule → AnyRef → Any
  48. val inputShape: Shape

    Permalink
  49. def inputs(first: (ModuleNode[T], Int), nodesWithIndex: (ModuleNode[T], Int)*): ModuleNode[T]

    Permalink
    Definition Classes
    KerasLayer → AbstractModule
  50. def inputs(nodes: Array[ModuleNode[T]]): ModuleNode[T]

    Permalink
    Definition Classes
    KerasLayer → AbstractModule
  51. def inputs(nodes: ModuleNode[T]*): ModuleNode[T]

    Permalink
    Definition Classes
    KerasLayer → AbstractModule
  52. def isBuilt(): Boolean

    Permalink
    Definition Classes
    KerasLayer → InferShape
  53. def isFrozen[T]()(implicit arg0: ClassTag[T]): Boolean

    Permalink
    Definition Classes
    Net
  54. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any
  55. def isKerasStyle(): Boolean

    Permalink
    Definition Classes
    KerasLayer → InferShape
  56. final def isTraining(): Boolean

    Permalink
    Definition Classes
    AbstractModule
  57. def labor: AbstractModule[Tensor[T], Tensor[T], T]

    Permalink
    Definition Classes
    KerasLayer
  58. def labor_=(value: AbstractModule[Tensor[T], Tensor[T], T]): Unit

    Permalink
    Definition Classes
    KerasLayer
  59. var line: String

    Permalink
    Attributes
    protected
    Definition Classes
    AbstractModule
  60. final def loadModelWeights(srcModel: Module[Float], matchAll: Boolean): LayerNorm.this.type

    Permalink
    Definition Classes
    AbstractModule
  61. final def loadWeights(weightPath: String, matchAll: Boolean): LayerNorm.this.type

    Permalink
    Definition Classes
    AbstractModule
  62. val modules: ArrayBuffer[AbstractModule[Activity, Activity, T]]

    Permalink
    Definition Classes
    Container
  63. val nOutput: Int

    Permalink

    The size of output dimension.

  64. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  65. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef
  66. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef
  67. var output: Tensor[T]

    Permalink
    Definition Classes
    AbstractModule
  68. def parameters(): (Array[Tensor[T]], Array[Tensor[T]])

    Permalink
    Definition Classes
    Container → AbstractModule
  69. final def predict(dataset: RDD[Sample[T]], batchSize: Int, shareBuffer: Boolean): RDD[Activity]

    Permalink
    Definition Classes
    AbstractModule
  70. final def predictClass(dataset: RDD[Sample[T]], batchSize: Int): RDD[Int]

    Permalink
    Definition Classes
    AbstractModule
  71. final def predictImage(imageFrame: ImageFrame, outputLayer: String, shareBuffer: Boolean, batchPerPartition: Int, predictKey: String, featurePaddingParam: Option[PaddingParam[T]]): ImageFrame

    Permalink
    Definition Classes
    AbstractModule
  72. def processInputs(first: (ModuleNode[T], Int), nodesWithIndex: (ModuleNode[T], Int)*): ModuleNode[T]

    Permalink
    Attributes
    protected
    Definition Classes
    AbstractModule
  73. def processInputs(nodes: Seq[ModuleNode[T]]): ModuleNode[T]

    Permalink
    Attributes
    protected
    Definition Classes
    AbstractModule
  74. final def quantize(): Module[T]

    Permalink
    Definition Classes
    AbstractModule
  75. def release(): Unit

    Permalink
    Definition Classes
    Container → AbstractModule
  76. def reset(): Unit

    Permalink
    Definition Classes
    Container → AbstractModule
  77. def resetTimes(): Unit

    Permalink
    Definition Classes
    Container → AbstractModule
  78. final def saveCaffe(prototxtPath: String, modelPath: String, useV2: Boolean, overwrite: Boolean): LayerNorm.this.type

    Permalink
    Definition Classes
    AbstractModule
  79. final def saveDefinition(path: String, overWrite: Boolean): LayerNorm.this.type

    Permalink
    Definition Classes
    AbstractModule
  80. final def saveModule(path: String, weightPath: String, overWrite: Boolean): LayerNorm.this.type

    Permalink
    Definition Classes
    AbstractModule
  81. final def saveTF(inputs: Seq[(String, Seq[Int])], path: String, byteOrder: ByteOrder, dataFormat: TensorflowDataFormat): LayerNorm.this.type

    Permalink
    Definition Classes
    AbstractModule
  82. final def saveTorch(path: String, overWrite: Boolean): LayerNorm.this.type

    Permalink
    Definition Classes
    AbstractModule
  83. final def saveWeights(path: String, overWrite: Boolean): Unit

    Permalink
    Definition Classes
    AbstractModule
  84. var scaleB: Double

    Permalink
    Attributes
    protected
    Definition Classes
    AbstractModule
  85. var scaleW: Double

    Permalink
    Attributes
    protected
    Definition Classes
    AbstractModule
  86. final def setExtraParameter(extraParam: Array[Tensor[T]]): LayerNorm.this.type

    Permalink
    Definition Classes
    AbstractModule
  87. final def setLine(line: String): LayerNorm.this.type

    Permalink
    Definition Classes
    AbstractModule
  88. final def setName(name: String): LayerNorm.this.type

    Permalink
    Definition Classes
    AbstractModule
  89. def setScaleB(b: Double): LayerNorm.this.type

    Permalink
    Definition Classes
    Container → AbstractModule
  90. def setScaleW(w: Double): LayerNorm.this.type

    Permalink
    Definition Classes
    Container → AbstractModule
  91. final def setWeightsBias(newWeights: Array[Tensor[T]]): LayerNorm.this.type

    Permalink
    Definition Classes
    AbstractModule
  92. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef
  93. def toGraph(startNodes: ModuleNode[T]*): Graph[T]

    Permalink
    Definition Classes
    AbstractModule
  94. def toString(): String

    Permalink
    Definition Classes
    AbstractModule → AnyRef → Any
  95. var train: Boolean

    Permalink
    Attributes
    protected
    Definition Classes
    AbstractModule
  96. final def training(): LayerNorm.this.type

    Permalink
    Definition Classes
    Container → AbstractModule
  97. def unFreeze(names: String*): LayerNorm.this.type

    Permalink
    Definition Classes
    Container → AbstractModule
  98. def updateGradInput(input: Tensor[T], gradOutput: Tensor[T]): Tensor[T]

    Permalink
    Definition Classes
    KerasLayer → AbstractModule
  99. def updateOutput(input: Tensor[T]): Tensor[T]

    Permalink
    Definition Classes
    KerasLayer → AbstractModule
  100. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  101. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  102. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  103. def zeroGradParameters(): Unit

    Permalink
    Definition Classes
    AbstractModule

Deprecated Value Members

  1. final def save(path: String, overWrite: Boolean): LayerNorm.this.type

    Permalink
    Definition Classes
    AbstractModule
    Annotations
    @deprecated
    Deprecated

    (Since version 0.3.0) please use recommended saveModule(path, overWrite)

Inherited from Net

Inherited from KerasLayer[Tensor[T], Tensor[T], T]

Inherited from Container[Tensor[T], Tensor[T], T]

Inherited from AbstractModule[Tensor[T], Tensor[T], T]

Inherited from InferShape

Inherited from Serializable

Inherited from Serializable

Inherited from AnyRef

Inherited from Any

Ungrouped