package bert
Ordering
- Alphabetic
Visibility
- Public
- Protected
Type Members
- case class BertEncoder(tokenEmbedding: Embedding, segmentEmbedding: Embedding, positionalEmbedding: Constant, blocks: Seq[TransformerEncoderBlock]) extends GenericModule[(Variable, Variable), Variable] with Product with Serializable
BertEncoder module
BertEncoder module
Input is
(tokens, segments)
wheretokens
andsegments
are both (batch,num tokens) long tensor.Output is (batch, num tokens, out dimension)
- case class BertLoss(pretrain: BertPretrainModule, mlmLoss: LossFunction, wholeSentenceLoss: LossFunction) extends GenericModule[BertLossInput, Variable] with Product with Serializable
- case class BertLossInput(input: BertPretrainInput, maskedLanguageModelTarget: STen, wholeSentenceTarget: STen) extends Product with Serializable
- case class BertPretrainInput(tokens: Variable, segments: Variable, positions: STen) extends Product with Serializable
- case class BertPretrainModule(encoder: BertEncoder, mlm: MaskedLanguageModelModule, wholeSentenceBinaryClassifier: MLP) extends GenericModule[BertPretrainInput, BertPretrainOutput] with Product with Serializable
- case class BertPretrainOutput(encoded: Variable, languageModelScores: Variable, wholeSentenceBinaryClassifierScore: Variable) extends Product with Serializable
- case class MaskedLanguageModelModule(mlp: MLP) extends GenericModule[(Variable, STen), Variable] with Product with Serializable
Masked Language Model Input of (embedding, positions) Embedding of size (batch, num tokens, embedding dim) Positions of size (batch, max num tokens) long tensor indicating which positions to make predictions on Output (batch, len(Positions), vocabulary size)
Value Members
- object BertEncoder extends Serializable
- object BertLoss extends Serializable
- object BertPretrainModule extends Serializable
- object MaskedLanguageModelModule extends Serializable