com.johnsnowlabs.nlp.annotators.ner.crf
NerCrfApproach
Related Docs:
object NerCrfApproach
| package crf
com.johnsnowlabs.nlp.annotators.ner.crf
c0 params defining decay speed for gradient
CRF based Named Entity Recognition Tagger
CRF based Named Entity Recognition Tagger
Entities to recognize
Entities to recognize
Additional dictionaries to use as a features
c0 params defining decay speed for gradient
Whether or not to calculate prediction confidence by token, includes in metadata
input annotations columns currently used
L2 regularization coefficient
If Epoch relative improvement less than eps then training is stopped
Maximum number of epochs to train
Maximum number of epochs to train
Minimum number of epochs to train
Minimum number of epochs to train
Features with less weights then this param value will be filtered
Gets annotation column name going to generate
Gets annotation column name going to generate
Random seed
Random seed
Level of verbosity during training
Level of verbosity during training
includeConfidence", "whether or not to calculate prediction confidence by token, includes in metadata
Input annotator types : DOCUMENT, TOKEN, POS, WORD_EMBEDDINGS
Input annotator types : DOCUMENT, TOKEN, POS, WORD_EMBEDDINGS
columns that contain annotations necessary to run this annotator AnnotatorType is used both as input and output columns if not specified
columns that contain annotations necessary to run this annotator AnnotatorType is used both as input and output columns if not specified
L2 regularization coefficient
Column with label per each token
Column with label per each token
If Epoch relative improvement less than eps then training is stopped
Maximum number of epochs to train
Maximum number of epochs to train
Minimum number of epochs to train
Minimum number of epochs to train
Features with less weights then this param value will be filtered
Input annotator types : NAMED_ENTITY
Input annotator types : NAMED_ENTITY
Random seed
Random seed
c0 params defining decay speed for gradient
Entities to recognize
Entities to recognize
Additional dictionaries to use as a features
Additional dictionaries to use as a features
Whether or not to calculate prediction confidence by token, includes in metadata
Overrides required annotators column if different than default
Overrides required annotators column if different than default
L2 regularization coefficient
Column with label per each token
Column with label per each token
If Epoch relative improvement less than eps then training is stopped
Maximum number of epochs to train
Maximum number of epochs to train
Minimum number of epochs to train
Minimum number of epochs to train
Features with less weights then this param value will be filtered
Overrides annotation column name when transforming
Overrides annotation column name when transforming
Random seed
Random seed
Level of verbosity during training
Level of verbosity during training
Level of verbosity during training
Level of verbosity during training
requirement for pipeline transformation validation.
requirement for pipeline transformation validation. It is called on fit()
takes a Dataset and checks to see if all the required annotation types are present.
takes a Dataset and checks to see if all the required annotation types are present.
to be validated
True if all the required types are present, else false
Level of verbosity during training
Level of verbosity during training
Required input and expected output annotator types
Algorithm for training Named Entity Recognition Model
This Named Entity recognition annotator allows for a generic model to be trained by utilizing a CRF machine learning algorithm. Its train data (train_ner) is either a labeled or an external CoNLL 2003 IOB based spark dataset with Annotations columns. Also the user has to provide word embeddings annotation column. Optionally the user can provide an entity dictionary file for better accuracy
See https://github.com/JohnSnowLabs/spark-nlp/tree/master/src/test/scala/com/johnsnowlabs/nlp/annotators/ner/crf for further reference on this API.