ner

Type Members

1. case class NamedEntity(start: Int, end: Int, entity: String, text: String, sentenceId: String, confidence: Option[Float]) extends Product with Serializable

   

2. trait NerApproach[T <: NerApproach[_]] extends Params

   

3. class NerConverter extends AnnotatorModel[NerConverter] with HasSimpleAnnotate[NerConverter]

   

   Converts a IOB or IOB2 representation of NER to a user-friendly one, by associating the tokens of recognized entities and their label.

   Converts a IOB or IOB2 representation of NER to a user-friendly one, by associating the tokens of recognized entities and their label. Results in CHUNK Annotation type.

   NER chunks can then be filtered by setting a whitelist with setWhiteList. Chunks with no associated entity (tagged "O") are filtered.

   See also Inside–outside–beginning (tagging) for more information.

   Example

   This is a continuation of the example of the NerDLModel. See that class on how to extract the entities.

   The output of the NerDLModel follows the Annotator schema and can be converted like so:

   result.selectExpr("explode(ner)").show(false)
   +----------------------------------------------------+
   |col                                                 |
   +----------------------------------------------------+
   |[named_entity, 0, 2, B-ORG, [word -> U.N], []]      |
   |[named_entity, 3, 3, O, [word -> .], []]            |
   |[named_entity, 5, 12, O, [word -> official], []]    |
   |[named_entity, 14, 18, B-PER, [word -> Ekeus], []]  |
   |[named_entity, 20, 24, O, [word -> heads], []]      |
   |[named_entity, 26, 28, O, [word -> for], []]        |
   |[named_entity, 30, 36, B-LOC, [word -> Baghdad], []]|
   |[named_entity, 37, 37, O, [word -> .], []]          |
   +----------------------------------------------------+

   After the converter is used:

   val converter = new NerConverter()
     .setInputCols("sentence", "token", "ner")
     .setOutputCol("entities")
     .setPreservePosition(false)
   
   converter.transform(result).selectExpr("explode(entities)").show(false)
   +------------------------------------------------------------------------+
   |col                                                                     |
   +------------------------------------------------------------------------+
   |[chunk, 0, 2, U.N, [entity -> ORG, sentence -> 0, chunk -> 0], []]      |
   |[chunk, 14, 18, Ekeus, [entity -> PER, sentence -> 0, chunk -> 1], []]  |
   |[chunk, 30, 36, Baghdad, [entity -> LOC, sentence -> 0, chunk -> 2], []]|
   +------------------------------------------------------------------------+

4. class NerOverwriter extends AnnotatorModel[NerOverwriter] with HasSimpleAnnotate[NerOverwriter]