gate.util
Class OntologyMeasures

java.lang.Object
  gate.util.OntologyMeasures

public class OntologyMeasures
extends Object

Modified version of Precision and Recall called BDM that takes into account the distance of two concepts in an ontology.

Field Summary

protected Map<String,Float>	bdmByConceptsMap
protected Map<String,Float>	bdmByTypeMap
protected URL	bdmFileUrl
protected Map<String,AnnotationDiffer>	differByTypeMap

Constructor Summary

OntologyMeasures()

OntologyMeasures(Collection<OntologyMeasures> measures)
Constructor to be used when you have a collection of OntologyMeasures and want to consider it as only one OntologyMeasures.

Method Summary

void	calculateBdm(Collection<AnnotationDiffer> differs) For a document get the annotation differs that contain the type to compare and the annotation differs that may have miscategorized annotations for this type.
Map<String,AnnotationDiffer>	getDifferByTypeMap() Be careful, don't modify it.
double	getFMeasureAverageBdm(double beta) Gets the average of strict and lenient F-Measure values.
double	getFMeasureAverageBdm(String type, double beta)
double	getFMeasureLenientBdm(double beta)
double	getFMeasureLenientBdm(String type, double beta)
double	getFMeasureStrictBdm(double beta)
double	getFMeasureStrictBdm(String type, double beta)
List<String>	getMeasuresRow(Object[] measures, String title)
double	getPrecisionAverageBdm() Gets the average of the strict and lenient precision values.
double	getPrecisionAverageBdm(String type)
double	getPrecisionLenientBdm()
double	getPrecisionLenientBdm(String type)
double	getPrecisionStrictBdm()
double	getPrecisionStrictBdm(String type) AP = (sum of BDMs for BDM-matching pair spurious/missing + Correct) / (Correct + Spurious)
double	getRecallAverageBdm() Gets the average of the strict and lenient recall values.
double	getRecallAverageBdm(String type)
double	getRecallLenientBdm()
double	getRecallLenientBdm(String type)
double	getRecallStrictBdm()
double	getRecallStrictBdm(String type)
Map<String,Float>	read(URL bdmFile) Read the BDM scores from a file.
void	setBdmFile(URL url)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

bdmByTypeMap

protected Map<String,Float> bdmByTypeMap

bdmFileUrl

protected URL bdmFileUrl

differByTypeMap

protected Map<String,AnnotationDiffer> differByTypeMap

bdmByConceptsMap

protected Map<String,Float> bdmByConceptsMap

Constructor Detail

OntologyMeasures

public OntologyMeasures()

OntologyMeasures

public OntologyMeasures(Collection<OntologyMeasures> measures)

Constructor to be used when you have a collection of OntologyMeasures and want to consider it as only one OntologyMeasures. Then you can only use the methods getPrecision/Recall/FMeasure...().

Parameters:

measures - collection to be regrouped in one OntologyMeasures

Method Detail

calculateBdm

public void calculateBdm(Collection<AnnotationDiffer> differs)

For a document get the annotation differs that contain the type to compare and the annotation differs that may have miscategorized annotations for this type. Then we try to find miscategorized types that are close enough from the main type and use their BDM value to get an augmented precision, recall and fscore.

Parameters:

differs - annotation differ for the type and for possible miscategorized types.

getPrecisionStrictBdm

public double getPrecisionStrictBdm(String type)

AP = (sum of BDMs for BDM-matching pair spurious/missing + Correct) / (Correct + Spurious)

Parameters:

type - annotation type

Returns:

strict precision with BDM correction

getPrecisionStrictBdm

public double getPrecisionStrictBdm()

getRecallStrictBdm

public double getRecallStrictBdm(String type)

getRecallStrictBdm

public double getRecallStrictBdm()

getFMeasureStrictBdm

public double getFMeasureStrictBdm(String type,
                                   double beta)

getFMeasureStrictBdm

public double getFMeasureStrictBdm(double beta)

getPrecisionLenientBdm

public double getPrecisionLenientBdm(String type)

getPrecisionLenientBdm

public double getPrecisionLenientBdm()

getRecallLenientBdm

public double getRecallLenientBdm(String type)

getRecallLenientBdm

public double getRecallLenientBdm()

getFMeasureLenientBdm

public double getFMeasureLenientBdm(String type,
                                    double beta)

getFMeasureLenientBdm

public double getFMeasureLenientBdm(double beta)

getPrecisionAverageBdm

public double getPrecisionAverageBdm(String type)

getPrecisionAverageBdm

public double getPrecisionAverageBdm()

Gets the average of the strict and lenient precision values.

Returns:

a double value.

getRecallAverageBdm

public double getRecallAverageBdm(String type)

getRecallAverageBdm

public double getRecallAverageBdm()

Gets the average of the strict and lenient recall values.

Returns:

a double value.

getFMeasureAverageBdm

public double getFMeasureAverageBdm(String type,
                                    double beta)

getFMeasureAverageBdm

public double getFMeasureAverageBdm(double beta)

Gets the average of strict and lenient F-Measure values.

Parameters:

beta - The relative weight of precision and recall. A value of 1 gives equal weights to precision and recall. A value of 0 takes the recall value completely out of the equation.

Returns:

a doublevalue.

setBdmFile

public void setBdmFile(URL url)

read

public Map<String,Float> read(URL bdmFile)

Read the BDM scores from a file.

Parameters:

bdmFile - URL of the BDM file

Returns:

map from a pair of concepts to their BDM score

getMeasuresRow

public List<String> getMeasuresRow(Object[] measures,
                                   String title)

getDifferByTypeMap

public Map<String,AnnotationDiffer> getDifferByTypeMap()

Be careful, don't modify it. That's not a copy because it would take too much memory.

Returns:

differ by type map