gate.util
Class TestApplication

java.lang.Object
  junit.framework.Assert
      junit.framework.TestCase
          gate.util.TestApplication

All Implemented Interfaces:

junit.framework.Test

public class TestApplication
extends junit.framework.TestCase

Test an application against its previous run on a corpus. If the results have changed more than fail.treshold parameter then compare both previous and current application with the gold standard.

Field Summary

protected Set<String>	annotationFeatures
protected Set<String>	annotationTypes
protected String	applicationAnnotationSet
protected File	applicationFile
protected File	cleanDocumentsDirectory
protected String	documentEncoding
protected double	failThreshold
protected String	goldAnnotationSet
protected File	goldStandardDirectory
protected File	previousRunDirectory
protected Properties	properties
protected File	propertiesFile
protected File	resultsFile

Constructor Summary

TestApplication()

Method Summary

protected void	setUp() The config file includes: - the application to be run - three directories containing - a copy of the clean documents - these are annotated with the current application - the annotated version becomes a build artifact (which can be used to update the reference set, in the case of good changes) - a set of documents annotated with the previous version - the gold standard - for each such directory configuration includes the annotation set and annotation types to be used
static junit.framework.Test	suite()
protected void	tearDown()
void	test() The logic is: - annotate the clean docs with the current version - perform anndif between current version and the previous version - if there are any differences: - perform anndif between current version and the GS - perform anndif between previous version and the GS - produce a report containing the evaluation numbers for the 3 diffs performed - produce a build artifact with the detailed changes (actual individual annotations that are different, in e.g.

Methods inherited from class junit.framework.TestCase

countTestCases, createResult, getName, run, run, runBare, runTest, setName, toString

Methods inherited from class junit.framework.Assert

assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertFalse, assertFalse, assertNotNull, assertNotNull, assertNotSame, assertNotSame, assertNull, assertNull, assertSame, assertSame, assertTrue, assertTrue, fail, fail, failNotEquals, failNotSame, failSame, format

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

properties

protected Properties properties

propertiesFile

protected File propertiesFile

documentEncoding

protected String documentEncoding

applicationFile

protected File applicationFile

cleanDocumentsDirectory

protected File cleanDocumentsDirectory

previousRunDirectory

protected File previousRunDirectory

goldStandardDirectory

protected File goldStandardDirectory

applicationAnnotationSet

protected String applicationAnnotationSet

goldAnnotationSet

protected String goldAnnotationSet

annotationTypes

protected Set<String> annotationTypes

annotationFeatures

protected Set<String> annotationFeatures

failThreshold

protected double failThreshold

resultsFile

protected File resultsFile

Constructor Detail

TestApplication

public TestApplication()

Method Detail

suite

public static junit.framework.Test suite()

setUp

protected void setUp()
              throws Exception

   The config file includes:
   - the application to be run
   - three directories containing
     - a copy of the clean documents
       - these are annotated with the current application
       - the annotated version becomes a build artifact (which can be used
         to update the reference set, in the case of good changes)
     - a set of documents annotated with the previous version
     - the gold standard
   - for each such directory configuration includes the annotation set
     and annotation types to be used
  

Overrides:

setUp in class junit.framework.TestCase

Throws:

Exception

tearDown

protected void tearDown()
                 throws Exception

Overrides:

tearDown in class junit.framework.TestCase

Throws:

Exception

test

public void test()

   The logic is:
    - annotate the clean docs with the current version
    - perform anndif between current version and the previous version
    - if there are any differences:
      - perform anndif between current version and the GS
      - perform anndif between previous version and the GS
      - produce a report containing the evaluation numbers
        for the 3 diffs performed
      - produce a build artifact with the detailed changes
        (actual individual annotations that are different, in e.g. HTML format,
         similar to what CBT currently produces).