Package uk.gov.dstl.baleen.transports.components

Class AbstractTransportConsumer

  • All Implemented Interfaces:
    org.apache.uima.analysis_component.AnalysisComponent
    Direct Known Subclasses:
    MemoryTransportSender
    public abstract class AbstractTransportConsumer
extends uk.gov.dstl.baleen.uima.BaleenConsumer
    This class provides basic functionality for a transport consumer. Such a consumer is responsible for publishing the serialised JCas to a transport system, such as a message queue. This class provides the common serialization, implementations should manage the transport by implementing the abstract createQueue() closeQueue() and writeToQueue(String, String).

    It also supports providing an optional whitelist or blacklist of types to filter the transported types.

    A simple throttling mechanism is provided using the queue capacity and the getQueueLength method, implemented with exponential backoff. This process is not threadsafe, so if your queue has a hard maximum capacity it must be check in you implementation.

    Configuration Parameters:
    ParameterDescriptionDefault Value(s)
    topicThe topic for transport use{@link AbstractTransportCollectionReader#PARAM_TOPIC_DEFAULT}

    • Field Summary

      Fields 
      Modifier and Type Field Description
      static java.lang.String PARAM_BLACKLIST
      A list of types to blacklist (optionally including package)
      static java.lang.String PARAM_TOPIC
      The topic for transport use
      static java.lang.String PARAM_WHITELIST
      A list of types to whitelist (optionally including package)
      static java.lang.String QUEUE_CAPACITY
      The optional capacity of the blocking queue.
      protected java.lang.String topic  

      • Fields inherited from class uk.gov.dstl.baleen.uima.BaleenAnnotator

        KEY_HISTORY

    • Method Summary

      All Methods Instance Methods Abstract Methods Concrete Methods 
      Modifier and Type Method Description
      protected abstract void closeQueue()
      Close the queue and release and resources
      protected abstract void createQueue()
      Create the Queue for the transport of the data
      protected void doDestroy()  
      void doInitialize​(org.apache.uima.UimaContext aContext)  
      protected void doProcess​(org.apache.uima.jcas.JCas jCas)  
      protected int getDefaultCapacity()  
      protected abstract int getQueueLength()
      Get the current length of the queue.
      protected abstract void writeToQueue​(java.lang.String id, java.lang.String data)
      Write the given data to the queue

      • Methods inherited from class uk.gov.dstl.baleen.uima.BaleenConsumer

        getAction

      • Methods inherited from class uk.gov.dstl.baleen.uima.BaleenAnnotator

        addToJCasIndex, addToJCasIndex, createMonitor, createSupport, destroy, getDocumentAnnotation, getMonitor, getSupport, initialize, mergeWithExisting, mergeWithExisting, mergeWithNew, mergeWithNew, process, removeFromJCasIndex, removeFromJCasIndex

      • Methods inherited from class org.apache.uima.analysis_component.JCasAnnotator_ImplBase

        getRequiredCasInterface, process

      • Methods inherited from class org.apache.uima.analysis_component.Annotator_ImplBase

        getCasInstancesRequired, hasNext, next

      • Methods inherited from class org.apache.uima.analysis_component.AnalysisComponent_ImplBase

        batchProcessComplete, collectionProcessComplete, getContext, getLogger, getResultSpecification, reconfigure, setResultSpecification

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Field Detail

      • PARAM_TOPIC

        public static final java.lang.String PARAM_TOPIC
        The topic for transport use
        See Also:
        Constant Field Values

      • topic

        protected java.lang.String topic

      • PARAM_BLACKLIST

        public static final java.lang.String PARAM_BLACKLIST
        A list of types to blacklist (optionally including package)
        See Also:
        Constant Field Values

      • PARAM_WHITELIST

        public static final java.lang.String PARAM_WHITELIST
        A list of types to whitelist (optionally including package)
        See Also:
        Constant Field Values

      • QUEUE_CAPACITY

        public static final java.lang.String QUEUE_CAPACITY
        The optional capacity of the blocking queue. This can be set larger if working with small documents.
        See Also:
        Constant Field Values

    • Constructor Detail

      • AbstractTransportConsumer

        public AbstractTransportConsumer()

    • Method Detail

      • doInitialize

        public void doInitialize​(org.apache.uima.UimaContext aContext)
                  throws org.apache.uima.resource.ResourceInitializationException
        Overrides:
        doInitialize in class uk.gov.dstl.baleen.uima.BaleenAnnotator
        Throws:
        org.apache.uima.resource.ResourceInitializationException

      • doDestroy

        protected void doDestroy()
        Overrides:
        doDestroy in class uk.gov.dstl.baleen.uima.BaleenAnnotator

      • getDefaultCapacity

        protected int getDefaultCapacity()
        Returns:
        the default capacity for the queue

      • doProcess

        protected void doProcess​(org.apache.uima.jcas.JCas jCas)
                  throws org.apache.uima.analysis_engine.AnalysisEngineProcessException
        Specified by:
        doProcess in class uk.gov.dstl.baleen.uima.BaleenAnnotator
        Throws:
        org.apache.uima.analysis_engine.AnalysisEngineProcessException

      • createQueue

        protected abstract void createQueue()
                             throws uk.gov.dstl.baleen.exceptions.BaleenException
        Create the Queue for the transport of the data
        Throws:
        uk.gov.dstl.baleen.exceptions.BaleenException - if the queue can not be created correctly

      • closeQueue

        protected abstract void closeQueue()
                            throws java.io.IOException
        Close the queue and release and resources
        Throws:
        java.io.IOException - if the queue can not be closed correctly

      • writeToQueue

        protected abstract void writeToQueue​(java.lang.String id,
                                     java.lang.String data)
                              throws java.io.IOException
        Write the given data to the queue
        Parameters:
        id - the id
        data - to write
        Throws:
        java.io.IOException - if writing fails

      • getQueueLength

        protected abstract int getQueueLength()
        Get the current length of the queue. This is used, with the capacity to determine if we should backoff writing to the queue.
        Returns:
        the current length of the queue or 0 if this can not be determined.