Package com.yahoo.docproc

Class DocprocService

  • All Implemented Interfaces:
    com.yahoo.component.Component, com.yahoo.component.Deconstructable, Comparable<com.yahoo.component.Component>
    public class DocprocService
extends com.yahoo.component.AbstractComponent

    The document processing service. Use this to set up a document processing chain and to process documents using that chain. Note that there may be multiple named instances of this service in the same runtime. The default service is called "default" and is always present.

    To create a server which receives documents from the network and processes them, have a look at com.yahoo.docproc.server.Server.

    This class is thread safe.

    Author:
    bratseth

    • Field Summary

      Fields 
      Modifier and Type Field Description
      static SchemaMap schemaMap  

      • Fields inherited from class com.yahoo.component.AbstractComponent

        isDeconstructable

    • Constructor Summary

      Constructors 
      Constructor Description
      DocprocService​(com.yahoo.component.ComponentId id)  
      DocprocService​(com.yahoo.component.ComponentId id, CallStack stack, com.yahoo.document.DocumentTypeManager mgr)
      Deprecated.
      DocprocService​(com.yahoo.component.ComponentId id, CallStack stack, com.yahoo.document.DocumentTypeManager mgr, int numThreads)
      Creates a new docproc service, which is set to be in service.
      DocprocService​(String name)
      Creates a service with a name with an unbounded input queue.

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void deconstruct()  
      boolean doWork()
      Do some work in this service.
      boolean doWorkBlocking()
      Do some work in this service.
      CallStack getCallStack()
      Returns the processing chain of this service.
      com.yahoo.document.DocumentTypeManager getDocumentTypeManager()  
      DocprocExecutor getExecutor()
      Returns the DocprocExecutor of this DocprocService.
      String getName()  
      int getQueueSize()  
      ThreadPoolExecutor getThreadPoolExecutor()  
      boolean isAcceptingNewProcessings()
      Returns true if this service currently accepts new incoming processings via process(...). Default is true.
      boolean isInService()
      Returns true if this is currently processing incoming processings (in service), or false if they are just queued up (out of service).
      void process​(Processing processing)
      Asynchronously process the given Processing using the processing chain of this service
      void process​(Processing processing, ProcessingEndpoint endp)
      Asynchronously process the given Processing using the processing chain of this service, and call the specified ProcessingEndpoint when done.
      void process​(com.yahoo.document.DocumentOperation documentOperation)
      Asynchronously process the given document operation using the processing chain of this service.
      void process​(com.yahoo.document.DocumentOperation documentOperation, ProcessingEndpoint endp)
      Asynchronously process the given document put or document update using the processing chain of this service, and call the specified ProcessingEndpoint when done.
      void processDocumentOperations​(List<com.yahoo.document.DocumentOperation> documentOperations)
      Asynchronously process the given document operations as one unit using the processing chain of this service.
      void processDocumentOperations​(List<com.yahoo.document.DocumentOperation> documentOperations, ProcessingEndpoint endp)
      Asynchronously process the given document operations as one unit using the processing chain of this service, and call the specified ProcessingEndpoint when done.
      void setAcceptingNewProcessings​(boolean acceptingNewProcessings)
      Sets whether this service should accept new incoming processings via process(...).
      void setCallStack​(CallStack stack)
      Sets a new processing stack for this service.
      void setDocumentTypeManager​(com.yahoo.document.DocumentTypeManager documentTypeManager)  
      void setInService​(boolean inService)
      Sets whether this should currently perform any processing.

      • Methods inherited from class com.yahoo.component.AbstractComponent

        clone, compareTo, getClassName, getId, getIdString, hasInitializedId, initId, isDeconstructable, setIsDeconstructable, toString

    • Field Detail

      • schemaMap

        public static SchemaMap schemaMap

    • Constructor Detail

      • DocprocService

        public DocprocService​(com.yahoo.component.ComponentId id)

      • DocprocService

        public DocprocService​(com.yahoo.component.ComponentId id,
                      CallStack stack,
                      com.yahoo.document.DocumentTypeManager mgr,
                      int numThreads)
        Creates a new docproc service, which is set to be in service.
        Parameters:
        id - the component id of the new service.
        stack - the call stack to use.
        mgr - the document type manager to use.
        numThreads - to have in the thread pool

      • DocprocService

        @Deprecated
public DocprocService​(com.yahoo.component.ComponentId id,
                      CallStack stack,
                      com.yahoo.document.DocumentTypeManager mgr)
        Deprecated.

      • DocprocService

        public DocprocService​(String name)
        Creates a service with a name with an unbounded input queue. If the given name is null or the empty string, it will become the name "default". Testing only

    • Method Detail

      • deconstruct

        public void deconstruct()
        Specified by:
        deconstruct in interface com.yahoo.component.Deconstructable
        Overrides:
        deconstruct in class com.yahoo.component.AbstractComponent

      • getDocumentTypeManager

        public com.yahoo.document.DocumentTypeManager getDocumentTypeManager()

      • setDocumentTypeManager

        public void setDocumentTypeManager​(com.yahoo.document.DocumentTypeManager documentTypeManager)

      • getQueueSize

        public int getQueueSize()

      • getExecutor

        public DocprocExecutor getExecutor()
        Returns the DocprocExecutor of this DocprocService. This can be used to synchronously process one Processing.
        Returns:
        the DocprocExecutor of this DocprocService, or null if a CallStack has not yet been set.

      • setInService

        public void setInService​(boolean inService)
        Sets whether this should currently perform any processing. New processings will be accepted also when this is out of service, but no processing will happen when it is out of service.

      • isInService

        public boolean isInService()
        Returns true if this is currently processing incoming processings (in service), or false if they are just queued up (out of service). By default, this is out of service.

      • isAcceptingNewProcessings

        public boolean isAcceptingNewProcessings()
        Returns true if this service currently accepts new incoming processings via process(...). Default is true.
        Returns:
        true if accepting new incoming processings

      • setAcceptingNewProcessings

        public void setAcceptingNewProcessings​(boolean acceptingNewProcessings)
        Sets whether this service should accept new incoming processings via process(...).
        Parameters:
        acceptingNewProcessings - true if service should accept new incoming processings

      • getName

        public String getName()

      • getCallStack

        public CallStack getCallStack()
        Returns the processing chain of this service. This stack can not be modified. To change the stack, set a new one.

      • setCallStack

        public void setCallStack​(CallStack stack)
        Sets a new processing stack for this service. This will be the Prototype for the call stacks of individual processings in this service

      • process

        public void process​(Processing processing,
                    ProcessingEndpoint endp)
        Asynchronously process the given Processing using the processing chain of this service, and call the specified ProcessingEndpoint when done.
        Throws:
        RuntimeException - caused by a QueueFullException if this DocprocService has a bounded input queue and the queue is full
        IllegalStateException - if this DocprocService is not accepting new incoming processings

      • process

        public void process​(Processing processing)
        Asynchronously process the given Processing using the processing chain of this service
        Throws:
        RuntimeException - caused by a QueueFullException if this DocprocService has a bounded input queue and the queue is full
        IllegalStateException - if this DocprocService is not accepting new incoming processings

      • process

        public void process​(com.yahoo.document.DocumentOperation documentOperation,
                    ProcessingEndpoint endp)
        Asynchronously process the given document put or document update using the processing chain of this service, and call the specified ProcessingEndpoint when done.
        Throws:
        RuntimeException - caused by a QueueFullException if this DocprocService has a bounded input queue and the queue is full
        IllegalStateException - if this DocprocService is not accepting new incoming processings

      • process

        public void process​(com.yahoo.document.DocumentOperation documentOperation)
        Asynchronously process the given document operation using the processing chain of this service.
        Throws:
        RuntimeException - caused by a QueueFullException if this DocprocService has a bounded input queue and the queue is full
        IllegalStateException - if this DocprocService is not accepting new incoming processings

      • processDocumentOperations

        public void processDocumentOperations​(List<com.yahoo.document.DocumentOperation> documentOperations,
                                      ProcessingEndpoint endp)
        Asynchronously process the given document operations as one unit using the processing chain of this service, and call the specified ProcessingEndpoint when done.
        Throws:
        RuntimeException - caused by a QueueFullException if this DocprocService has a bounded input queue and the queue is full
        IllegalStateException - if this DocprocService is not accepting new incoming processings

      • processDocumentOperations

        public void processDocumentOperations​(List<com.yahoo.document.DocumentOperation> documentOperations)
        Asynchronously process the given document operations as one unit using the processing chain of this service.
        Throws:
        RuntimeException - caused by a QueueFullException if this DocprocService has a bounded input queue and the queue is full
        IllegalStateException - if this DocprocService is not accepting new incoming processings

      • doWork

        public boolean doWork()

        Do some work in this service. This will perform some processing and return in a "short" while, as long as individual processors also returns.

        This method is thread safe - multiple threads may call doWork at any time. Note that processors should be non-blocking, so multiple threads should be used primarily to utilize multiple processors.

        Returns:
        true if some work was performed, false if no work could be performed at this time, because there are no outstanding processings, or because this is out of service. Note that since processings may arrive or be put back by another thread at any time, this return value does not mean that no work will be available if doWork as called again immediately.

      • doWorkBlocking

        public boolean doWorkBlocking()
                       throws InterruptedException

        Do some work in this service. This will perform some processing and return in a "short" while, as long as individual processors also returns. Note that if the internal queue is empty when this method is called, it will block until some work is submitted by a call to process() by another thread.

        This method is thread safe - multiple threads may call doWorkBlocking at any time. Note that processors should be non-blocking, so multiple threads should be used primarily to utilize multiple processors.

        Returns:
        always true, since if the internal queue is empty when this method is called, it will block until some work is submitted by a call to process() by another thread.
        Throws:
        InterruptedException - if a call to this method is interrupted while waiting for data to become available