Class Spider


  • @Deprecated
    public class Spider
    extends java.lang.Object
    Deprecated.
    (2.12.0) See the spider add-on in zap-extensions instead.
    The Class Spider.
    • Constructor Detail

      • Spider

        @Deprecated
        public Spider​(java.lang.String id,
                      ExtensionSpider extension,
                      SpiderParam spiderParam,
                      ConnectionParam connectionParam,
                      Model model,
                      Context scanContext)
        Deprecated.
        (2.12.0) Use #Spider(String, ExtensionSpider, SpiderParam, Model, Context) instead.
        Constructs a Spider with the given data.
        Parameters:
        id - the ID of the spider, usually a unique integer
        extension - the extension
        spiderParam - the spider param
        connectionParam - the connection param
        model - the model
        scanContext - if a scan context is set, only URIs within the context are fetched and processed
        Since:
        2.6.0
      • Spider

        public Spider​(java.lang.String id,
                      ExtensionSpider extension,
                      SpiderParam spiderParam,
                      Model model,
                      Context scanContext)
        Deprecated.
        Constructs a Spider with the given data.
        Parameters:
        id - the ID of the spider, usually a unique integer
        extension - the extension
        spiderParam - the spider param
        model - the model
        scanContext - if a scan context is set, only URIs within the context are fetched and processed
        Since:
        2.12.0
    • Method Detail

      • addSeed

        public void addSeed​(HttpMessage msg)
        Deprecated.
        Adds a new seed for the Spider.
        Parameters:
        msg - the message used for seed. The request URI is used from the Request Header
      • addSeed

        public void addSeed​(org.apache.commons.httpclient.URI uri)
        Deprecated.
        Adds a new seed for the Spider.
        Parameters:
        uri - the uri
      • setExcludeList

        public void setExcludeList​(java.util.List<java.lang.String> excludeList)
        Deprecated.
        Sets the exclude list which contains a List of strings, defining the uris that should be excluded.
        Parameters:
        excludeList - the new exclude list
      • addFetchFilter

        public void addFetchFilter​(FetchFilter filter)
        Deprecated.
        Adds a new fetch filter to the spider.
        Parameters:
        filter - the filter
      • addParseFilter

        public void addParseFilter​(ParseFilter filter)
        Deprecated.
        Adds a new parse filter to the spider.
        Parameters:
        filter - the filter
      • getHttpSender

        protected HttpSender getHttpSender()
        Deprecated.
        Gets the http sender. Can be called from the SpiderTask.
        Returns:
        the http sender
      • getSpiderParam

        protected SpiderParam getSpiderParam()
        Deprecated.
        Gets the spider parameters. Can be called from the SpiderTask.
        Returns:
        the spider parameters
      • getController

        protected SpiderController getController()
        Deprecated.
        Gets the controller.
        Returns:
        the controller
      • getModel

        protected Model getModel()
        Deprecated.
        Gets the model.
        Returns:
        the model
      • submitTask

        protected void submitTask​(SpiderTask task)
        Deprecated.
        Submit a new task to the spidering task pool.
        Parameters:
        task - the task
      • getExtensionSpider

        protected ExtensionSpider getExtensionSpider()
        Deprecated.
        Gets the extension.
        Returns:
        the extension
      • start

        public void start()
        Deprecated.
        Starts the Spider crawling.
      • stop

        public void stop()
        Deprecated.
        Stops the Spider crawling. Must not be called from any of the threads in the thread pool.
      • pause

        public void pause()
        Deprecated.
        Pauses the Spider crawling.
      • resume

        public void resume()
        Deprecated.
        Resumes the Spider crawling.
      • setScanAsUser

        public void setScanAsUser​(User user)
        Deprecated.
        Sets the spider so it will scan from the point of view of a user.
        Parameters:
        user - the user to be scanned as
      • getScanUser

        protected User getScanUser()
        Deprecated.
        Gets the user that will be used in the scanning.
        Returns:
        the scan user
      • preTaskExecution

        protected void preTaskExecution()
        Deprecated.
        This method is run by each thread in the Thread Pool before the task execution. Particularly, it checks if the Spidering process is paused and, if it is, it waits on the corresponding condition for the process to be resumed. Called from the SpiderTask.
      • checkPauseAndWait

        protected void checkPauseAndWait()
        Deprecated.
        This method is run by Threads in the ThreadPool and checks if the scan is paused and, if it is, waits until it's unpaused.
      • postTaskExecution

        protected void postTaskExecution()
        Deprecated.
        This method is run by each thread in the Thread Pool after the task execution. Particularly, it notifies the listeners of the progress and checks if the scan is complete. Called from the SpiderTask.
      • isPaused

        public boolean isPaused()
        Deprecated.
        Checks if is paused.
        Returns:
        true, if is paused
      • isStopped

        public boolean isStopped()
        Deprecated.
        Checks if is stopped, i.e. a shutdown was issued or it is not running.
        Returns:
        true, if is stopped
      • isTerminated

        public boolean isTerminated()
        Deprecated.
        Checks if is terminated.
        Returns:
        true, if is terminated
      • addSpiderListener

        public void addSpiderListener​(SpiderListener listener)
        Deprecated.
        Adds a new spider listener.
        Parameters:
        listener - the listener
      • removeSpiderListener

        public void removeSpiderListener​(SpiderListener listener)
        Deprecated.
        Removes a spider listener.
        Parameters:
        listener - the listener
      • notifyListenersSpiderProgress

        protected void notifyListenersSpiderProgress​(int percentageComplete,
                                                     int numberCrawled,
                                                     int numberToCrawl)
        Deprecated.
        Notifies all the listeners regarding the spider progress.
        Parameters:
        percentageComplete - the percentage complete
        numberCrawled - the number of pages crawled
        numberToCrawl - the number of pages left to crawl
      • notifyListenersFoundURI

        protected void notifyListenersFoundURI​(java.lang.String uri,
                                               java.lang.String method,
                                               FetchFilter.FetchStatus status)
        Deprecated.
        Notifies the listeners regarding a found uri.
        Parameters:
        uri - the uri
        method - the method used for fetching the resource
        status - the FetchStatus stating if this uri will be processed, and, if not, stating the reason of the filtering
      • notifyListenersSpiderTaskResult

        protected void notifyListenersSpiderTaskResult​(SpiderTaskResult result)
        Deprecated.
        Notifies the listeners of a SpiderTask's result.
        Parameters:
        result - the result of a spider task.
      • notifyListenersSpiderComplete

        protected void notifyListenersSpiderComplete​(boolean successful)
        Deprecated.
        Notifies the listeners that the spider is complete.
        Parameters:
        successful - true if the spider completed successfully (e.g. was not stopped), false otherwise
      • addCustomParser

        public void addCustomParser​(SpiderParser sp)
        Deprecated.