Package org.archive.crawler.datamodel

package org.archive.crawler.datamodel
  • Interfaces
    Class
    Description
    UriUniqFilter
    A UriUniqFilter passes URI objects to a destination (receiver) if the passed URI object has not been previously seen.
    UriUniqFilter.CrawlUriReceiver
    URIs that pass the filter (are new / unique / not already-seen) are passed to this object, typically a frontier.