Class DefaultScheduler
- java.lang.Object
-
- com.digitalpebble.stormcrawler.persistence.Scheduler
-
- com.digitalpebble.stormcrawler.persistence.DefaultScheduler
-
- Direct Known Subclasses:
AdaptiveScheduler
public class DefaultScheduler extends Scheduler
Schedules a nextFetchDate based on the configuration
-
-
Field Summary
Fields Modifier and Type Field Description static String
DELAY_METADATA
Key used to pass a custom delay via metadata.-
Fields inherited from class com.digitalpebble.stormcrawler.persistence.Scheduler
schedulerClassParamName
-
-
Constructor Summary
Constructors Constructor Description DefaultScheduler()
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description protected Optional<Integer>
checkCustomInterval(Metadata metadata, Status s)
Returns the first matching custom intervalvoid
init(Map<String,Object> stormConf)
Configuration of the scheduler based on the config.Optional<Date>
schedule(Status status, Metadata metadata)
Returns an optional Date indicating when the document should be refetched next, based on its status.-
Methods inherited from class com.digitalpebble.stormcrawler.persistence.Scheduler
getInstance
-
-
-
-
Field Detail
-
DELAY_METADATA
public static final String DELAY_METADATA
Key used to pass a custom delay via metadata. Used by the sitemaps to stagger the scheduling of URLs.- See Also:
- Constant Field Values
-
-
Method Detail
-
init
public void init(Map<String,Object> stormConf)
Description copied from class:Scheduler
Configuration of the scheduler based on the config. Should be called by Scheduler.getInstance() *
-
schedule
public Optional<Date> schedule(Status status, Metadata metadata)
Description copied from class:Scheduler
Returns an optional Date indicating when the document should be refetched next, based on its status. It is empty if the URL should never be refetched.
-
-