SchedulingURLBuffer (storm-crawler-core 2.8 API)

java.lang.Object
- com.digitalpebble.stormcrawler.persistence.urlbuffer.AbstractURLBuffer
- - com.digitalpebble.stormcrawler.persistence.urlbuffer.SchedulingURLBuffer

All Implemented Interfaces:

URLBuffer, com.github.benmanes.caffeine.cache.RemovalListener<String,Object[]>
```
public class SchedulingURLBuffer
extends AbstractURLBuffer
implements com.github.benmanes.caffeine.cache.RemovalListener<String,Object[]>
```
Checks how long the last N URLs took to work out whether a queue should release a URL.

Nested Class Summary
- Nested classes/interfaces inherited from class com.digitalpebble.stormcrawler.persistence.urlbuffer.AbstractURLBuffer
  AbstractURLBuffer.URLMetadata

Field Summary

Fields
Modifier and Type Field Description

static String MAXTIMEPARAM
- Fields inherited from class com.digitalpebble.stormcrawler.persistence.urlbuffer.AbstractURLBuffer
  in_buffer, listener, partitioner, queues
- Fields inherited from interface com.digitalpebble.stormcrawler.persistence.urlbuffer.URLBuffer
  bufferClassParamName

Constructor Summary

Constructors
Constructor Description

SchedulingURLBuffer()

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method	Description
`void`	`acked(String url)`	Notify the buffer that a URL has been successfully processed used e.g to compute an ideal delay for a host queue
`void`	`configure(Map<String,Object> stormConf)`
`org.apache.storm.tuple.Values`	`next()`	Retrieves the next available URL, guarantees that the URLs are always perfectly shuffled
`void`	`onRemoval(@Nullable String key, Object @Nullable [] value, @NotNull com.github.benmanes.caffeine.cache.RemovalCause cause)`

Methods inherited from class com.digitalpebble.stormcrawler.persistence.urlbuffer.AbstractURLBuffer
add, add, hasNext, numQueues, setEmptyQueueListener, size

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - MAXTIMEPARAM
```
public static final String MAXTIMEPARAM
```
    See Also:
    
    Constant Field Values
- Constructor Detail
  - SchedulingURLBuffer
```
public SchedulingURLBuffer()
```
- Method Detail
  - configure
```
public void configure(Map<String,Object> stormConf)
```
    Specified by:
    
    configure in interface URLBuffer
    
    Overrides:
    
    configure in class AbstractURLBuffer
  - next
```
public org.apache.storm.tuple.Values next()
```
    Retrieves the next available URL, guarantees that the URLs are always perfectly shuffled
    
    Specified by:
    
    next in interface URLBuffer
    
    Returns:
    
    null if no entries are available
  - acked
```
public void acked(String url)
```
    Description copied from interface: URLBuffer
    
    Notify the buffer that a URL has been successfully processed used e.g to compute an ideal delay for a host queue
    
    Specified by:
    
    acked in interface URLBuffer
  - onRemoval
```
public void onRemoval(@Nullable
                      @Nullable String key,
                      Object @Nullable [] value,
                      @NotNull
                      @NotNull com.github.benmanes.caffeine.cache.RemovalCause cause)
```
    Specified by:
    
    onRemoval in interface com.github.benmanes.caffeine.cache.RemovalListener<String,Object[]>