Package org.archive.modules.net

Class RobotsTxtOnlyPolicy

java.lang.Object

org.archive.modules.net.RobotsPolicy

org.archive.modules.net.ObeyRobotsPolicy

org.archive.modules.net.RobotsTxtOnlyPolicy

public class RobotsTxtOnlyPolicy
extends ObeyRobotsPolicy

Policy to obey robots.txt but ignore meta nofollow.

Field Summary

Fields

Modifier and Type	Field	Description
static RobotsPolicy	INSTANCE

Fields inherited from class org.archive.modules.net.RobotsPolicy

STANDARD_POLICIES

Constructor Summary

Constructors

Constructor	Description
RobotsTxtOnlyPolicy()

Method Summary

Modifier and Type	Method	Description
boolean	obeyMetaRobotsNofollow()

Methods inherited from class org.archive.modules.net.ObeyRobotsPolicy

allows

Methods inherited from class org.archive.modules.net.RobotsPolicy

getPathQuery

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

INSTANCE

public static RobotsPolicy INSTANCE

Constructor Details

RobotsTxtOnlyPolicy

public RobotsTxtOnlyPolicy()

Method Details

obeyMetaRobotsNofollow

public boolean obeyMetaRobotsNofollow()

Overrides:

obeyMetaRobotsNofollow in class ObeyRobotsPolicy