org.zaproxy.zap.spider

Class URLCanonicalizer

java.lang.Object

org.zaproxy.zap.spider.URLCanonicalizer

public final class URLCanonicalizer
extends Object

The URLCanonicalizer is used for the process of converting an URL into a canonical (normalized) form. See URL Normalization for a reference.

Note: some parts of the code are adapted from: stackoverflow

Added support for OData URLs

Method Summary

Modifier and Type	Method and Description
static String	buildCleanedParametersURIRepresentation(org.apache.commons.httpclient.URI uri, SpiderParam.HandleParametersOption handleParameters, boolean handleODataParametersVisited) Builds a String representation of the URI with cleaned parameters, that can be used when checking if an URI was already visited.
static String	getCanonicalURL(String url) Gets the canonical url.
static String	getCanonicalURL(String url, String baseURL) Gets the canonical url, starting from a relative or absolute url found in a given context (baseURL).

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

getCanonicalURL

public static String getCanonicalURL(String url)

Gets the canonical url.

Parameters:

url - the url

Returns:

the canonical url

getCanonicalURL

public static String getCanonicalURL(String url,
                                     String baseURL)

Gets the canonical url, starting from a relative or absolute url found in a given context (baseURL).

Parameters:

url - the url string defining the reference

baseURL - the context in which this url was found

Returns:

the canonical url

buildCleanedParametersURIRepresentation

public static String buildCleanedParametersURIRepresentation(org.apache.commons.httpclient.URI uri,
                                                             SpiderParam.HandleParametersOption handleParameters,
                                                             boolean handleODataParametersVisited)
                                                      throws org.apache.commons.httpclient.URIException

Builds a String representation of the URI with cleaned parameters, that can be used when checking if an URI was already visited. The URI provided as a parameter should be already cleaned and canonicalized, so it should be build with a result from getCanonicalURL(String).

When building the URI representation, the same format should be used for all the cases, as it may affect the number of times the pages are visited and reported if the option HandleParametersOption is changed while the spider is running.

Parameters:

uri - the uri

handleParameters - the handle parameters option

handleODataParametersVisited - Should we handle specific OData parameters

Returns:

the string representation of the URI

Throws:

org.apache.commons.httpclient.URIException - the URI exception