Class/Object

net.ruippeixotog.scalascraper.browser

JsoupBrowser

Related Docs: object JsoupBrowser | package browser

Permalink

class JsoupBrowser extends Browser

A Browser implementation based on jsoup, a Java HTML parser library. JsoupBrowser provides powerful and efficient document querying, but it doesn't run JavaScript in the pages. As such, it is limited to working strictly with the HTML send in the page source.

Currently, JsoupBrowser does not keep separate cookie stores for different domains and paths. In each request all cookies set previously will be sent, regardless of the domain they were set on. If you do requests to different domains and do not want this behavior, use different JsoupBrowser instances.

As the documents parsed by JsoupBrowser instances are not changed after loading, Document and Element instances obtained from them are guaranteed to be immutable.

Linear Supertypes
Browser, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. JsoupBrowser
  2. Browser
  3. AnyRef
  4. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new JsoupBrowser(userAgent: String = "jsoup/1.8", proxy: Proxy = null)

    Permalink

    userAgent

    the user agent with which requests should be made

Type Members

  1. type DocumentType = JsoupDocument

    Permalink

    The concrete type of documents created by this browser.

    The concrete type of documents created by this browser.

    Definition Classes
    JsoupBrowser → Browser

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any

  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  4. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any

  5. def clearCookies(): Unit

    Permalink

    Clears the cookie store of this browser.

    Clears the cookie store of this browser.

    Definition Classes
    JsoupBrowser → Browser

  6. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  7. def cookies(url: String): Map[String, String]

    Permalink

    Returns the current set of cookies stored in this browser for a given URL.

    Returns the current set of cookies stored in this browser for a given URL.

    url

    the URL whose stored cookies are to be returned

    returns

    a mapping of cookie names to their respective values.

    Definition Classes
    JsoupBrowser → Browser

  8. def defaultRequestSettings(conn: Connection): Connection

    Permalink
    Attributes
    protected[this]

  9. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef

  10. def equals(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  11. def executeRequest(conn: Connection): Response

    Permalink
    Attributes
    protected[this]

  12. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )

  13. def get(url: String): JsoupDocument

    Permalink

    Retrieves and parses a web page using a GET request.

    Retrieves and parses a web page using a GET request.

    url

    the URL of the page to retrieve

    returns

    a Document containing the retrieved web page.

    Definition Classes
    JsoupBrowser → Browser

  14. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any

  15. def hashCode(): Int

    Permalink
    Definition Classes
    AnyRef → Any

  16. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any

  17. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef

  18. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef

  19. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef

  20. def parseFile(file: File, charset: String): JsoupDocument

    Permalink

    Parses a local HTML file with a specified charset.

    Parses a local HTML file with a specified charset.

    file

    the HTML file to parse

    charset

    the charset of the file

    returns

    a Document containing the parsed web page.

    Definition Classes
    JsoupBrowser → Browser

  21. def parseFile(path: String): DocumentType

    Permalink

    Parses a local HTML file encoded in UTF-8.

    Parses a local HTML file encoded in UTF-8.

    path

    the path in the local filesystem where the HTML file is located

    returns

    a Document containing the parsed web page.

    Definition Classes
    Browser

  22. def parseFile(path: String, charset: String): DocumentType

    Permalink

    Parses a local HTML file with a specified charset.

    Parses a local HTML file with a specified charset.

    path

    the path in the local filesystem where the HTML file is located

    charset

    the charset of the file

    returns

    a Document containing the parsed web page.

    Definition Classes
    Browser

  23. def parseFile(file: File): DocumentType

    Permalink

    Parses a local HTML file encoded in UTF-8.

    Parses a local HTML file encoded in UTF-8.

    file

    the HTML file to parse

    returns

    a Document containing the parsed web page.

    Definition Classes
    Browser

  24. def parseInputStream(inputStream: InputStream, charset: String): JsoupDocument

    Permalink

    Parses an input stream with its content in a specified charset.

    Parses an input stream with its content in a specified charset. The provided input stream is always closed before this method returns or throws an exception.

    inputStream

    the input stream to parse

    charset

    the charset of the input stream content

    returns

    a Document containing the parsed web page.

    Definition Classes
    JsoupBrowser → Browser

  25. def parseResource(name: String, charset: String = "UTF-8"): DocumentType

    Permalink

    Parses a resource with a specified charset.

    Parses a resource with a specified charset.

    name

    the name of the resource to parse

    charset

    the charset of the resource

    returns

    a Document containing the parsed web page.

    Definition Classes
    Browser

  26. def parseString(html: String): JsoupDocument

    Permalink

    Parses an HTML string.

    Parses an HTML string.

    html

    the HTML string to parse

    returns

    a Document containing the parsed web page.

    Definition Classes
    JsoupBrowser → Browser

  27. def post(url: String, form: Map[String, String]): JsoupDocument

    Permalink

    Submits a form via a POST request and parses the resulting page.

    Submits a form via a POST request and parses the resulting page.

    url

    the URL of the page to retrieve

    form

    a map containing the form fields to submit with their respective values

    returns

    a Document containing the resulting web page.

    Definition Classes
    JsoupBrowser → Browser

  28. def processResponse(res: Response): JsoupDocument

    Permalink
    Attributes
    protected[this]

  29. val proxy: Proxy

    Permalink

  30. def requestSettings(conn: Connection): Connection

    Permalink

  31. def setCookie(url: String, key: String, value: String): Map[String, String]

    Permalink

  32. def setCookies(url: String, m: Map[String, String]): Map[String, String]

    Permalink

  33. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef

  34. def toString(): String

    Permalink
    Definition Classes
    AnyRef → Any

  35. val userAgent: String

    Permalink

    the user agent with which requests should be made

    the user agent with which requests should be made

    Definition Classes
    JsoupBrowser → Browser

  36. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  37. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  38. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from Browser

Inherited from AnyRef

Inherited from Any

Ungrouped