org.apache.spark.deploy.history

HistoryServer

class HistoryServer extends WebUI with Logging with UIRoot

A web server that renders SparkUIs of completed applications.

For the standalone mode, MasterWebUI already achieves this functionality. Thus, the main use case of the HistoryServer is in other deploy modes (e.g. Yarn or Mesos).

The logging directory structure is as follows: Within the given base directory, each application's event logs are maintained in the application's own sub-directory. This is the same structure as maintained in the event log write code path in EventLoggingListener.

Linear Supertypes
UIRoot, WebUI, Logging, AnyRef, Any
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. HistoryServer
  2. UIRoot
  3. WebUI
  4. Logging
  5. AnyRef
  6. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Instance Constructors

  1. new HistoryServer(conf: SparkConf, provider: ApplicationHistoryProvider, securityManager: SecurityManager, port: Int)

Value Members

  1. final def !=(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  2. final def !=(arg0: Any): Boolean

    Definition Classes
    Any

  3. final def ##(): Int

    Definition Classes
    AnyRef → Any

  4. final def ==(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  5. final def ==(arg0: Any): Boolean

    Definition Classes
    Any

  6. def addStaticHandler(resourceBase: String, path: String): Unit

    Add a handler for static content.

    Add a handler for static content.

    resourceBase

    Root of where to find resources to serve.

    path

    Path in UI where to mount the resources.

    Definition Classes
    WebUI

  7. final def asInstanceOf[T0]: T0

    Definition Classes
    Any

  8. def attachHandler(handler: ServletContextHandler): Unit

    Attach a handler to this UI.

    Attach a handler to this UI.

    Definition Classes
    WebUI

  9. def attachPage(page: WebUIPage): Unit

    Attach a page to this UI.

    Attach a page to this UI.

    Definition Classes
    WebUI

  10. def attachTab(tab: WebUITab): Unit

    Attach a tab to this UI, along with all of its attached pages.

    Attach a tab to this UI, along with all of its attached pages.

    Definition Classes
    WebUI

  11. def bind(): Unit

    Bind to the HTTP server behind this web interface.

    Bind to the HTTP server behind this web interface.

    Definition Classes
    HistoryServer → WebUI

  12. def boundPort: Int

    Return the actual port to which this server is bound.

    Return the actual port to which this server is bound. Only valid after bind().

    Definition Classes
    WebUI

  13. def clone(): AnyRef

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  14. def detachHandler(handler: ServletContextHandler): Unit

    Detach a handler from this UI.

    Detach a handler from this UI.

    Definition Classes
    WebUI

  15. def detachPage(page: WebUIPage): Unit

    Definition Classes
    WebUI

  16. def detachTab(tab: WebUITab): Unit

    Definition Classes
    WebUI

  17. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  18. def equals(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any

  19. def finalize(): Unit

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )

  20. def getApplicationInfoList: Iterator[ApplicationInfo]

    Definition Classes
    HistoryServer → UIRoot

  21. def getApplicationList(): Iterable[ApplicationHistoryInfo]

    Returns a list of available applications, in descending order according to their end time.

    Returns a list of available applications, in descending order according to their end time.

    returns

    List of all known applications.

  22. def getBasePath: String

    Definition Classes
    WebUI

  23. final def getClass(): Class[_]

    Definition Classes
    AnyRef → Any

  24. def getHandlers: Seq[ServletContextHandler]

    Definition Classes
    WebUI

  25. def getProviderConfig(): Map[String, String]

    Returns the provider configuration to show in the listing page.

    Returns the provider configuration to show in the listing page.

    returns

    A map with the provider's configuration.

  26. def getSecurityManager: SecurityManager

    Definition Classes
    WebUI

  27. def getSparkUI(appKey: String): Option[SparkUI]

    Definition Classes
    HistoryServer → UIRoot

  28. def getTabs: Seq[WebUITab]

    Definition Classes
    WebUI

  29. val handlers: ArrayBuffer[ServletContextHandler]

    Attributes
    protected
    Definition Classes
    WebUI

  30. def hashCode(): Int

    Definition Classes
    AnyRef → Any

  31. def initialize(): Unit

    Initialize the history server.

    Initialize the history server.

    This starts a background thread that periodically synchronizes information displayed on this UI with the event logs in the provided base directory.

    Definition Classes
    HistoryServer → WebUI

  32. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any

  33. def isTraceEnabled(): Boolean

    Attributes
    protected
    Definition Classes
    Logging

  34. val localHostName: String

    Attributes
    protected
    Definition Classes
    WebUI

  35. def log: Logger

    Attributes
    protected
    Definition Classes
    Logging

  36. def logDebug(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging

  37. def logDebug(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging

  38. def logError(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging

  39. def logError(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging

  40. def logInfo(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging

  41. def logInfo(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging

  42. def logName: String

    Attributes
    protected
    Definition Classes
    Logging

  43. def logTrace(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging

  44. def logTrace(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging

  45. def logWarning(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging

  46. def logWarning(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging

  47. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  48. final def notify(): Unit

    Definition Classes
    AnyRef

  49. final def notifyAll(): Unit

    Definition Classes
    AnyRef

  50. val pageToHandlers: HashMap[WebUIPage, ArrayBuffer[ServletContextHandler]]

    Attributes
    protected
    Definition Classes
    WebUI

  51. val publicHostName: String

    Attributes
    protected
    Definition Classes
    WebUI

  52. def removeStaticHandler(path: String): Unit

    Remove a static content handler.

    Remove a static content handler.

    path

    Path in UI to unmount.

    Definition Classes
    WebUI

  53. var serverInfo: Option[ServerInfo]

    Attributes
    protected
    Definition Classes
    WebUI

  54. def stop(): Unit

    Stop the server and close the file system.

    Stop the server and close the file system.

    Definition Classes
    HistoryServer → WebUI

  55. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef

  56. val tabs: ArrayBuffer[WebUITab]

    Attributes
    protected
    Definition Classes
    WebUI

  57. def toString(): String

    Definition Classes
    AnyRef → Any

  58. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  59. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  60. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  61. def withSparkUI[T](appId: String, attemptId: Option[String])(f: (SparkUI) ⇒ T): T

    Get the spark UI with the given appID, and apply a function to it.

    Get the spark UI with the given appID, and apply a function to it. If there is no such app, throw an appropriate exception

    Definition Classes
    UIRoot

  62. def writeEventLogs(appId: String, attemptId: Option[String], zipStream: ZipOutputStream): Unit

    Write the event logs for the given app to the ZipOutputStream instance.

    Write the event logs for the given app to the ZipOutputStream instance. If attemptId is None, event logs for all attempts of this application will be written out.

    Definition Classes
    HistoryServer → UIRoot

Inherited from UIRoot

Inherited from WebUI

Inherited from Logging

Inherited from AnyRef

Inherited from Any

Ungrouped