org.apache.spark.memory

SnappyUnifiedMemoryManager

class SnappyUnifiedMemoryManager extends UnifiedMemoryManager

When there is request for execution or storage memory, critical up and eviction up events are checked. If they are set, try to free the memory cached by Spark rdds by calling memoryStore.evictBlocksToFreeSpace. If enough memory cannot be freed, return the call and let Spark take a corrective action. In such cases Spark either fails the task or move the current RDDs data to disk. If the critical and eviction events are not set, it asks the UnifiedMemoryManager to allocate the space.

Linear Supertypes
UnifiedMemoryManager, MemoryManager, Logging, AnyRef, Any
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. SnappyUnifiedMemoryManager
  2. UnifiedMemoryManager
  3. MemoryManager
  4. Logging
  5. AnyRef
  6. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Instance Constructors

  1. new SnappyUnifiedMemoryManager(conf: SparkConf, numCores: Int)

Value Members

  1. final def !=(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  2. final def !=(arg0: Any): Boolean

    Definition Classes
    Any

  3. final def ##(): Int

    Definition Classes
    AnyRef → Any

  4. final def ==(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  5. final def ==(arg0: Any): Boolean

    Definition Classes
    Any

  6. def acquireStorageMemory(blockId: BlockId, numBytes: Long, evictedBlocks: Buffer[(BlockId, BlockStatus)]): Boolean

    Definition Classes
    SnappyUnifiedMemoryManager → UnifiedMemoryManager → MemoryManager

  7. def acquireUnrollMemory(blockId: BlockId, numBytes: Long, evictedBlocks: Buffer[(BlockId, BlockStatus)]): Boolean

    Definition Classes
    UnifiedMemoryManager → MemoryManager

  8. final def asInstanceOf[T0]: T0

    Definition Classes
    Any

  9. def clone(): AnyRef

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  10. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  11. def equals(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any

  12. final def executionMemoryUsed: Long

    Definition Classes
    MemoryManager

  13. def finalize(): Unit

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )

  14. final def getClass(): Class[_]

    Definition Classes
    AnyRef → Any

  15. def hashCode(): Int

    Definition Classes
    AnyRef → Any

  16. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any

  17. def isTraceEnabled(): Boolean

    Attributes
    protected
    Definition Classes
    Logging

  18. def log: Logger

    Attributes
    protected
    Definition Classes
    Logging

  19. def logDebug(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging

  20. def logDebug(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging

  21. def logError(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging

  22. def logError(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging

  23. def logInfo(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging

  24. def logInfo(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging

  25. def logName: String

    Attributes
    protected
    Definition Classes
    Logging

  26. def logTrace(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging

  27. def logTrace(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging

  28. def logWarning(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging

  29. def logWarning(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging

  30. val maxMemory: Long

    Definition Classes
    SnappyUnifiedMemoryManager → UnifiedMemoryManager

  31. def maxStorageMemory: Long

    Definition Classes
    UnifiedMemoryManager → MemoryManager

  32. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  33. final def notify(): Unit

    Definition Classes
    AnyRef

  34. final def notifyAll(): Unit

    Definition Classes
    AnyRef

  35. val offHeapExecutionMemoryPool: ExecutionMemoryPool

    Attributes
    protected
    Definition Classes
    MemoryManager

  36. val onHeapExecutionMemoryPool: ExecutionMemoryPool

    Attributes
    protected
    Definition Classes
    MemoryManager

  37. val pageSizeBytes: Long

    Definition Classes
    MemoryManager

  38. final def releaseAllStorageMemory(): Unit

    Definition Classes
    MemoryManager

  39. def releaseStorageMemory(numBytes: Long): Unit

    Definition Classes
    MemoryManager

  40. final def releaseUnrollMemory(numBytes: Long): Unit

    Definition Classes
    MemoryManager

  41. final def setMemoryStore(store: MemoryStore): Unit

    Definition Classes
    MemoryManager

  42. val storageMemoryPool: StorageMemoryPool

    Attributes
    protected
    Definition Classes
    MemoryManager

  43. final def storageMemoryUsed: Long

    Definition Classes
    MemoryManager

  44. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef

  45. def toString(): String

    Definition Classes
    AnyRef → Any

  46. final val tungstenMemoryMode: MemoryMode

    Definition Classes
    MemoryManager

  47. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  48. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  49. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from UnifiedMemoryManager

Inherited from MemoryManager

Inherited from Logging

Inherited from AnyRef

Inherited from Any

Ungrouped