Packages

  • package root

    The Scala compiler API.

    The Scala compiler API.

    The following resources are useful for Scala plugin/compiler development:

    Definition Classes
    root
  • package scala
    Definition Classes
    root
  • package tools
    Definition Classes
    scala
  • package nsc
    Definition Classes
    tools
  • package backend
    Definition Classes
    nsc
  • package jvm
    Definition Classes
    backend
  • package analysis

    Summary on the ASM analyzer framework --------------------------------------

    Summary on the ASM analyzer framework --------------------------------------

    Value

    • Abstract, needs to be implemented for each analysis.
    • Represents the desired information about local variables and stack values, for example:
      • Is this value known to be null / not null?
      • What are the instructions that could potentially have produced this value?

    Interpreter

    • Abstract, needs to be implemented for each analysis. Sometimes one can subclass an existing interpreter, e.g., SourceInterpreter or BasicInterpreter.
    • Multiple abstract methods that receive an instruction and the instruction's input values, and return a value representing the result of that instruction.
      • Note: due to control flow, the interpreter can be invoked multiple times for the same instruction, until reaching a fixed point.
    • Abstract merge function that computes the least upper bound of two values. Used by Frame.merge (see below).

    Frame

    • Can be used directly for many analyses, no subclass required.
    • Every frame has an array of values: one for each local variable and for each stack slot.
      • A top index stores the index of the current stack top
      • NOTE: for a size-2 local variable at index i, the local variable at i+1 is set to an empty value. However, for a size-2 value at index i on the stack, the value at i+1 holds the next stack value. IMPORTANT: this is only the case in ASM's analysis framework, not in bytecode. See comment below.
    • Defines the execute(instruction) method.
      • executing mutates the state of the frame according to the effect of the instruction
        • pop consumed values from the stack
        • pass them to the interpreter together with the instruction
        • if applicable, push the resulting value on the stack
    • Defines the merge(otherFrame) method
      • called by the analyzer when multiple control flow paths lead to an instruction
        • the frame at the branching instruction is merged into the current frame of the instruction (held by the analyzer)
        • mutates the values of the current frame, merges all values using interpreter.merge.

    Analyzer

    • Stores a frame for each instruction
    • merge function takes an instruction and a frame, merges the existing frame for that instr (from the frames array) with the new frame passed as argument. if the frame changed, puts the instruction on the work queue (fixpoint).
    • initial frame: initialized for first instr by calling interpreter.new[...]Value for each slot (locals and params), stored in frames[firstInstr] by calling merge
    • work queue of instructions (queue array, top index for next instruction to analyze)
    • analyze(method): simulate control flow. while work queue non-empty:
      • copy the state of frames[instr] into a local frame current
      • call current.execute(instr, interpreter), mutating the current frame
      • if it's a branching instruction
        • for all potential destination instructions
          • merge the destination instruction frame with the current frame (this enqueues the destination instr if its frame changed)
        • invoke newControlFlowEdge (see below)
    • the analyzer also tracks active exception handlers at each instruction
    • the empty method newControlFlowEdge can be overridden to track control flow if required

    MaxLocals and MaxStack ----------------------

    At the JVM level, long and double values occupy two slots, both as local variables and on the stack, as specified in the JVM spec 2.6.2: "At any point in time, an operand stack has an associated depth, where a value of type long or double contributes two units to the depth and a value of any other type contributes one unit."

    For example, a method class A { def f(a: Long, b: Long) = a + b } has MAXSTACK=4 in the classfile. This value is computed by the ClassWriter / MethodWriter when generating the classfile (we always pass COMPUTE_MAXS to the ClassWriter).

    For running an ASM Analyzer, long and double values occupy two local variable slots, but only a single slot on the call stack, as shown by the following snippet:

    import scala.tools.nsc.backend.jvm._ import scala.tools.nsc.backend.jvm.opt.BytecodeUtils._ import scala.collection.convert.decorateAsScala._ import scala.tools.asm.tree.analysis._

    val cn = AsmUtils.readClass("/Users/luc/scala/scala/sandbox/A.class") val m = cn.methods.iterator.asScala.find(_.name == "f").head

    // the value is read from the classfile, so it's 4 println(s"maxLocals: ${m.maxLocals}, maxStack: ${m.maxStack}") // maxLocals: 5, maxStack: 4

    // we can safely set it to 2 for running the analyzer. m.maxStack = 2

    val a = new Analyzer(new BasicInterpreter) a.analyze(cn.name, m) val addInsn = m.instructions.iterator.asScala.find(_.getOpcode == 97).get // LADD Opcode val addFrame = a.frameAt(addInsn, m)

    addFrame.getStackSize // 2: the two long values only take one slot each addFrame.getLocals // 5: this takes one slot, the two long parameters take 2 slots each

    While running the optimizer, we need to make sure that the maxStack value of a method is large enough for running an ASM analyzer. We don't need to worry if the value is incorrect in the JVM perspective: the value will be re-computed and overwritten in the ClassWriter.

    Lessons learnt while benchmarking the alias tracking analysis -------------------------------------------------------------

    Profiling

    • Use YourKit for finding hotspots (cpu profiling). when it comes to drilling down into the details of a hotspot, don't pay too much attention to the percentages / time counts.
    • Should also try other profilers.
    • Use timers. When a method showed up as a hotspot, i added a timer around that method, and a second one within the method to measure specific parts. The timers slow things down, but the relative numbers show what parts of a method are slow.

    ASM analyzer insights

    • The time for running an analysis depends on the number of locals and the number of instructions. Reducing the number of locals helps speeding up the analysis: there are less values to merge when merging to frames. See also https://github.com/scala/scala-dev/issues/47
    • The common hot spot of an ASM analysis is Frame.merge, for example in producers / consumers.
    • For nullness analysis the time is spent as follows
      • 20% merging nullness values. this is as expected: for example, the same absolute amount of time is spent in merging BasicValues when running a BasicInterpreter.
      • 50% merging alias sets. i tried to optimize what i could out of this.
      • 20% is spent creating new frames from existing ones, see comment on AliasingFrame.init.
    • The implementation of Frame.merge (the main hot spot) contains a megamorphic callsite to interpreter.merge. This can be observed easily by running a test program that either runs a BasicValue analysis only, versus a program that first runs a nullness analysis and then a BasicValue. In an example, the time for the BasicValue analysis goes from 519ms to 1963ms, a 3.8x slowdown.
    • I added counters to the Frame.merge methods for nullness and BasicValue analysis. In the examples I benchmarked, the number of merge invocations was always exactly the same. It would probably be possible to come up with an example where alias set merging forces additional analysis rounds until reaching the fixpoint, but I did not observe such cases.

    To benchmark an analysis, instead of benchmarking analysis while it runs in the compiler backend, one can easily run it from a separate program (or the repl). The bytecode to analyze can simply be parsed from a classfile. See example at the end of this comment.

    Nullness Analysis in Miguel's Optimizer ---------------------------------------

    Miguel implemented alias tracking for nullness analysis differently [1]. Remember that every frame has an array of values. Miguel's idea was to represent aliasing using reference equality in the values array: if two entries in the array point to the same value object, the two entries are aliases in the frame of the given instruction.

    While this idea seems elegant at first sight, Miguel's implementation does not merge frames correctly when it comes to aliasing. Assume in frame 1, values (a, b, c) are aliases, while in frame 2 (a, b) are aliases. When merging the second into the first, we have to make sure that c is removed as an alias of (a, b).

    It would be possible to implement correct alias set merging in Miguel's approach. However, frame merging is the main hot spot of analysis. The computational complexity of implementing alias set merging by traversing the values array and comparing references is too high. The concrete alias set representation that is used in the current implementation (see class AliasingFrame) makes alias set merging more efficient.

    [1] https://github.com/scala-opt/scala/blob/opt/rebase/src/compiler/scala/tools/nsc/backend/bcode/NullnessPropagator.java

    Complexity and scaling of analysis ----------------------------------

    The time complexity of a data flow analysis depends on:

    • The size of the method. The complexity factor is linear (assuming the number of locals and branching instructions remains constant). The main analysis loop runs through all instructions of a method once. Instructions are only re-enqueued if a control flow merge changes the frame at some instruction.
    • The branching instructions. When a second (third, ..) control flow edge arrives at an instruction, the existing frame at the instruction is merged with the one computed on the new branch. If the merge function changes the existing frame, the instruction is enqueued for another analysis. This results in a merge operation for the successors of the instruction.
    • The number of local variables. The hot spot of analysis is frame merging. The merge function iterates through the values in the frame (locals and stack values) and merges them.

    I measured the running time of an analysis for two examples:

    • Keep the number of locals and branching instructions constant, increase the number of instructions. The running time grows linearly with the method size.
    • Increase the size and number of locals in a method. The method size and number of locals grow in the same pace. Here, the running time increase is polynomial. It looks like the complexity is be #instructions * #locals^2 (see below).

    I measured nullness analysis (which tracks aliases) and a SimpleValue analysis. Nullness runs roughly 5x slower (because of alias tracking) at every problem size - this factor doesn't change.

    The numbers below are for nullness. Note that the last column is constant, i.e., the running time is proportional to #ins * #loc^2. Therefore we use this factor when limiting the maximal method size for running an analysis.

    #insns #locals time (ms) time / #ins * #loc2 * 106 1305 156 34 1.07 2610 311 165 0.65 3915 466 490 0.57 5220 621 1200 0.59 6525 776 2220 0.56 7830 931 3830 0.56 9135 1086 6570 0.60 10440 1241 9700 0.60 11745 1396 13800 0.60

    As a second experiment, nullness analysis was run with varying #insns but constant #locals. The last column shows linear complexity with respect to the method size (linearOffset = 2279):

    #insns #locals time (ms) (time + linearOffset) / #insns 5220 621 1090 0.645 6224 621 1690 0.637 7226 621 2280 0.630 8228 621 2870 0.625 9230 621 3530 0.629 10232 621 4130 0.626 11234 621 4770 0.627 12236 621 5520 0.637 13238 621 6170 0.638

    When running a BasicValue analysis, the complexity observation is the same (time is proportional to #ins * #loc^2).

    Measuring analysis execution time ---------------------------------

    See code below.

    Definition Classes
    jvm
  • AliasSet
  • AliasingAnalyzer
  • AliasingFrame
  • BackendUtils
  • ExceptionProducer
  • InitialProducer
  • InitialProducerSourceInterpreter
  • IntIterator
  • NonLubbingTypeFlowInterpreter
  • NullnessAnalyzer
  • NullnessFrame
  • NullnessInterpreter
  • NullnessValue
  • ParameterProducer
  • ProdConsAnalyzerImpl
  • TypeFlowInterpreter
  • UninitializedLocalProducer

class NullnessFrame extends AliasingFrame[NullnessValue]

Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. NullnessFrame
  2. AliasingFrame
  3. Frame
  4. AnyRef
  5. Any
Implicitly
  1. by any2stringadd
  2. by StringFormat
  3. by Ensuring
  4. by ArrowAssoc
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new NullnessFrame(src: Frame[_ <: NullnessValue])
  2. new NullnessFrame(nLocals: Int, nStack: Int)

Value Members

  1. final def !=(arg0: Any): Boolean

    Test two objects for inequality.

    Test two objects for inequality.

    returns

    true if !(this == that), false otherwise.

    Definition Classes
    AnyRef → Any
  2. final def ##(): Int

    Equivalent to x.hashCode except for boxed numeric types and null.

    Equivalent to x.hashCode except for boxed numeric types and null. For numerics, it returns a hash value which is consistent with value equality: if two value type instances compare as true, then ## will produce the same hash value for each of them. For null returns a hashcode where null.hashCode throws a NullPointerException.

    returns

    a hash value consistent with ==

    Definition Classes
    AnyRef → Any
  3. def +(other: String): String
    Implicit
    This member is added by an implicit conversion from NullnessFrame to any2stringadd[NullnessFrame] performed by method any2stringadd in scala.Predef.
    Definition Classes
    any2stringadd
  4. def ->[B](y: B): (NullnessFrame, B)
    Implicit
    This member is added by an implicit conversion from NullnessFrame to ArrowAssoc[NullnessFrame] performed by method ArrowAssoc in scala.Predef.
    Definition Classes
    ArrowAssoc
    Annotations
    @inline()
  5. final def ==(arg0: Any): Boolean

    The expression x == that is equivalent to if (x eq null) that eq null else x.equals(that).

    The expression x == that is equivalent to if (x eq null) that eq null else x.equals(that).

    returns

    true if the receiver object is equivalent to the argument; false otherwise.

    Definition Classes
    AnyRef → Any
  6. val aliases: Array[AliasSet]

    For every value the set of values that are aliases of it.

    For every value the set of values that are aliases of it.

    Invariants:

    • If aliases(i) == null then i has no aliases. This is equivalent to having aliases(i) == SingletonSet(i).
    • If aliases(i) != null then aliases(i) contains i.
    • If aliases(i) contains j then aliases(i) eq aliases(j), i.e., they are references to the same (mutable) AliasSet.
    Definition Classes
    AliasingFrame
  7. def aliasesOf(entry: Int): AliasSet

    The set of aliased values for a given entry in the values array.

    The set of aliased values for a given entry in the values array.

    Definition Classes
    AliasingFrame
  8. final def asInstanceOf[T0]: T0

    Cast the receiver object to be of type T0.

    Cast the receiver object to be of type T0.

    Note that the success of a cast at runtime is modulo Scala's erasure semantics. Therefore the expression 1.asInstanceOf[String] will throw a ClassCastException at runtime, while the expression List(1).asInstanceOf[List[String]] will not. In the latter example, because the type argument is erased as part of compilation it is not possible to check whether the contents of the list are of the requested type.

    returns

    the receiver object.

    Definition Classes
    Any
    Exceptions thrown

    ClassCastException if the receiver object is not an instance of the erasure of type T0.

  9. def clearStack(): Unit

    When entering an exception handler, all values are dropped from the stack (and the exception value is pushed).

    When entering an exception handler, all values are dropped from the stack (and the exception value is pushed). The ASM analyzer invokes firstHandlerInstructionFrame.clearStack(). To ensure consistent aliasing sets, we need to remove the dropped values from aliasing sets.

    Definition Classes
    AliasingFrame → Frame
  10. def clone(): AnyRef

    Create a copy of the receiver object.

    Create a copy of the receiver object.

    The default implementation of the clone method is platform dependent.

    returns

    a copy of the receiver object.

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
    Note

    not specified by SLS as a member of AnyRef

  11. def ensuring(cond: (NullnessFrame) ⇒ Boolean, msg: ⇒ Any): NullnessFrame
    Implicit
    This member is added by an implicit conversion from NullnessFrame to Ensuring[NullnessFrame] performed by method Ensuring in scala.Predef.
    Definition Classes
    Ensuring
  12. def ensuring(cond: (NullnessFrame) ⇒ Boolean): NullnessFrame
    Implicit
    This member is added by an implicit conversion from NullnessFrame to Ensuring[NullnessFrame] performed by method Ensuring in scala.Predef.
    Definition Classes
    Ensuring
  13. def ensuring(cond: Boolean, msg: ⇒ Any): NullnessFrame
    Implicit
    This member is added by an implicit conversion from NullnessFrame to Ensuring[NullnessFrame] performed by method Ensuring in scala.Predef.
    Definition Classes
    Ensuring
  14. def ensuring(cond: Boolean): NullnessFrame
    Implicit
    This member is added by an implicit conversion from NullnessFrame to Ensuring[NullnessFrame] performed by method Ensuring in scala.Predef.
    Definition Classes
    Ensuring
  15. final def eq(arg0: AnyRef): Boolean

    Tests whether the argument (that) is a reference to the receiver object (this).

    Tests whether the argument (that) is a reference to the receiver object (this).

    The eq method implements an equivalence relation on non-null instances of AnyRef, and has three additional properties:

    • It is consistent: for any non-null instances x and y of type AnyRef, multiple invocations of x.eq(y) consistently returns true or consistently returns false.
    • For any non-null instance x of type AnyRef, x.eq(null) and null.eq(x) returns false.
    • null.eq(null) returns true.

    When overriding the equals or hashCode methods, it is important to ensure that their behavior is consistent with reference equality. Therefore, if two objects are references to each other (o1 eq o2), they should be equal to each other (o1 == o2) and they should hash to the same value (o1.hashCode == o2.hashCode).

    returns

    true if the argument is a reference to the receiver object; false otherwise.

    Definition Classes
    AnyRef
  16. def equals(arg0: Any): Boolean

    The equality method for reference types.

    The equality method for reference types. Default implementation delegates to eq.

    See also equals in scala.Any.

    returns

    true if the receiver object is equivalent to the argument; false otherwise.

    Definition Classes
    AnyRef → Any
  17. def execute(insn: AbstractInsnNode, interpreter: Interpreter[NullnessValue]): Unit
    Definition Classes
    NullnessFrame → AliasingFrame → Frame
  18. def finalize(): Unit

    Called by the garbage collector on the receiver object when there are no more references to the object.

    Called by the garbage collector on the receiver object when there are no more references to the object.

    The details of when and if the finalize method is invoked, as well as the interaction between finalize and non-local returns and exceptions, are all platform dependent.

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
    Note

    not specified by SLS as a member of AnyRef

  19. def formatted(fmtstr: String): String
    Implicit
    This member is added by an implicit conversion from NullnessFrame to StringFormat[NullnessFrame] performed by method StringFormat in scala.Predef.
    Definition Classes
    StringFormat
    Annotations
    @inline()
  20. final def getClass(): Class[_]

    Returns the runtime class representation of the object.

    Returns the runtime class representation of the object.

    returns

    a class object corresponding to the runtime type of the receiver.

    Definition Classes
    AnyRef → Any
  21. def getLocal(arg0: Int): NullnessValue
    Definition Classes
    Frame
    Annotations
    @throws( ... )
  22. def getLocals(): Int
    Definition Classes
    Frame
  23. def getMaxStackSize(): Int
    Definition Classes
    Frame
  24. def getStack(arg0: Int): NullnessValue
    Definition Classes
    Frame
    Annotations
    @throws( ... )
  25. def getStackSize(): Int
    Definition Classes
    Frame
  26. def hashCode(): Int

    The hashCode method for reference types.

    The hashCode method for reference types. See hashCode in scala.Any.

    returns

    the hash code value for this object.

    Definition Classes
    AnyRef → Any
  27. def init(src: Frame[_ <: NullnessValue]): Frame[NullnessValue]
    Definition Classes
    AliasingFrame → Frame
  28. final def isInstanceOf[T0]: Boolean

    Test whether the dynamic type of the receiver object is T0.

    Test whether the dynamic type of the receiver object is T0.

    Note that the result of the test is modulo Scala's erasure semantics. Therefore the expression 1.isInstanceOf[String] will return false, while the expression List(1).isInstanceOf[List[String]] will return true. In the latter example, because the type argument is erased as part of compilation it is not possible to check whether the contents of the list are of the specified type.

    returns

    true if the receiver object is an instance of erasure of type T0; false otherwise.

    Definition Classes
    Any
  29. def merge(other: Frame[_ <: NullnessValue], interpreter: Interpreter[NullnessValue]): Boolean

    Merge the AliasingFrame other into this AliasingFrame.

    Merge the AliasingFrame other into this AliasingFrame.

    Aliases that are common in both frames are kept. Example:

    var x, y = null if (...) { x = a y = a // (x, y, a) are aliases } else { x = a y = b // (x, a) and (y, b) } [...] // (x, a) -- merge of ((x, y, a)) and ((x, a), (y, b))

    Definition Classes
    AliasingFrame → Frame
  30. def merge(arg0: Frame[_ <: NullnessValue], arg1: Array[Boolean]): Boolean
    Definition Classes
    Frame
  31. final def ne(arg0: AnyRef): Boolean

    Equivalent to !(this eq that).

    Equivalent to !(this eq that).

    returns

    true if the argument is not a reference to the receiver object; false otherwise.

    Definition Classes
    AnyRef
  32. final def notify(): Unit

    Wakes up a single thread that is waiting on the receiver object's monitor.

    Wakes up a single thread that is waiting on the receiver object's monitor.

    Definition Classes
    AnyRef
    Note

    not specified by SLS as a member of AnyRef

  33. final def notifyAll(): Unit

    Wakes up all threads that are waiting on the receiver object's monitor.

    Wakes up all threads that are waiting on the receiver object's monitor.

    Definition Classes
    AnyRef
    Note

    not specified by SLS as a member of AnyRef

  34. def pop(): NullnessValue
    Definition Classes
    Frame
    Annotations
    @throws( ... )
  35. def push(arg0: NullnessValue): Unit
    Definition Classes
    Frame
    Annotations
    @throws( ... )
  36. def setLocal(arg0: Int, arg1: NullnessValue): Unit
    Definition Classes
    Frame
    Annotations
    @throws( ... )
  37. def setReturn(arg0: NullnessValue): Unit
    Definition Classes
    Frame
  38. def setStack(arg0: Int, arg1: NullnessValue): Unit
    Definition Classes
    Frame
    Annotations
    @throws( ... )
  39. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  40. def toString(): String

    Creates a String representation of this object.

    Creates a String representation of this object. The default representation is platform dependent. On the java platform it is the concatenation of the class name, "@", and the object's hashcode in hexadecimal.

    returns

    a String representation of the object.

    Definition Classes
    AliasingFrame → Frame → AnyRef → Any
  41. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  42. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  43. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  44. def →[B](y: B): (NullnessFrame, B)
    Implicit
    This member is added by an implicit conversion from NullnessFrame to ArrowAssoc[NullnessFrame] performed by method ArrowAssoc in scala.Predef.
    Definition Classes
    ArrowAssoc

Inherited from AliasingFrame[NullnessValue]

Inherited from Frame[NullnessValue]

Inherited from AnyRef

Inherited from Any

Inherited by implicit conversion any2stringadd from NullnessFrame to any2stringadd[NullnessFrame]

Inherited by implicit conversion StringFormat from NullnessFrame to StringFormat[NullnessFrame]

Inherited by implicit conversion Ensuring from NullnessFrame to Ensuring[NullnessFrame]

Inherited by implicit conversion ArrowAssoc from NullnessFrame to ArrowAssoc[NullnessFrame]

Ungrouped