Class/Object

org.platanios.tensorflow.api.learn.hooks

Evaluator

Related Docs: object Evaluator | package hooks

Permalink

class Evaluator[IT, IO, ID, IS, I, TT, TO, TD, TS, EI] extends TriggeredHook with ModelDependentHook[IT, IO, ID, IS, I, TT, TO, TD, TS, EI]

Hooks that can be used to evaluate the performance of an estimator for a separate dataset, while training. This hook creates a new session whenever invoked that loads the latest saved checkpoint and evaluates performance using the provided set of evaluation metrics.

Linear Supertypes
ModelDependentHook[IT, IO, ID, IS, I, TT, TO, TD, TS, EI], TriggeredHook, Hook, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. Evaluator
  2. ModelDependentHook
  3. TriggeredHook
  4. Hook
  5. AnyRef
  6. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new Evaluator(log: Boolean = true, summaryDir: Path = null, datasets: Seq[(String, () ⇒ Dataset[TT, TO, TD, TS])], metrics: Seq[Metric[EI, ops.Output]], trigger: HookTrigger = StepHookTrigger(100), triggerAtEnd: Boolean = true, numDecimalPoints: Int = 4, randomSeed: Option[Int] = None, name: String = "Evaluator")

    Permalink

    log

    If true, the step rate is logged using the current logging configuration.

    summaryDir

    If provided, summaries for the step rate will be saved in this directory. This is useful for visualization using TensorBoard, for example.

    datasets

    Datasets over which to evaluate and which produce elements of the same type as the train dataset elements.

    metrics

    Evaluation metrics to use.

    trigger

    Hook trigger specifying when this hook is triggered (i.e., when it executes). If you only want to trigger this hook at the end of a run and not during, then you should set trigger to NoHookTrigger and triggerAtEnd to true.

    triggerAtEnd

    If true, the hook will be triggered at the end of the run. Note that if this flag is set to true, then the global step must be computable without using a feed map for the Session.run() call (which should always be the case by default).

    numDecimalPoints

    Number of decimal points to use when logging floating point values.

    randomSeed

    Random number generator seed to use.

    name

    Name to use for the evaluation hook when logging and saving metric values. This must follow the same formatting guidelines as the name scopes used when constructing graphs.

    Attributes
    protected

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  4. def afterSessionCreation(session: core.client.Session): Unit

    Permalink

    Called after a new session is created.

    Called after a new session is created. This is called to signal the hooks that a new session has been created. This callback has two essential differences with the situation in which begin() is called:

    • When this is called, the graph is finalized and ops can no longer be added to it.
    • This method will also be called as a result of recovering a wrapped session (i.e., not only at the beginning of the overall session).
    session

    The session that has been created.

    Attributes
    protected
    Definition Classes
    Hook
  5. final def afterSessionRun[F, E, R](runContext: SessionRunContext[F, E, R], runResult: SessionRunResult[Seq[ops.Output], Seq[tensors.Tensor[types.DataType]]])(implicit executableEv: Executable[E], fetchableEv: Aux[F, R]): Unit

    Permalink

    Called after each call to Session.run().

    Called after each call to Session.run().

    The runContext argument is the same one passed to beforeSessionRun(). runContext.requestStop() can be called to stop the iteration.

    The runResult argument contains fetched values for the tensors requested by beforeSessionRun().

    If Session.run() throws any exception, then afterSessionRun() will not be called. Note the difference between the end() and the afterSessionRun() behavior when Session.run() throws an OutOfRangeException. In that case, end() is called but afterSessionRun() is not called.

    runContext

    Provides information about the run call (i.e., the originally requested ops/tensors, the session, etc.). Same value as that passed to beforeSessionRun.

    runResult

    Result of the Session.run() call that includes the fetched values for the tensors requested by beforeSessionRun().

    Attributes
    protected
    Definition Classes
    TriggeredHookHook
  6. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any
  7. final def beforeSessionRun[F, E, R](runContext: SessionRunContext[F, E, R])(implicit executableEv: Executable[E], fetchableEv: Aux[F, R]): Option[SessionRunArgs[Seq[ops.Output], Traversable[ops.Op], Seq[tensors.Tensor[types.DataType]]]]

    Permalink

    Called before each call to Session.run().

    Called before each call to Session.run(). You can return from this call a Hook.SessionRunArgs object indicating ops or tensors to add to the upcoming run call. These ops/tensors will be run together with the ops/tensors originally passed to the original run call. The run arguments you return can also contain feeds to be added to the run call.

    The runContext argument is a Hook.SessionRunContext that provides information about the upcoming run call (i.e., the originally requested ops/tensors, the session, etc.).

    At this point the graph is finalized and you should not add any new ops.

    runContext

    Provides information about the upcoming run call (i.e., the originally requested ops/tensors, the session, etc.).

    Attributes
    protected
    Definition Classes
    TriggeredHookHook
  8. def begin(): Unit

    Permalink

    Called once before creating the session.

    Called once before creating the session. When called, the default graph is the one that will be launched in the session. The hook can modify the graph by adding new operations to it. After the begin call the graph will be finalized and the other callbacks will not be able to modify the graph anymore. A second begin call on the same graph, should not change that graph.

    Attributes
    protected
    Definition Classes
    EvaluatorHook
  9. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  10. val datasets: Seq[(String, () ⇒ Dataset[TT, TO, TD, TS])]

    Permalink

    Datasets over which to evaluate and which produce elements of the same type as the train dataset elements.

  11. def end(session: core.client.Session): Unit

    Permalink

    Called at the end of the session usage (i.e., Session.run() will not be invoked again after this call).

    Called at the end of the session usage (i.e., Session.run() will not be invoked again after this call).

    The session argument can be used in case the hook wants to execute any final ops, such as saving a last checkpoint.

    If Session.run() throws any exception other than OutOfRangeException then end() will not be called. Note the difference between the end() and the afterSessionRun() behavior when Session.run() throws an OutOfRangeException. In that case, end() is called but afterSessionRun() is not called.

    session

    Session that will not be used again after this call.

    Attributes
    protected
    Definition Classes
    Hook
  12. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  13. def equals(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  14. def fetches: Seq[ops.Output]

    Permalink
    Attributes
    protected
    Definition Classes
    TriggeredHook
  15. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  16. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any
  17. def hashCode(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  18. val internalTrigger: HookTrigger

    Permalink
    Attributes
    protected
    Definition Classes
    TriggeredHook
  19. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any
  20. val log: Boolean

    Permalink

    If true, the step rate is logged using the current logging configuration.

  21. val metrics: Seq[Metric[EI, ops.Output]]

    Permalink

    Evaluation metrics to use.

  22. var modelInstance: ModelInstance[IT, IO, ID, IS, I, TT, TO, TD, TS, EI]

    Permalink
    Attributes
    protected
    Definition Classes
    ModelDependentHook
  23. val name: String

    Permalink

    Name to use for the evaluation hook when logging and saving metric values.

    Name to use for the evaluation hook when logging and saving metric values. This must follow the same formatting guidelines as the name scopes used when constructing graphs.

  24. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  25. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef
  26. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef
  27. val numDecimalPoints: Int

    Permalink

    Number of decimal points to use when logging floating point values.

  28. def onFirstTrigger[F, E, R](runContext: SessionRunContext[F, E, R])(implicit executableEv: Executable[E], fetchableEv: Aux[F, R]): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    TriggeredHook
  29. def onTrigger(step: Long, elapsed: Option[(Double, Int)], runResult: SessionRunResult[Seq[ops.Output], Seq[tensors.Tensor[types.DataType]]], session: core.client.Session): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    EvaluatorTriggeredHook
  30. val randomSeed: Option[Int]

    Permalink

    Random number generator seed to use.

  31. def runOptions: Option[RunOptions]

    Permalink
    Attributes
    protected
    Definition Classes
    TriggeredHook
  32. val summaryDir: Path

    Permalink

    If provided, summaries for the step rate will be saved in this directory.

    If provided, summaries for the step rate will be saved in this directory. This is useful for visualization using TensorBoard, for example.

  33. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef
  34. def toString(): String

    Permalink
    Definition Classes
    AnyRef → Any
  35. val trigger: HookTrigger

    Permalink

    Hook trigger specifying when this hook is triggered (i.e., when it executes).

    Hook trigger specifying when this hook is triggered (i.e., when it executes). If you only want to trigger this hook at the end of a run and not during, then you should set trigger to NoHookTrigger and triggerAtEnd to true.

  36. val triggerAtEnd: Boolean

    Permalink

    If true, the hook will be triggered at the end of the run.

    If true, the hook will be triggered at the end of the run. Note that if this flag is set to true, then the global step must be computable without using a feed map for the Session.run() call (which should always be the case by default).

  37. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  38. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  39. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  40. def wantMetadata: Boolean

    Permalink
    Attributes
    protected
    Definition Classes
    TriggeredHook

Inherited from ModelDependentHook[IT, IO, ID, IS, I, TT, TO, TD, TS, EI]

Inherited from TriggeredHook

Inherited from Hook

Inherited from AnyRef

Inherited from Any

Ungrouped