package data
Ordering
- Alphabetic
Visibility
- Public
- All
Type Members
- trait BatchStream[+I] extends AnyRef
- sealed trait LoopState extends AnyRef
- case class NonEmptyBatch[I](batch: I) extends StreamControl[I] with Product with Serializable
- case class Peek(label: String) extends Module with Product with Serializable
- case class SWALoopState(model: Seq[STen], optimizer: Seq[STen], epoch: Int, lastValidationLoss: Option[Double], minValidationLoss: Option[Double], numberOfAveragedModels: Int, averagedModels: Option[Seq[Tensor]], learningCurve: List[(Int, Double, Option[Double])]) extends LoopState with Product with Serializable
- case class SimpleLoopState(model: Seq[STen], optimizer: Seq[STen], epoch: Int, lastValidationLoss: Option[Double], minValidationLoss: Option[Double], minValidationLossModel: Option[(Int, Seq[Tensor])], learningCurve: List[(Int, Double, Option[Double])]) extends LoopState with Product with Serializable
- case class SimpleThenSWALoopState(simple: Option[SimpleLoopState], swa: Option[SWALoopState]) extends LoopState with Product with Serializable
- sealed trait StreamControl[+I] extends AnyRef
- trait TrainingCallback extends AnyRef
- trait ValidationCallback extends AnyRef
Value Members
- object BatchStream
- object BufferedImageHelper
- object DataParallel
- object EmptyBatch extends StreamControl[Nothing] with Product with Serializable
- object EndStream extends StreamControl[Nothing] with Product with Serializable
- object GraphBatchStream
-
object
IOLoops
Contains a training loops and helpers around it
Contains a training loops and helpers around it
The two training loops implemented here are:
- lamp.data.IOLoops.epochs
- lamp.data.IOLoops.withSWA implements Stochastic Weight Averaging
- object Reader
- object SWA
- object StateIO
- object StreamControl
- object Text
- object TrainingCallback
- object ValidationCallback
-
object
Writer
Serializes tensors
Serializes tensors
This format is similar to the ONNX external tensor serialization format, but it uses JSON rather then protobuf.
Format specification
Sequences of tensors are serialized into a JSON descriptor and a data blob. The schema of the descriptor is the case class lamp.data.schemas.TensorList. The location field in this schema holds a path to the data blob. If this the location a relative POSIX then it is relative to the file path where the descriptor itself is written.
The descriptor may be embedded into larger JSON structures.
The data blob itself is the raw data in little endian byte order. Floating point is IEEE-754. The descriptor specifies the byte offset and byte length of the tensors inside the data blob. As such, the data blob contains no framing or other control bytes, but it may contain padding bytes between tensors.