train

Type Members

1. class AEType extends ManipulationType[ScalarMatrix, Null]

   

   Input Operation : Vector as Input & Auto Encoder Training (no output type)

   Input Operation : Vector as Input & Auto Encoder Training (no output type)

   Example:

   1. var make = new AEType(error = CrossEntropyErr)
                            var corruptedIn = make corrupted in
                            var out = make onewayTrip (net, corruptedIn)

2. trait Corruption extends (ScalarMatrix) ⇒ ScalarMatrix with Serializable

   

   Type of Corruption

3. case class DistBeliefCriteria(miniBatch: Int = 100, negSamplingRatio: Int = 0, submitInterval: Duration = 1.minute, updateStep: Int = 2, fetchStep: Int = 10, numCores: Int = 1, storageLevel: StorageLevel = StorageLevel.DISK_ONLY_2) extends TrainingCriteria with Product with Serializable

   

   Criteria: How to train (for DistBeliefTrainStyle)

   Criteria: How to train (for DistBeliefTrainStyle)

   This case class defines how to train the network. Training parameter is defined in this class.

   miniBatch

   size of mini-batch (default 100)

   negSamplingRatio

   ratio of negative samples per positive instance. (default 0)

   submitInterval

   Time interval between batch submission. (default 1.minute)

   updateStep

   number of mini-batches between update (default 2)

   fetchStep

   number of mini-batches between fetching (default 10)

   numCores

   number of v-cores in the spark cluster. (default 1)

   storageLevel

   StorageLevel that will be used in Spark. (default DISK_ONLY_2)

   Note

   We recommend set numCores as similar as possible with allocated spark v-cores.

4. class DistBeliefTrainStyle[IN, OUT] extends MultiThreadTrainStyle[IN, OUT]

   

   Train Style : Semi-DistBelief Style, Spark-based.

   Train Style : Semi-DistBelief Style, Spark-based.

   Note

   Unlike with DistBelief, this trainer do updates and fetch by master not the workers.

5. case class DroppingCorruption(presence: Float = 0.95f) extends Corruption with Product with Serializable

   

   Input Corruption: Drop input as zero.

   Input Corruption: Drop input as zero.

   If network uses drop-out training, we recommend that you do not use this.

   presence

   probability of not-dropped. (default 95% = 0.95)

   Example:

   1. var corrupt = DroppingCorruption(presence = 0.99)
             var corrupted = corrupt(vector)

   Note

   If the presence probability is P%, then this corruption leaves P% entries of the matrix

6. case class GaussianCorruption(mean: Double = 0.0, variance: Double = 0.1) extends Corruption with Product with Serializable

   

   Input Corruption: Gaussian

   Input Corruption: Gaussian

   mean

   Mean of noise (default 0.0)

   variance

   Variance of noise (default 0.1)

   Example:

   1. var corrupt = GaussianCorruption(variance = 0.1)
            var corrupted = corrupt(vector)

7. trait ManipulationType[IN, OUT] extends Serializable

   

   Trait that describes how to convert input into corrupted matrix

   Trait that describes how to convert input into corrupted matrix

   Input operation corrupts the given input, and apply network propagations onto matrix representation of input

   IN

   the type of input

   OUT

   the type of output

8. class MultiThreadTrainStyle[IN, OUT] extends TrainStyle[IN, OUT]

   

   Trainer : Stochastic-Style, Multi-Threaded using Spark.

   Trainer : Stochastic-Style, Multi-Threaded using Spark.

   Note

   This is not a implementation using DistBelief Paper. This is between DistBeliefTrainStyle(DBTS) and SingleThreadTrainStyle(STTS). The major difference is whether "updating" is asynchronous(DBTS) or not(MTTS).

9. class RAEType extends TreeType

   

   Input Operation : VectorTree as Input & Recursive Auto-Encoder Training (no output type)

   Input Operation : VectorTree as Input & Recursive Auto-Encoder Training (no output type)

   Example:

   1. var make = new RAEType(error = CrossEntropyErr)
                 var corruptedIn = make corrupted in
                 var out = make onewayTrip (net, corruptedIn)

   Note

   This implementation designed as a replica of the traditional RAE in this paper

   ,

   We recommend that you should not apply this method to non-AutoEncoder tasks

10. class RandomEqualPartitioner extends Partitioner

    

    Spark Partitioner that gives almost-equal partitions.

    Spark Partitioner that gives almost-equal partitions.

    Note

    Use this with RDD.zipWithUniqueId()

11. case class SimpleTrainingCriteria(miniBatch: Int = 100, negSamplingRatio: Int = 0) extends TrainingCriteria with Product with Serializable

    

    Criteria: How to train (for SingleThreadTrainStyle)

    Criteria: How to train (for SingleThreadTrainStyle)

    This case class defines how to train the network. Training parameter is defined in this class.

    miniBatch

    size of mini-batch (default 100)

    negSamplingRatio

    ratio of negative samples per positive instance. (default 0)

12. class SingleThreadTrainStyle[IN, OUT] extends TrainStyle[IN, OUT]

    

    Trainer : Stochastic-Style, Single-Threaded

13. class StandardRAEType extends TreeType

    

    Input Operation : VectorTree as Input & Recursive Auto-Encoder Training (no output type)

    Input Operation : VectorTree as Input & Recursive Auto-Encoder Training (no output type)

    Example:

    1. var make = new RAEType(error = CrossEntropyErr)
                   var corruptedIn = make corrupted in
                   var out = make onewayTrip (net, corruptedIn)

    Note

    This implementation designed as a replica of the standard RAE (RAE + normalization) in this paper

    ,

    We recommend that you should not apply this method to non-AutoEncoder tasks

14. case class StoppingCriteria(maxIter: Int = 100000, patience: Int = 5000, patienceStep: Int = 2, improveThreshold: Float = 0.995f, lossThreshold: Float = 0.0001f, validationFreq: Int = 100) extends Serializable with Product

    

    Criteria: When to stop training

    Criteria: When to stop training

    This case class defines when to stop training. Training stops if one of the following condition is satisfied.

    • #Iteration ≥ maxIter
      • #Iteration ≥ current patience value, which is calculated by max(patience, bestIteration * patienceStep)
      • Amount of loss < lossThreshold

    Validation is done for each validationFreq iterations, and whenever current/best loss ratio below improveThreshold, that iteration is marked as best iteration.

    maxIter

    maximum mini-batch iteration count (default 100,000)

    patience

    default patience count (default 5,000)

    patienceStep

    multiplier for calculating patience (default x2)

    improveThreshold

    threshold that iteration is marked as "improved" (default 95% = 0.95)

    lossThreshold

    maximum-tolerant loss value. (default 0.0001)

    validationFreq

    step count for validation (default 100)

15. trait TrainStyle[IN, OUT] extends Serializable

    

    Trait that describes style of training

    Trait that describes style of training

    This trait controls how to train, i.e. Single-threaded or Distributed.

    IN

    the type of input

    OUT

    the type of output

16. class Trainer[IN, OUT] extends Serializable

    

    General Trainer Implementation.

    General Trainer Implementation.

    This class trains with help of Training Style and Input Operation.

    IN

    the type of input. Currently, ScalarMatrix and DAG are supported

    OUT

    the type of output Currently, ScalarMatrix and Null are supported

    Example:

    1. val net:Network = ...
       
                // Define Manipulation Type. VectorType, AEType, RAEType and URAEType.
                val operation = new VectorType(
                   corrupt = GaussianCorruption(variance = 0.1)
                )
       
               // Define Training Style. SingleThreadTrainStyle vs DistBeliefTrainStyle
                val style = new SingleThreadTrainStyle(
                  net = net,
                  algorithm = new StochasticGradientDescent(l2decay = 0.0001),
                   make = operation,
                  param = SimpleTrainingCriteria(miniBatch = 8))
       
               // Define Trainer
               val train = new Trainer(
                  style = style,
                  stops = StoppingCriteria(maxIter = 100000))
       
               // Do Train
               train.train(set, valid)

    Note

    To train an autoencoder, you can provide same training set as validation set.

    ,

    This trainer is generalized class. Further implementation, you should see several styles.

17. trait TrainingCriteria extends Serializable

    

    Trait that describes Training Criteria

18. trait TreeType extends ManipulationType[BinaryTree, Null]

    

    Trait of Input Operation : VectorTree as Input.

    Trait of Input Operation : VectorTree as Input. This is an Abstract Implementation

19. class URAEType extends TreeType

    

    Input Operation : VectorTree as Input & Unfolding Recursive Auto Encoder Training (no output type)

    Input Operation : VectorTree as Input & Unfolding Recursive Auto Encoder Training (no output type)

    ::Experimental::

    Annotations

    @Experimental()

    Example:

    1. var make = new URAEType(error = CrossEntropyErr)
                  var corruptedIn = make corrupted in
                  var out = make onewayTrip (net, corruptedIn)

    Note

    This is designed for Unfolding RAE, in this paper

    ,

    This cannot be applied into non-AutoEncoder tasks

20. class VectorType extends ManipulationType[ScalarMatrix, ScalarMatrix]

    

    Input Operation : Vector as Input and output

    Input Operation : Vector as Input and output

    Example:

    1. var make = new VectorType(error = CrossEntropyErr)
                  var corruptedIn = make corrupted in
                  var out = make onewayTrip (net, corruptedIn)