Package org.deeplearning4j.util

Class NetworkUtils

  • public class NetworkUtils
extends Object

    • Method Detail

      • toComputationGraph

        public static ComputationGraph toComputationGraph​(MultiLayerNetwork net)
        Convert a MultiLayerNetwork to a ComputationGraph
        Returns:
        ComputationGraph equivalent to this network (including parameters and updater state)

      • setLearningRate

        public static void setLearningRate​(MultiLayerNetwork net,
                                   double newLr)
        Set the learning rate for all layers in the network to the specified value. Note that if any learning rate schedules are currently present, these will be removed in favor of the new (fixed) learning rate.

        Note: This method not free from a performance point of view: a proper learning rate schedule should be used in preference to calling this method at every iteration.
        Parameters:
        net - Network to set the LR for
        newLr - New learning rate for all layers

      • setLearningRate

        public static void setLearningRate​(MultiLayerNetwork net,
                                   ISchedule newLrSchedule)
        Set the learning rate schedule for all layers in the network to the specified schedule. This schedule will replace any/all existing schedules, and also any fixed learning rate values.
        Note that the iteration/epoch counts will not be reset. Use MultiLayerConfiguration#setIterationCount(int) and MultiLayerConfiguration.setEpochCount(int) if this is required
        Parameters:
        newLrSchedule - New learning rate schedule for all layers

      • setLearningRate

        public static void setLearningRate​(MultiLayerNetwork net,
                                   int layerNumber,
                                   double newLr)
        Set the learning rate for a single layer in the network to the specified value. Note that if any learning rate schedules are currently present, these will be removed in favor of the new (fixed) learning rate.

        Note: This method not free from a performance point of view: a proper learning rate schedule should be used in preference to calling this method at every iteration. Note also that setLearningRate(MultiLayerNetwork, double) should also be used in preference, when all layers need to be set to a new LR
        Parameters:
        layerNumber - Number of the layer to set the LR for
        newLr - New learning rate for a single layers

      • setLearningRate

        public static void setLearningRate​(MultiLayerNetwork net,
                                   int layerNumber,
                                   ISchedule lrSchedule)
        Set the learning rate schedule for a single layer in the network to the specified value.
        Note also that setLearningRate(MultiLayerNetwork, ISchedule) should also be used in preference, when all layers need to be set to a new LR schedule.
        This schedule will replace any/all existing schedules, and also any fixed learning rate values.
        Note also that the iteration/epoch counts will not be reset. Use MultiLayerConfiguration#setIterationCount(int) and MultiLayerConfiguration.setEpochCount(int) if this is required
        Parameters:
        layerNumber - Number of the layer to set the LR schedule for
        lrSchedule - New learning rate for a single layer

      • getLearningRate

        public static Double getLearningRate​(MultiLayerNetwork net,
                                     int layerNumber)
        Get the current learning rate, for the specified layer, fromthe network. Note: If the layer has no learning rate (no parameters, or an updater without a learning rate) then null is returned
        Parameters:
        net - Network
        layerNumber - Layer number to get the learning rate for
        Returns:
        Learning rate for the specified layer, or null

      • setLearningRate

        public static void setLearningRate​(ComputationGraph net,
                                   double newLr)
        Set the learning rate for all layers in the network to the specified value. Note that if any learning rate schedules are currently present, these will be removed in favor of the new (fixed) learning rate.

        Note: This method not free from a performance point of view: a proper learning rate schedule should be used in preference to calling this method at every iteration.
        Parameters:
        net - Network to set the LR for
        newLr - New learning rate for all layers

      • setLearningRate

        public static void setLearningRate​(ComputationGraph net,
                                   ISchedule newLrSchedule)
        Set the learning rate schedule for all layers in the network to the specified schedule. This schedule will replace any/all existing schedules, and also any fixed learning rate values.
        Note that the iteration/epoch counts will not be reset. Use ComputationGraphConfiguration#setIterationCount(int) and ComputationGraphConfiguration#setEpochCount(int) if this is required
        Parameters:
        newLrSchedule - New learning rate schedule for all layers

      • setLearningRate

        public static void setLearningRate​(ComputationGraph net,
                                   String layerName,
                                   double newLr)
        Set the learning rate for a single layer in the network to the specified value. Note that if any learning rate schedules are currently present, these will be removed in favor of the new (fixed) learning rate.

        Note: This method not free from a performance point of view: a proper learning rate schedule should be used in preference to calling this method at every iteration. Note also that setLearningRate(ComputationGraph, double) should also be used in preference, when all layers need to be set to a new LR
        Parameters:
        layerName - Name of the layer to set the LR for
        newLr - New learning rate for a single layers

      • setLearningRate

        public static void setLearningRate​(ComputationGraph net,
                                   String layerName,
                                   ISchedule lrSchedule)
        Set the learning rate schedule for a single layer in the network to the specified value.
        Note also that setLearningRate(ComputationGraph, ISchedule) should also be used in preference, when all layers need to be set to a new LR schedule.
        This schedule will replace any/all existing schedules, and also any fixed learning rate values.
        Note also that the iteration/epoch counts will not be reset. Use ComputationGraphConfiguration#setIterationCount(int) and ComputationGraphConfiguration#setEpochCount(int) if this is required
        Parameters:
        layerName - Name of the layer to set the LR schedule for
        lrSchedule - New learning rate for a single layer

      • getLearningRate

        public static Double getLearningRate​(ComputationGraph net,
                                     String layerName)
        Get the current learning rate, for the specified layer, from the network. Note: If the layer has no learning rate (no parameters, or an updater without a learning rate) then null is returned
        Parameters:
        net - Network
        layerName - Layer name to get the learning rate for
        Returns:
        Learning rate for the specified layer, or null

      • removeInstances

        public static void removeInstances​(List<?> list,
                                   Class<?> remove)
        Remove any instances of the specified type from the list. This includes any subtypes.
        Parameters:
        list - List. May be null
        remove - Type of objects to remove

      • removeInstancesWithWarning

        public static void removeInstancesWithWarning​(List<?> list,
                                              Class<?> remove,
                                              String warning)

      • rebuildUpdaterStateArray

        protected static INDArray rebuildUpdaterStateArray​(INDArray origUpdaterState,
                                                   List<UpdaterBlock> orig,
                                                   List<UpdaterBlock> newUpdater)
        Rebuild the updater state after a learning rate change. With updaters like Adam, they have 2 components... m and v array, for a total updater state size of 2*numParams. Because we combine across parameters and layers where possible (smaller number of larger operations -> more efficient) we can sometimes need to rearrange the updater state array. For example, if the original updater state for Adam is organized like [mParam1, mParam2, vParam1, vParam2] in one block and we change the learning rate for one of the layers, param 1 and param2 now belong to different updater blocks. Consequently, we need to rearrange the updater state to be like [mParam1][vParam1] in block 1, [mParam2][vParam2] in block 2
        Parameters:
        origUpdaterState - Original updater state view array
        orig - Original updater blocks
        newUpdater - New updater blocks
        Returns:
        New state view array