Class ExperimentRunner<T extends Experiment>
- Type Parameters:
T
- the type ofExperiment
the runner will execute
- All Implemented Interfaces:
java.lang.Runnable
public abstract class ExperimentRunner<T extends Experiment>
extends java.lang.Object
implements java.lang.Runnable
- Since:
- CloudSim Plus 1.0
- Author:
- Manoel Campos da Silva Filho
-
Constructor Summary
Constructors Constructor Description ExperimentRunner(boolean antitheticVariatesTechnique)
Creates an experiment runner, setting thebase seed
as the current time.ExperimentRunner(boolean antitheticVariatesTechnique, long baseSeed)
Creates an experiment runner with a givenbase seed
. -
Method Summary
Modifier and Type Method Description int
batchSizeCeil()
protected java.util.List<java.lang.Double>
computeAntitheticMeans(java.util.List<java.lang.Double> samples)
Computes the antithetic means for the given samples if the"Antithetic Variates Technique" is to be applied
.protected java.util.List<java.lang.Double>
computeBatchMeans(java.util.List<java.lang.Double> samples)
Gets an list of samples and apply the "Batch Means Method" to reduce samples correlation, if the "Batch Means Method"is to be applied
.protected double
computeConfidenceErrorMargin(org.apache.commons.math3.stat.descriptive.SummaryStatistics stats, double confidenceLevel)
Computes the confidence interval error margin for a given set of samples in order to enable finding the interval lower and upper bound around a mean value.protected org.apache.commons.math3.stat.descriptive.SummaryStatistics
computeFinalStatistics(java.util.List<java.lang.Double> values)
Creates a SummaryStatistics object from a list of Double values, allowing computation of statistics such as mean over these values.protected abstract T
createExperiment(int i)
Creates an experiment to be run for the i'th time.protected abstract java.util.Map<java.lang.String,java.util.List<java.lang.Double>>
createMetricsMap()
Creates a Map adding a List of values for each metric to be computed.ContinuousDistribution
createRandomGen(int experimentIndex)
Creates a pseudo random number generator (PRNG) for a experiment run that generates uniform values between [0 and 1[.ContinuousDistribution
createRandomGen(int experimentIndex, double minInclusive, double maxExclusive)
Creates a pseudo random number generator (PRNG) for a experiment run that generates uniform values between [min and max[.long
getBaseSeed()
Gets the seed to be used for the first executed experiment.long
getExperimentsFinishTime()
Time in seconds the experiments finished.long
getExperimentsStartTime()
Time in seconds the experiments started.int
getNumberOfBatches()
Gets the number of batches in which the simulation runs will be divided.int
getSimulationRuns()
Gets the number of times the experiment will be executed in order to get values such as means and standard deviations.int
halfSimulationRuns()
boolean
isApplyAntitheticVariatesTechnique()
Checks if the "Antithetic Variates Technique" is to be applied to reduce results variance.boolean
isApplyBatchMeansMethod()
Checks if the "Batch Means Method" is to be applied to reduce correlation between the results for different experiment runs.boolean
isToReuseSeedFromFirstHalfOfExperiments(int currentExperimentIndex)
boolean
isVerbose()
Indicates if the runner will output execution logs or not.protected abstract void
printFinalResults(java.lang.String metricName, org.apache.commons.math3.stat.descriptive.SummaryStatistics stats)
Prints final simulation results such as means, standard deviations and confidence intervals.protected abstract void
printSimulationParameters()
void
run()
Setups and starts the execution of all experiments.ExperimentRunner
setBaseSeed(long baseSeed)
ExperimentRunner
setNumberOfBatches(int numberOfBatches)
Sets the number of batches in which the simulation runs will be divided.protected ExperimentRunner
setSimulationRuns(int simulationRuns)
protected ExperimentRunner
setSimulationRunsAsMultipleOfBatchNumber()
Adjusts the current number of simulations to be equal to its closer multiple of the number of batches.protected abstract void
setup()
Setup experiment attributes considering the dependency between each other.ExperimentRunner
setVerbose(boolean verbose)
Defines if the runner will output execution logs or not.boolean
simulationRunsAndNumberOfBatchesAreCompatible()
Checks if the number of simulation runs and the number of batches are compatible
-
Constructor Details
-
ExperimentRunner
public ExperimentRunner(boolean antitheticVariatesTechnique)Creates an experiment runner, setting thebase seed
as the current time.- Parameters:
antitheticVariatesTechnique
- indicates if it's to be applied the antithetic variates technique.
-
ExperimentRunner
public ExperimentRunner(boolean antitheticVariatesTechnique, long baseSeed)Creates an experiment runner with a givenbase seed
.- Parameters:
antitheticVariatesTechnique
- indicates if it's to be applied the antithetic variates technique.baseSeed
- the seed to be used as base for each experiment seed
-
-
Method Details
-
setup
protected abstract void setup()Setup experiment attributes considering the dependency between each other. The method is called by the
run()
method, just after all the attributes were set. By this way, it initializes internal attributes and validates other ones.NOTE: As a good practice, it is tried to reduce the number of parameters for the class constructor, as it tends to increase as the experiment code evolves. Accordingly, all the parameters have to be defined using the corresponding setters. By this way, it has to be avoided setting up attributes inside the constructor, once they can become invalid or out-of-date because dependency between parameters. The constructor has just to initialize objects to avoid
NullPointerException
. This way, one have to set all the parameters inside this method. For instance, if the constructor creates and Random Number Generator (PRNG) using a default seed but the method setSeed is called after the constructor, the PRNG will not be update to use the new seed. -
batchSizeCeil
public int batchSizeCeil()- Returns:
- the batch size rounded by the
Math.ceil(double)
method.
-
simulationRunsAndNumberOfBatchesAreCompatible
public boolean simulationRunsAndNumberOfBatchesAreCompatible()Checks if the number of simulation runs and the number of batches are compatible- Returns:
-
isApplyBatchMeansMethod
public boolean isApplyBatchMeansMethod()Checks if the "Batch Means Method" is to be applied to reduce correlation between the results for different experiment runs.- Returns:
-
computeBatchMeans
protected java.util.List<java.lang.Double> computeBatchMeans(java.util.List<java.lang.Double> samples)Gets an list of samples and apply the "Batch Means Method" to reduce samples correlation, if the "Batch Means Method"is to be applied
.- Parameters:
samples
- the list with samples to apply the "Batch Means Method". Samples size is defined by thegetSimulationRuns()
.- Returns:
- the samples list after applying the "Batch Means Method", in case
the method is enabled to be applied, which will reduce the array to the
number of batches defined by
getNumberOfBatches()
(each value in the returned array will be the mean of every sample batch). Otherwise, returns the same given array
-
computeConfidenceErrorMargin
protected double computeConfidenceErrorMargin(org.apache.commons.math3.stat.descriptive.SummaryStatistics stats, double confidenceLevel)Computes the confidence interval error margin for a given set of samples in order to enable finding the interval lower and upper bound around a mean value. By this way, the confidence interval can be computed as [mean + errorMargin .. mean - errorMargin].
To reduce the confidence interval by half, one have to execute the experiments 4 more times. This is called the "Replication Method" and just works when the samples are i.i.d. (independent and identically distributed). Thus, if you have correlation between samples of each simulation run, a different method such as a bias compensation,
NOTE: How to compute the error margin is a little bit confusing. The Harry Perros' book states that if less than 30 samples are collected, the t-Distribution has to be used to that purpose. However, this article Wikipedia article says that if the standard deviation of the real population is known, it has to be used the z-value from the Standard Normal Distribution. Otherwise, it has to be used the t-value from the t-Distribution to calculate the critical value for defining the error margin (also called standard error). The book "Numeric Computation and Statistical Data Analysis on the Java Platform" confirms the last statement and such approach was followed.batch means
or regenerative method has to be used.- Parameters:
stats
- the statistic object with the values to compute the error margin of the confidence intervalconfidenceLevel
- the confidence level, in the interval from ]0 to 1[, such as 0.95 to indicate 95% of confidence.- Returns:
- the error margin to compute the lower and upper bound of the confidence interval
- See Also:
- Critical Values of the Student's t Distribution, t-Distribution, Harry Perros, "Computer Simulation Techniques: The definitive introduction!," 2009, Numeric Computation and Statistical Data Analysis on the Java Platform
-
isApplyAntitheticVariatesTechnique
public boolean isApplyAntitheticVariatesTechnique()Checks if the "Antithetic Variates Technique" is to be applied to reduce results variance.- Returns:
- See Also:
- Antithetic variates
-
getSimulationRuns
public int getSimulationRuns()Gets the number of times the experiment will be executed in order to get values such as means and standard deviations. It has to be an even number if the"Antithetic Variates Technique"
is to be used.- Returns:
-
setSimulationRuns
-
setSimulationRunsAsMultipleOfBatchNumber
Adjusts the current number of simulations to be equal to its closer multiple of the number of batches.- Returns:
-
getNumberOfBatches
public int getNumberOfBatches()Gets the number of batches in which the simulation runs will be divided. If this number is greater than 1, the "Batch Means Method" is used to reduce the correlation between experiment runs.- Returns:
-
setNumberOfBatches
Sets the number of batches in which the simulation runs will be divided.- Parameters:
numberOfBatches
- number of simulation run batches- Returns:
- See Also:
getNumberOfBatches()
-
getBaseSeed
public long getBaseSeed()Gets the seed to be used for the first executed experiment. The seed for each subsequent experiment is this seed plus the index of the experiment.- Returns:
-
createRandomGen
Creates a pseudo random number generator (PRNG) for a experiment run that generates uniform values between [0 and 1[. If it is to apply the"Antithetic Variates Technique"
to reduce results variance, the second half of experiments will used the seeds from the first half.- Parameters:
experimentIndex
- index of the experiment run to create a PRNG- Returns:
- the created PRNG
- See Also:
UniformDistr.isApplyAntitheticVariates()
,createRandomGen(int, double, double)
-
createRandomGen
public ContinuousDistribution createRandomGen(int experimentIndex, double minInclusive, double maxExclusive)Creates a pseudo random number generator (PRNG) for a experiment run that generates uniform values between [min and max[. If it is to apply the"Antithetic Variates Technique"
to reduce results' variance, the second half of experiments will use the seeds from the first half.- Parameters:
experimentIndex
- index of the experiment run to create a PRNGminInclusive
- the minimum value the generator will return (inclusive)maxExclusive
- the maximum value the generator will return (exclusive)- Returns:
- the created PRNG
- See Also:
UniformDistr.isApplyAntitheticVariates()
,createRandomGen(int)
-
isToReuseSeedFromFirstHalfOfExperiments
public boolean isToReuseSeedFromFirstHalfOfExperiments(int currentExperimentIndex) -
halfSimulationRuns
public int halfSimulationRuns()- Returns:
- the half of
getSimulationRuns()
-
getExperimentsFinishTime
public long getExperimentsFinishTime()Time in seconds the experiments finished.- Returns:
-
getExperimentsStartTime
public long getExperimentsStartTime()Time in seconds the experiments started.- Returns:
-
run
public void run()Setups and starts the execution of all experiments.- Specified by:
run
in interfacejava.lang.Runnable
-
createMetricsMap
protected abstract java.util.Map<java.lang.String,java.util.List<java.lang.Double>> createMetricsMap()Creates a Map adding a List of values for each metric to be computed. The computation of final experiments results are performed on this map.Each key is the name of metric and each value is a List of Double containing the values collected for that metric, for each experiment run. These values will be then summarized to compute the final value for each metric.
- Returns:
- the populated metricsMap
-
createExperiment
Creates an experiment to be run for the i'th time.- Parameters:
i
- a number that identifies the experiment- Returns:
- the created experiment
-
computeAntitheticMeans
protected java.util.List<java.lang.Double> computeAntitheticMeans(java.util.List<java.lang.Double> samples)Computes the antithetic means for the given samples if the
"Antithetic Variates Technique" is to be applied
. These values are the mean between the first half of samples with the second half. By this way, the resulting value is an array with half of the samples length.NOTE: To correctly compute the antithetic values the seeds from the first half of experiments must be used for the second half.
- Parameters:
samples
- the list of samples to compute the antithetic means from- Returns:
- the computed antithetic means from the given samples if the "Antithetic Variates Technique" is to be applied, otherwise return the same given samples list.
- See Also:
createRandomGen(int, double, double)
-
printSimulationParameters
protected abstract void printSimulationParameters() -
computeFinalStatistics
protected org.apache.commons.math3.stat.descriptive.SummaryStatistics computeFinalStatistics(java.util.List<java.lang.Double> values)Creates a SummaryStatistics object from a list of Double values, allowing computation of statistics such as mean over these values. The method also checks if theAntithetic Variates
and theBatch Means
techniques are enabled and then apply them over the given list of Doubles. These techniques are used for variance reduction.- Parameters:
values
- the List of values to add to theSummaryStatistics
object- Returns:
- the
SummaryStatistics
object containing the double values, after applying the the techniques for variance reduction.
-
printFinalResults
protected abstract void printFinalResults(java.lang.String metricName, org.apache.commons.math3.stat.descriptive.SummaryStatistics stats)Prints final simulation results such as means, standard deviations and confidence intervals.- Parameters:
metricName
- the name of the metric to be printedstats
- theSummaryStatistics
containing means of each experiment run that will be used to computed an overall mean and other statistics
-
setBaseSeed
-
isVerbose
public boolean isVerbose()Indicates if the runner will output execution logs or not. This doesn't affect the verbosity of individual experiments executed. EachExperiment
has its own verbose attribute.- Returns:
-
setVerbose
Defines if the runner will output execution logs or not. This doesn't affect the verbosity of individual experiments executed. EachExperiment
has its own verbose attribute.- Parameters:
verbose
- true if results have to be output, false otherwise- Returns:
-