Package org.cloudsimplus.testbeds
Class ExperimentRunner<T extends Experiment>
java.lang.Object
org.cloudsimplus.testbeds.AbstractRunnable
org.cloudsimplus.testbeds.ExperimentRunner<T>
- Type Parameters:
T
- the type ofExperiment
the runner will execute
- All Implemented Interfaces:
Runnable
A base class to run a given experiment a defined number of times and collect
statistics about the execution. The runner represents a testbed compounded of
a set of experiments that it runs.
- Since:
- CloudSim Plus 1.0
- Author:
- Manoel Campos da Silva Filho
-
Field Summary
Fields inherited from class org.cloudsimplus.testbeds.AbstractRunnable
verbose
-
Constructor Summary
ModifierConstructorDescriptionprotected
ExperimentRunner
(long baseSeed, int simulationRuns) Creates an experiment runner with a givenbase seed
that runs sequentially.protected
ExperimentRunner
(long baseSeed, int simulationRuns, boolean applyAntitheticVariates) Creates an experiment runner with a givenbase seed
that runs sequentially.protected
ExperimentRunner
(long baseSeed, int simulationRuns, boolean latexTableResultsGeneration, boolean parallel) Creates an experiment runner with a givenbase seed
that runs sequentially.protected
ExperimentRunner
(long baseSeed, int simulationRuns, int batchesNumber, boolean applyAntitheticVariates) Creates an experiment runner with a givenbase seed
that runs sequentially.protected
ExperimentRunner
(long baseSeed, int simulationRuns, int batchesNumber, boolean applyAntitheticVariates, boolean parallel, boolean latexTableResultsGeneration) Creates an experiment runner with a givenbase seed
. -
Method Summary
Modifier and TypeMethodDescriptionprotected final void
addMetricValue
(String metricName, Double value) Add a value to a given metric inside themetricsMap
.int
computeAntitheticMeans
(List<Double> samples) Computes the antithetic means for the given samples if the"Antithetic Variates Technique" is to be applied
.computeBatchMeans
(List<Double> samples) Gets an list of samples and apply the "Batch Means Method" to reduce samples correlation, if the "Batch Means Method"is to be applied
.protected ConfidenceInterval
computeFinalResults
(Map.Entry<String, List<Double>> metricEntry) Computes final simulation results, including mean, standard deviations and confidence intervals for a given metric computed across all simulation runs.protected final org.apache.commons.math3.stat.descriptive.SummaryStatistics
computeFinalStatistics
(List<Double> values) Creates aSummaryStatistics
object from a list of Double values, allowing computation of statistics such as mean over these values.protected abstract T
createExperimentInternal
(int index) Creates an experiment to be run for the i'th time.createRandomGen
(int experimentIndex) Creates a pseudo random number generator (PRNG) for a experiment run that generates uniform values between [0 and 1[.createRandomGen
(int experimentIndex, double minInclusive, double maxExclusive) Creates a pseudo random number generator (PRNG) for a experiment run that generates uniform values between [min and max[.<S extends StatisticalDistribution>
ScreateRandomGen
(int experimentIndex, Function<Long, S> randomGenCreator) Uses the providedFunction
to create a pseudo random number generator (PRNG) for a experiment run.Enables the generation of a result table in Latex format for computed metrics.long
Gets the seed to be used for the first executed experiment.int
Gets the number of batches in which the simulation runs will be divided.long
Time in seconds the experiments took to finish.long
Time in seconds the experiments started.int
int
getMetricValues
(String metricName) long
getSeed
(int experimentIndex) final int
Gets the number of times the experiment will be executed in order to get values such as means and standard deviations.int
final boolean
Checks if the "Antithetic Variates Technique" is to be applied to reduce results variance.boolean
Checks if the "Batch Means Method" is to be applied to reduce correlation between the results for different experiment runs.boolean
Checks if generation of a result table in Latex format for computed metrics is enabled.boolean
If experiments are executed in parallel, each experiment verbosity is disabled, otherwise, you'll see mixed log messages from different experiment runs.boolean
Checks if a progress bar is to be printed to show when each experiment run finishes.boolean
Checks if the experiment will run a single time or not.boolean
isToReuseSeedFromFirstHalfOfExperiments
(int currentExperimentIndex) boolean
Checks if simulation results of the experiment have to be output.protected abstract void
void
run()
Setups and starts the execution of all experiments sequentially or inparallel
.setDescription
(String description) Sets a description for this experiment which is shown when it starts.void
setFirstExperimentCreated
(int firstExperimentCreated) setProgressBarInNewLine
(boolean progressBarInNewLine) setResultsTableId
(String resultsTableId) An id used to identify the experiment results table generated in formats such as Latex for computed metrics.setShowProgress
(boolean showProgress) Enable or disables a progress bar to show when each experiment run finishes.Methods inherited from class org.cloudsimplus.testbeds.AbstractRunnable
isNotVerbose, print, print, println, println, println, setVerbose
-
Constructor Details
-
ExperimentRunner
protected ExperimentRunner(long baseSeed, int simulationRuns) Creates an experiment runner with a givenbase seed
that runs sequentially.- Parameters:
baseSeed
- the seed to be used as base for each experiment seedsimulationRuns
- the number of times the experiment will be executed
-
ExperimentRunner
protected ExperimentRunner(long baseSeed, int simulationRuns, boolean latexTableResultsGeneration, boolean parallel) Creates an experiment runner with a givenbase seed
that runs sequentially.- Parameters:
baseSeed
- the seed to be used as base for each experiment seedsimulationRuns
- the number of times the experiment will be executedlatexTableResultsGeneration
- Enables/disables the generation of a result table in Latex format for computed metrics.parallel
- whether experiments will run in parallel or sequentially
-
ExperimentRunner
protected ExperimentRunner(long baseSeed, int simulationRuns, boolean applyAntitheticVariates) Creates an experiment runner with a givenbase seed
that runs sequentially.- Parameters:
baseSeed
- the seed to be used as base for each experiment seedsimulationRuns
- the number of times the experiment will be executedapplyAntitheticVariates
- indicates if it's to be applied the antithetic variates technique.
-
ExperimentRunner
protected ExperimentRunner(long baseSeed, int simulationRuns, int batchesNumber, boolean applyAntitheticVariates) Creates an experiment runner with a givenbase seed
that runs sequentially.- Parameters:
baseSeed
- the seed to be used as base for each experiment seedsimulationRuns
- the number of times the experiment will be executedbatchesNumber
- number of simulation run batches (zero disables the batch means method)applyAntitheticVariates
- indicates if it's to be applied the antithetic variates technique.
-
ExperimentRunner
protected ExperimentRunner(long baseSeed, int simulationRuns, int batchesNumber, boolean applyAntitheticVariates, boolean parallel, boolean latexTableResultsGeneration) Creates an experiment runner with a givenbase seed
.- Parameters:
baseSeed
- the seed to be used as base for each experiment seedsimulationRuns
- the number of times the experiment will be executedbatchesNumber
- number of simulation run batches (zero disables the batch means method)applyAntitheticVariates
- indicates if it's to be applied the antithetic variates technique.parallel
- whether experiments will run in parallel or sequentially. It's just actually enabled when the simulation runs is larger than 1.latexTableResultsGeneration
- Enables/disables the generation of a result table in Latex format for computed metrics.
-
-
Method Details
-
batchSizeCeil
public int batchSizeCeil()- Returns:
- the batch size rounded by the
Math.ceil(double)
method.
-
isApplyBatchMeansMethod
public boolean isApplyBatchMeansMethod()Checks if the "Batch Means Method" is to be applied to reduce correlation between the results for different experiment runs. That happens if the number of simulation runs and the number of batches are compatible.- Returns:
-
computeBatchMeans
Gets an list of samples and apply the "Batch Means Method" to reduce samples correlation, if the "Batch Means Method"is to be applied
.- Parameters:
samples
- the list with samples to apply the "Batch Means Method". Samples size is defined by thegetSimulationRuns()
.- Returns:
- the samples list after applying the "Batch Means Method", in case
the method is enabled to be applied, which will reduce the array to the
number of batches defined by
getBatchesNumber()
(each value in the returned array will be the mean of every sample batch). Otherwise, returns the same given array
-
isApplyAntitheticVariates
public final boolean isApplyAntitheticVariates()Checks if the "Antithetic Variates Technique" is to be applied to reduce results variance.- Returns:
- See Also:
-
getSimulationRuns
public final int getSimulationRuns()Gets the number of times the experiment will be executed in order to get values such as means and standard deviations. It has to be an even number if the"Antithetic Variates Technique"
is to be used.- Returns:
-
isSingleRun
public boolean isSingleRun()Checks if the experiment will run a single time or not.- Returns:
- true if the experiment will run a single time, false if there are multiple simulation runs.
-
getBatchesNumber
public int getBatchesNumber()Gets the number of batches in which the simulation runs will be divided. If this number is greater than 1, the "Batch Means Method" is used to reduce the correlation between experiment runs.- Returns:
-
getBaseSeed
public long getBaseSeed()Gets the seed to be used for the first executed experiment. The seed for each subsequent experiment is this seed plus the index of the experiment.- Returns:
-
getSeed
public long getSeed(int experimentIndex) -
createRandomGen
public <S extends StatisticalDistribution> S createRandomGen(int experimentIndex, Function<Long, S> randomGenCreator) Uses the providedFunction
to create a pseudo random number generator (PRNG) for a experiment run. The kind and parameters for this PRNG is defined internally by the given Function. This method calls that Function just providing the seed to be used for the current experiment run. If it is to apply the"Antithetic Variates Technique"
to reduce results variance, the second half of experiments will used the seeds from the first half.- Parameters:
experimentIndex
- index of the experiment run to create a PRNGrandomGenCreator
- aFunction
that receives a seed generated by the runner and returns a new instance of some PRNG- Returns:
- the created PRNG with the seed provided by the runner
- See Also:
-
createRandomGen
Creates a pseudo random number generator (PRNG) for a experiment run that generates uniform values between [0 and 1[. If it is to apply the"Antithetic Variates Technique"
to reduce results variance, the second half of experiments will used the seeds from the first half.- Parameters:
experimentIndex
- index of the experiment run to create a PRNG- Returns:
- the created PRNG
- See Also:
-
createRandomGen
public ContinuousDistribution createRandomGen(int experimentIndex, double minInclusive, double maxExclusive) Creates a pseudo random number generator (PRNG) for a experiment run that generates uniform values between [min and max[. If it is to apply the"Antithetic Variates Technique"
to reduce results' variance, the second half of experiments will use the seeds from the first half.- Parameters:
experimentIndex
- index of the experiment run to create a PRNGminInclusive
- the minimum value the generator will return (inclusive)maxExclusive
- the maximum value the generator will return (exclusive)- Returns:
- the created PRNG
- See Also:
-
isToReuseSeedFromFirstHalfOfExperiments
public boolean isToReuseSeedFromFirstHalfOfExperiments(int currentExperimentIndex) -
halfSimulationRuns
public int halfSimulationRuns()- Returns:
- the half of
getSimulationRuns()
-
getExperimentsExecutionTimeSecs
public long getExperimentsExecutionTimeSecs()Time in seconds the experiments took to finish.- Returns:
-
getExperimentsStartTimeSecs
public long getExperimentsStartTimeSecs()Time in seconds the experiments started.- Returns:
-
run
public void run()Setups and starts the execution of all experiments sequentially or inparallel
. -
createExperimentInternal
Creates an experiment to be run for the i'th time.- Parameters:
index
- a number that identifies the experiment- Returns:
- the created experiment
- See Also:
-
createExperiment(int)
-
computeFinalResults
Computes final simulation results, including mean, standard deviations and confidence intervals for a given metric computed across all simulation runs.- Parameters:
metricEntry
- a map entry represented by the name of the metric and its list of values across multiple simulation runs- Returns:
- the computed
ConfidenceInterval
from the provided values for the metric
-
computeFinalStatistics
protected final org.apache.commons.math3.stat.descriptive.SummaryStatistics computeFinalStatistics(List<Double> values) Creates aSummaryStatistics
object from a list of Double values, allowing computation of statistics such as mean over these values. The method also checks if theAntithetic Variates
and theBatch Means
techniques are enabled, applying them over the given list of Doubles. These techniques are used for variance reduction.- Parameters:
values
- the List of values to add to theSummaryStatistics
object- Returns:
- the
SummaryStatistics
object containing the double values, after applying the techniques for variance reduction.
-
addMetricValue
Add a value to a given metric inside themetricsMap
.This method must be called for each metric inside the experiment finish listener. The listener can be set inside the runner's
createExperimentInternal(int)
.- Parameters:
metricName
- the name of the metric to collect the data for a simulation runvalue
- the value for at metric for a given simulation run. If null is given, that means no value was collected for that metric in the run, but the metric entry must exist in the metrics map, so that the final results table show the metric entry with 0.- See Also:
-
getMetricValues
-
computeAntitheticMeans
Computes the antithetic means for the given samples if the
"Antithetic Variates Technique" is to be applied
. These values are the mean between the first half of samples with the second half. By this way, the resulting value is an array with half of the samples length.NOTE: To correctly compute the antithetic values the seeds from the first half of experiments must be used for the second half.
- Parameters:
samples
- the list of samples to compute the antithetic means from- Returns:
- the computed antithetic means from the given samples if the "Antithetic Variates Technique" is to be applied, otherwise return the same given samples list.
- See Also:
-
printSimulationParameters
protected abstract void printSimulationParameters() -
setFirstExperimentCreated
public void setFirstExperimentCreated(int firstExperimentCreated) -
getFirstExperimentCreated
public int getFirstExperimentCreated() -
setDescription
Sets a description for this experiment which is shown when it starts. It's also used to generate a caption for the Latex table.- Parameters:
description
- the description to set- See Also:
-
getDescription
-
setResultsTableId
An id used to identify the experiment results table generated in formats such as Latex for computed metrics.- Parameters:
resultsTableId
- the name to set- See Also:
-
latexTableResultsGeneration
-
getResultsTableId
-
enableLatexTableResultsGeneration
Enables the generation of a result table in Latex format for computed metrics.- Returns:
-
isLatexTableResultsGeneration
public boolean isLatexTableResultsGeneration()Checks if generation of a result table in Latex format for computed metrics is enabled.- Returns:
-
isShowProgress
public boolean isShowProgress()Checks if a progress bar is to be printed to show when each experiment run finishes. It's just printed when the number of simulations is greater than 1 and experiments are not set as verbose. It's shown by default if those conditions are met.- Returns:
-
setShowProgress
Enable or disables a progress bar to show when each experiment run finishes. It's just printed when the number of simulations is greater than 1 and experiments are not set as verbose.- Parameters:
showProgress
- true to enable the progress bar, false to disable- Returns:
-
getFinishedRuns
public int getFinishedRuns() -
setProgressBarInNewLine
-
isVerbose
public boolean isVerbose()Checks if simulation results of the experiment have to be output. Ifparallel
execution is enabled, you may consider disabling verbosity for individualExperiment
s created, since messages from different runs will be mixed up and may cause confusion.- Overrides:
isVerbose
in classAbstractRunnable
- Returns:
-
isParallel
public boolean isParallel()If experiments are executed in parallel, each experiment verbosity is disabled, otherwise, you'll see mixed log messages from different experiment runs.
-