ANOVA separates the total variation in a set of measurements into a component due to random fluctuations in the measurements and a component due to the actual differences among the alternatives.
ANOVA separates the total variation in a set of measurements into a component due to random fluctuations in the measurements and a component due to the actual differences among the alternatives.
If the variation between the alternatives is larger than the variation within each alternative, then it can be concluded that there is a statistically significant difference between the alternatives.
For more information see: Andy Georges, Dries Buytaert, Lieven Eeckhout - Statistically Rigorous Java Performance Evaluation
Compares the coefficient of variance to some threshold
value.
Compares the coefficient of variance to some threshold
value.
This heuristic can be used to detect if the measurement has stabilized.
Compares two alternative sets of measurements given a significance level alpha
.
Compares two alternative sets of measurements given a significance level alpha
.
if true
, the confidence interval test is strict - the confidence interval overlap
of the alternatives will not be additionally compared
returns true
if there is no statistical difference for s.l. alpha
Computes the confidence interval of the two alternatives.
Computes the confidence interval of the two alternatives.
Passes if the confidence intervals overlap at the given significance level alpha
.
Applies relative noise before doing the actual overlap test - the relative noise may increase the confidence interval further, but it will not shrink it.
Computes sum-of-squares due to differences between alternatives.
Computes sum-of-squares due to errors in measurements.
Let Y = (Y_1, ..., Y_n) data resulting from a parametric law F of scalar parameter θ.
Let Y = (Y_1, ..., Y_n) data resulting from a parametric law F of scalar parameter θ. A confidence interval (B_i, B_s) is a statistic in the form of an interval containing θ with a specified probability.
Computes the mean of the sequence of measurements.
The sample standard sample deviation.
The sample standard sample deviation. It is the square root of S², unbiased estimator for the variance.
Standard statistics utilities.
Note: significance level
alpha
is equal to1 - confidenceLevel
. If you want to be sure that 2 sets of measurements do not differ with90
percent probability, then the significance levelalpha
should be set to0.1
. In this example, the confidence level is0.9
, and the significance level is0.1
.