Problems of a significance test with a low power.
Reproducible research findings are a cornerstone of the scientific method. However, studies have shown that results of published research can be difficult to replicate when only statistically significant results are published, particularly when the statistical significance tests are conducted with small sample sizes (hence with low statistical power). Letâ€™s consider a one sample t-test with 0Â versus a
(a) If the population standard deviation is 0.5, what is the power of the test when the effect size is Î´ = 0.1 and sample size is 1 = 10?
(b) In order to reject the null hypothesis, how large must the sample mean Â be?
(c) If you rejected the null using data from one experiment, how likely can the statistically significant result be verified if you repeat the same experiment?
(d) Suppose that you realized that the test has a low power and you decided to repeat the experiment with a much larger sample size (e.g., 2 = 100). What is the power of the new test?
(e) Assuming Î´ = 0.1, how likely is it that you will obtain a sample mean as large as the statistically significant result from the experiment with 1 = 10?