An introduction to the Statistics in reproducibility module

Reproducible analysis is strongly impacted by statistical analyses.
Reproducible research requires understanding the notions of sampling, testing, power, model selection.

Statistical basis for neuroimaging analyses: the basics

Be familiar with the concept of sampling
Know what we call a distribution, a pvalue, a confidence interval
Have some knowledge of Bayesian statistics and model comparison
This is in line with our overall goal of making science (including scientific training) more open.

Effect size and variation of effect sizes in brain imaging

Effect sizes come in many forms
Significance is not relevance
Difference between the raw effect size and the cohen’s d effect size
How can the effect size vary? Why is it important to know about this?
Effect sizes are under reported, not well understood, and are crucial for our scientific understanding. Let’s fix this.

Pvalues and their issues

A pvalue does not give you an idea of the importance of the result
A pvalue should always be complemented by other information (effect size, confidence interval)

About statistical power

The lack of power is much more problematic that it seems at first sight.  It would usually lead to wasted resources  If an under powered study yields some significant effects, these are likely to be overestimated  If an under powered study yields some significant effects, these are less likely to replicate

The positive Predictive Value

A significant (say at the 0.05 level) may have a low chance of replication.
The PPV estimates the probability that the alternative hypothesis H_A is true given that the test is significant at some \alpha level.
This probability depends on several factors such as power \beta, \alpha level, but also the prior chance that H_A is true.

Cultural and psychological issues

