ALL YOU NEED TO KNOW ABOUT STATISTICS

In 15 minutes

Roberto A. Vitillo

Setting a 95% confidence interval means that if you took repeated random samples from a population and calculated the statistics and CI for each sample, then the CIs for 95% of your samples would include the true value of the statistics.

Central Limit Theorem

For means it’s easy: the histogram of averages tends to look normal even when the histogram of the individuals doesn’t!

aka sampling distribution of the mean

It’s easy to derive a confidence interval once we know how the theoretical sampling distribution looks like.

~95% confidence interval

But I don’t care about means…

What now?call this guy if you live in the

early 20th century

Henry Berthold Mann known for the Mann-Whitney nonparametric test

throw some (virtual) dice on your laptop

not only compilers can be bootstrapped…

n bootstrap samples, each of size k, are generated by sampling with replacement from the original sample A

A X X X1 2 3* * *

In the next phase, a bootstrap statistic is calculated for all the bootstrap samples

bootstrap distribution

The bootstrap distribution is an approximation of the sampling distribution.

~95% confidence interval

• Resampling methods are powerful tools

• A similar procedure can be applied for A/B tests

• Checkout montecarlino