Probability & Statistics

Distributions, Bayes, and hypothesis testing

Go deeper into the statistical toolkit every ML practitioner reaches for: distributions, Bayes' rule, and the logic of hypothesis testing.

Before We Begin

What we are learning today

We touched on probability in Foundations. Now we sharpen the tools. This lesson explores the distributions that show up everywhere in data—normal, Poisson, binomial—and teaches Bayes' rule as a formal method for updating beliefs with evidence. We finish with hypothesis testing: the scientific method's way of asking "is this result real or just luck?"

How this lesson fits

Theory is only half the story. In this module we roll up our sleeves and learn the craft behind every successful ML project: preparing data, engineering features, and strengthening our statistical intuition. These are the skills that separate a notebook experiment from a model you can actually trust.

The big question

What does raw, messy, real-world data need before a model can learn anything useful from it?

Clean, encode, and scale a dataset so a model can digest itSplit data into training, validation, and test sets—and explain why each mattersApply probability and hypothesis testing to make data-driven decisions

Why You Should Care

Every model evaluation, every A/B test, every confidence interval rests on these ideas. Students who internalize them will never be tricked by a lucky result or a misleading p-value. These are the critical-thinking tools of the data age.

Where this is used today

✓A/B testing: deciding whether a website change actually improved sign-ups
✓Quality control: testing whether a factory batch meets safety standards
✓Clinical trials: determining whether a new drug works better than a placebo

Think of it like this

Imagine you are a jury member. The prosecution presents evidence (data), and you start with some prior belief about guilt. Each new piece of evidence shifts your confidence up or down. At the end, you decide: is there enough evidence to reject "not guilty"? That is Bayesian reasoning meets hypothesis testing.

Easy mistake to make

A small p-value does not prove your hypothesis is true. It only says the result would be unlikely under the null hypothesis. Sample size, effect size, and context all matter just as much.

By the end, you should be able to say:

Match real-world scenarios to the right probability distribution
Apply Bayes' rule to update a prior belief with new evidence
Explain null hypothesis, p-value, and significance level in plain language
Distinguish between statistical significance and practical importance

Think about this first

A medical test is 95% accurate. You test positive. Does that mean there is a 95% chance you are sick? Why might the real number be very different?

Words we will keep using

distributionpriorposteriornull hypothesisp-valuesignificance levelconfidence interval

Distributions: The Shapes of Data

Every dataset has a shape. Understanding which distribution your data follows tells you what to expect, what counts as unusual, and which statistical tools apply.

The bell curve. Heights, test scores, measurement errors.

f(x) = \frac{1}{\sigma\sqrt{2\pi}} e^{-\frac{(x-\mu)^2}{2\sigma^2}}

Interactive: Explore Distributions

Mean (\u03bc): 5.0Std dev (\u03c3): 1.5

Bayes' Theorem: Updating Beliefs

A medical test is 95% sensitive and 90% specific. The disease prevalence is 1%. If you test positive, what's the probability you actually have the disease? Most people guess ~90%. The real answer is shockingly different.

P(D|+) = \frac{P(+|D) \cdot P(D)}{P(+|D) \cdot P(D) + P(+|\neg D) \cdot P(\neg D)}

Prior P(D) \u2014 disease prevalence: 1.0%Sensitivity P(+|D): 95%Specificity P(\u2212|\u00acD): 90%

If you test positive, the real probability you have the disease is 8.8%. That's less than 50% \u2014 the prior was so low that even a good test can't overcome it!

Hypothesis Testing

The scientific method for data: start with a null hypothesis ("nothing is happening"), collect evidence, and ask: would this evidence be surprising if the null were true?

p-valueThe probability of seeing a result at least this extreme if the null hypothesis is true. A small p-value means the result is surprising under the null.

Significance level (\u03b1)Your threshold for "surprising enough." Convention is

\alpha = 0.05

. If p < \u03b1, you reject the null.

Statistical \u2260 PracticalA tiny p-value with a huge sample just says "the effect is probably not zero." It says nothing about whether the effect size matters in real life.

Common MisconceptionA p-value is NOT the probability the null is true. It is the probability of the data given the null \u2014 a subtle but critical difference.