Accuracy, precision, recall, F1, and ROC-AUC
Accuracy alone can lie. Here we learn the full scorecard—precision, recall, F1, confusion matrices, and ROC curves—so we can measure what truly matters.
Before We Begin
Imagine a smoke detector that never goes off. Its "accuracy" on fire-free days is 100%, but it is completely useless. This lesson teaches students to see past the accuracy number and ask sharper questions: Of the alarms we raised, how many were real? (Precision.) Of all the real fires, how many did we catch? (Recall.) The F1 score balances both. The confusion matrix lays out every type of mistake in a neat grid, and the ROC-AUC curve shows how the model performs across all possible thresholds.
How this lesson fits
Building a model is one thing; training it well and knowing whether it actually works is another. This module covers the engine room of ML: how optimization drives learning, how bias and variance shape model behavior, how to pick the right scoreboard, and how to run experiments you can trust.
The big question
How do we train a model effectively, measure its true performance, and make sure our results are not just a fluke?
Why You Should Care
In high-stakes domains—medicine, criminal justice, autonomous driving—the type of mistake matters as much as the rate. A model that misses 40% of cancers is dangerous even if its overall accuracy is 95%. Students must learn to choose metrics that match the real-world cost of errors.
Where this is used today
Think of it like this
Think of a spelling bee judge. Accuracy is "how many words did contestants spell correctly overall?" But that ignores the hard words. Precision asks "when the judge said correct, were they right?" Recall asks "of all the correctly spelled words, how many did the judge catch?" You need both lenses to know if the judging is fair.
Easy mistake to make
High accuracy does not mean a model is useful. On imbalanced data—where one class dominates—a model that always predicts the majority class can score high accuracy while being completely blind to the minority class.
Think about this first
A model predicts "no disease" for every patient and is 98% accurate. Is it a good model? Why or why not?
Words we will keep using
A smoke detector that never goes off has 99% accuracy on fire-free days \u2014 and is completely useless. To measure what truly matters, we need sharper tools: precision, recall, F1, and the ROC curve.
Adjust the classification threshold. Watch how every cell changes. Lower the threshold to catch more positives (higher recall) at the cost of more false alarms (lower precision).
Precision = TP / (TP + FP) \u2014 of all alarms, how many were real?
Recall = TP / (TP + FN) \u2014 of all real fires, how many did we catch?
F1 = 2 \u00d7 P \u00d7 R / (P + R) \u2014 the harmonic mean, balances both.
The ROC curve traces every possible threshold. The area under it (AUC) summarizes model quality in a single number: 0.5 = random guessing, 1.0 = perfect. AUC is threshold-independent, making it ideal for comparing models.