Back to all lessons
Machine LearningIntermediate

✂️Support Vector Machines

Finding the safest separating line

Take your time with this one. The interactive parts are here to help you test the idea, not rush through it.

25 min- Explore at your own pace

Before We Begin

What we are learning today

An SVM separates classes with a boundary that keeps the biggest possible buffer zone. It’s all about confident, tidy separation.

How this lesson fits

Here’s where the magic shows up: we stop hand-writing every rule and let data teach the model. Think of it as coaching instead of scripting.

The big question

How can a machine spot patterns from examples the way a student learns from practice problems?

Tell the difference between predicting numbers and discovering patternsInterpret simple models and talk through their outputs in plain EnglishCompare the strengths and tradeoffs of common ML methods

Why You Should Care

This builds geometric intuition: good classification isn’t just about separating—it’s about separating with confidence.

Where this is used today

  • ✓Handwriting recognition (OCR)
  • ✓Image classification (pre-Deep Learning)
  • ✓Bioinformatics (protein classification)

Think of it like this

Imagine two teams on opposite sides of a gym. You tape a line that leaves the widest safe gap so no one bumps into it.

Easy mistake to make

SVMs aren’t magic winners. They shine in some tasks and are overkill or slow in others.

By the end, you should be able to say:

  • Define support vectors and margin in simple language
  • Explain why a wider margin can improve generalization
  • Compare the SVM idea to logistic regression

Think about this first

If two teams line up on opposite sides of a room, where would you put the tape line so no one crosses accidentally?

Words we will keep using

marginsupport vectorboundarykernelgeneralization

Support Vector Machines

An SVM is a perfectionist. It doesn't just want to separate the red dots from the blue dots; it wants to build the widest possible street between them. The wider the street (margin), the safer the model is from making mistakes.

Maximize  2∥w∥subject toyi(w⋅xi+b)≥1\text{Maximize} \; \frac{2}{\|w\|} \quad \text{subject to} \quad y_i(w \cdot x_i + b) \geq 1
MarginThe safety zone. The empty space separating the two teams.
Support VectorsThe VIPs. The specific points that touch the edge of the street. They decide everything.
Kernel TrickA math hack that lets the model draw curvy boundaries by pretending to be in higher dimensions.

Drag Points → Watch Boundary Adapt

🔴 Class -1   🔵 Class +1   🟡 Support vectors   Dashed lines = margin boundaries

Kernel

Tip: Drag a red point close to the blue side and watch what changes. The SVM mostly cares about the points near the boundary, not the ones far away.
w₁=0.43 w₂=0.53
b=-4.59
Margin=2.939

SVM vs Logistic Regression

Logistic Regression focuses on probabilities and works well when you want a simple, interpretable classifier.
SVM focuses on geometry and margin, which can make it very strong on some hard classification problems.