Fatal Error ClubFatal Error
Roadmap

Week 2 · Classical ML

Supervised vs unsupervised

01

Find the function

We start simple. Below is a small table with a row for x and a row for f(x). The only goal is to find the function f(x):

x1234567
f(x)14916253649

Which function fits?

02

When you cannot write it

You just solved an easy case by hand. Real problems are messier. Here is another table with a pattern that exists, but almost nobody in the room will write a clean closed form for it. Try anyway. The struggle is the lesson.

x1234567
f(x)37184085175358

Which function fits?

03

An image is numbers

Same idea, messier inputs. Instead of a single number x, the input can be an image. An image is not magic. It is a grid of pixel values: a matrix of numbers. Slow down on that for a second.

CIFAR-10 dog sample

Each image is just a matrix of pixel values.

04

The ideal function

A model is a function that turns an image into an answer. But we are not there yet.

The function f⋆f^{\star} is the ideal function, the one that is always right. If you give it a dog image, it always returns 0. If you give it a cat image, it always returns 1.

There is no closed-form formula. You cannot express f⋆f^{\star} as an equation like f(x)=3x+2f(x) = 3x + 2, and you cannot derive it with pen and paper. But a person looking at the photo already computes it in their head. Let's understand this with a game.

05

Images: cats and dogs

Now the function guessing game again, with samples as an example. Your task is to label dogs images as 0 and cats images as 1 (this is an arbitrary choice, we could have chosen the opposite, it doesn't matter for our objective of understanding how machine learning works).

CIFAR-10 dogs sample 1
CIFAR-10 dogs sample 2
CIFAR-10 dogs sample 3
CIFAR-10 dogs sample 4
Dogs
0label
CIFAR-10 cats sample 1
CIFAR-10 cats sample 2
CIFAR-10 cats sample 3
CIFAR-10 cats sample 4
Cats
1label
06

The full picture

Now that we know all of this, let's clean up the full picture.

We need a dataset. We find one online, or we make it ourselves. It looks like this.

The general idea

Guess. Check. Correct. Repeat.

The input is a cat image. The ideal output is 1. The model never sees the rule, only its own guess, checked and corrected each round.

input

Cat

model guess

-

07

Supervised learning in general

The cat image was one case. Take the pictures off. The idea is the same for any problem.

We know the input xx, and we know the output it should produce, yy. That known output is the label. We do not know the function that links them, so the model is a box with a question mark. The box reads xx and returns a guess, y^\hat{y}.

Supervised learning is how we fill that box in. The question mark has to become one function for every pair in D\mathcal{D}, not a separate answer for a single xx. Take each pair (xi,yi)(x_i, y_i) in the dataset. The model reads xix_i and returns y^i\hat{y}_i. We look for the function that makes the total gap as small as possible,

∑(xi, yi)∈D∣yi−y^i∣\sum_{(x_i,\, y_i) \in \mathcal{D}} \left| y_i - \hat{y}_i \right|

Training replaces the question mark with that function. xx can be an image, a number, or a sentence. The diagram does not change.

08

Categories of clients

The last dataset was pairs: an image, and 0 or 1. This one never had a second number. As a business, you want to know the categories of clients that buy from you. For each client the records hold two things: age, and money spent. Nobody wrote the category down.

Each click moves the pins one step. After they stop, the lines show the clients closer to each pin. Distance is measured on the graph. The clients stay put. Those groups are the categories.

The pins start away from the clients. Next moves them, then draws who is closer.

09

Unsupervised learning in general

The clients were one case. Take the sales records off. Any pile of inputs with no answer works the same way. The dataset is only xx. The model returns structure, not a guess of yy.

There is no yy. Each item is only an input, xx. Nothing tells the model the right answer, so there is no y^\hat{y} to score against a label.

The model looks for structure in the pile of xx: groups, a shorter description, or the point that does not fit. A person can name a group later. The name is not part of training.

D={x1,x2,…}\mathcal{D} = \{ x_1, x_2, \ldots \}

Unsupervised

Recap: there is no yy. The model groups things, compresses them, or flags the point that does not fit. Next-word prediction is not unsupervised. The next word is already in the sentence, so the text made its own label. That is self-supervised.