Fatal Error ClubFatal Error
Roadmap

Week 4 · Classical ML

Classification + build a spam/sentiment classifier

01

Inbox: is this spam?

A pile of messages. Some are junk. Some are real. means not spam: the good mail. Someone already labeled them. That labeled pile is how supervised learning starts.

Labeled Data Example

Ham

  • Hey, are we still on for lunch tomorrow?

  • Your package arrives Thursday. Tracking inside.

  • Can you review the draft before standup?

  • See you at the club meeting at 18:00.

Spam

  • FREE money!!! Click now to claim your prize $$$

  • URGENT: verify your account or it will be closed

  • Congrats winner! Claim FREE gift card today

  • Limited offer: buy now, earn cash fast!!!

02

Labels are 0 and 1

Same game as cats and dogs. Input is the message. Target is spam (1) or ham (0). Drop a message into the model. Sentiment is the same idea with other labels: positive vs negative.

Supervised mapping

Message in. Label out.

Drop a message into the input. The model returns 0 or 1.

input

Drop a message here

Classifying

label

-

Click to input

03

What "classification" actually means

Classification is to sort an input into one of a fixed number of . Spam vs ham is the same game: two classes, every message gets classified.

Two buckets is classification. Three or more is .

04

Logistic regression

We can think of the logistic regression model as a box that outputs a probability. Features go in. One number between 0 and 1 comes out.

Features in. Probability out.

Inside the model

Inputs become a score. Sigmoid turns that score into a probability.

Training picks weights by minimizing binary cross entropy. For one labeled example with true label y and predicted probability σ(z):

L=−ylog⁡(σ(z))−(1−y)log⁡(1−σ(z))L = -y \log(\sigma(z)) - (1-y) \log(1 - \sigma(z))

For the scope of this course, it is a bit early to try to understand binary cross entropy loss (Error) in depth. We will take the time to understand it better in the second semester. For now, the intuition is simple: the loss (Error) stays small when the predicted probability matches the true label, and grows when the model is confidently wrong, which pushes training toward better predictions.

05

Unclear? Let's understand with an example.

Pass or fail from study hours, sleep, and quizzes. Train on labeled rows, read the learned weights, then score one new student and squash z into ŷ.

We want to know whether a student will pass or fail. Logistic regression takes study hours, sleep, and quizzes passed, and returns . Below is labeled data the model was trained on.

HoursSleepQuizzesLabel
9741
2500
5621
3410

Train on that table. Logistic regression learns these parameters:

b0b_0

-5

b1b_1hours

0.55

b2b_2sleep

0.35

b3b_3quizzes

0.8

Features in

Three numbers enter the trained box. Weights were already learned from the table above.

Featurex
hoursx1=6x_1 = 6
sleepx2=7x_2 = 7
quizzesx3=3x_3 = 3
b
b1=0.55b_1 = 0.55
b2=0.35b_2 = 0.35
b3=0.8b_3 = 0.8

Score z

Multiply each feature by its weight, then add the bias.

z=b0+b1x1+b2x2+b3x3z = b_0 + b_1x_1 + b_2x_2 + b_3x_3
z=−5+0.55⋅6+0.35⋅7+0.8⋅3z = -5 + 0.55\cdot 6 + 0.35\cdot 7 + 0.8\cdot 3
z=−5+3.30+2.45+2.40z = -5 + 3.30 + 2.45 + 2.40
z=3.15z = 3.15

Feed to sigmoid

z can be any size. σ(z) forces it into (0, 1): the probability of pass.

y^=σ(z)=11+e−z\hat{y} = \sigma(z) = \frac{1}{1 + e^{-z}}
y^=11+e−3.15=0.959\hat{y} = \frac{1}{1 + e^{-3.15}} = 0.959
y^=\hat{y} ==96%= 96\%

You now know how classification works, how logistic regression turns features into a probability, and how to build a spam classifier in the notebook. Next week covers classifiers that vote instead of relying on one line.