Inbox: is this spam?
A pile of messages. Some are junk. Some are real. means not spam: the good mail. Someone already labeled them. That labeled pile is how supervised learning starts.
Labeled Data Example
Ham
Hey, are we still on for lunch tomorrow?
Your package arrives Thursday. Tracking inside.
Can you review the draft before standup?
See you at the club meeting at 18:00.
Spam
FREE money!!! Click now to claim your prize $$$
URGENT: verify your account or it will be closed
Congrats winner! Claim FREE gift card today
Limited offer: buy now, earn cash fast!!!
Labels are 0 and 1
Same game as cats and dogs. Input is the message. Target is spam (1) or ham (0). Drop a message into the model. Sentiment is the same idea with other labels: positive vs negative.
Supervised mapping
Message in. Label out.
Drop a message into the input. The model returns 0 or 1.
input
Drop a message here
Classifying
label
-
Click to inputDrag and drop to the input
What "classification" actually means
Classification is to sort an input into one of a fixed number of . Spam vs ham is the same game: two classes, every message gets classified.
Two buckets is classification. Three or more is .
Classification
Sort into fixed buckets
Today
Logistic regression
Logistic regression
We can think of the logistic regression model as a box that outputs a probability. Features go in. One number between 0 and 1 comes out.
Features in. Probability out.
LR Model
Out
probability
Inside the model
Inputs become a score. Sigmoid turns that score into a probability.
inputs
score
out
Training picks weights by minimizing binary cross entropy. For one labeled example with true label y and predicted probability σ(z):
For the scope of this course, it is a bit early to try to understand binary cross entropy loss (Error) in depth. We will take the time to understand it better in the second semester. For now, the intuition is simple: the loss (Error) stays small when the predicted probability matches the true label, and grows when the model is confidently wrong, which pushes training toward better predictions.
Unclear? Let's understand with an example.
Pass or fail from study hours, sleep, and quizzes. Train on labeled rows, read the learned weights, then score one new student and squash z into ŷ.
We want to know whether a student will pass or fail. Logistic regression takes study hours, sleep, and quizzes passed, and returns . Below is labeled data the model was trained on.
| Hours | Sleep | Quizzes | Label |
|---|---|---|---|
| 9 | 7 | 4 | 1 |
| 2 | 5 | 0 | 0 |
| 5 | 6 | 2 | 1 |
| 3 | 4 | 1 | 0 |
Train on that table. Logistic regression learns these parameters:
-5
hours
0.55
sleep
0.35
quizzes
0.8
Features in
Three numbers enter the trained box. Weights were already learned from the table above.
| Feature | x |
|---|---|
| hours | |
| sleep | |
| quizzes |
| b |
|---|
Score z
Multiply each feature by its weight, then add the bias.
Feed to sigmoid
z can be any size. σ(z) forces it into (0, 1): the probability of pass.
You now know how classification works, how logistic regression turns features into a probability, and how to build a spam classifier in the notebook. Next week covers classifiers that vote instead of relying on one line.