Fatal Error ClubFatal Error
Roadmap

Week 7 · Classical ML

Linear regression on unclean data: data cleaning workshop

01

Inventory the damage

Nulls, weird units, duplicated rows, a leakage column. Name them before you impute.

Raw table · houses.csv

Real data is never clean. Toggle a lens to see what is hiding in this table.

idareabedspriceprice_next_year
1622141148
2743188197
3-2132139
491-224233
5743188197
6581121126
71034256-
8672150158

Eight rows. Nothing here is wrong at a glance. That is the trap.

02

Fit on train only

Fit transforms on train. Apply to test. If you fit on everything, the test already peeked.

Fit order matters

Where you call .fit() on the scaler decides whether test data leaks into training.

All dataTrainTestscaler.fit()✕
Contaminated: test statistics leaked into the scaler before training even started.
03

Outliers and scaling

Decide what is signal vs sensor glitch. Standardize when distance or gradients care.

Outliers & scale

Two extreme points can drag a whole feature's scale around.

raw units

With outliers left in, most points get crushed into a corner. Scale is not a cosmetic step.

04

Same model, cleaner world

Refit the same regression before and after cleaning. Different world. Same model class.

Same model, same houses

One fit trains on the raw dirty table. The other trains on the cleaned version. Everything else is identical.

Dirty data · R²-
Clean data · R²-

Most production ML failures are data failures wearing a model costume.