The Forger and the Judge
A working course on generative adversarial networks: how two neural networks turn noise into data, why their objective is a game, and what makes that game fail.
One model forges. One model judges.
A generative adversarial network contains two networks with opposing jobs. The generator turns random noise into synthetic samples. The discriminator inspects real and generated samples and tries to tell which is which.
Neither network receives a pixel-by-pixel target for what a generated sample should look like. The generator learns from the discriminator's criticism. The discriminator improves because the generator keeps inventing harder negatives.
One training round at a time
Generator
noise
sample fidelity
Discriminator
98%
classification accuracy
The destination
0.5, the score of a coin flip.The generator learns a distribution
Start with a vector z sampled from a simple distribution, often a Gaussian. The generator Gθ maps that vector into the shape of the training data: an image, waveform, or another structured object.
Different noise vectors must produce different plausible outputs. Training is therefore not about reproducing one correct answer. It is about pushing an entire generated distribution p_g toward the real distribution p_data.
What the latent vector controls
Position
Nearby z values often produce nearby outputs.
Variation
Moving through z changes features the model learned.
Coverage
The full noise space should reach every real mode.
Important distinction
The discriminator supplies the lesson
The discriminator Dφ is a binary classifier. It receives a sample and returns a score between zero and one. Higher means more likely to be real.
Its training batch mixes real examples x ~ p_data with generated examples G(z). Binary cross-entropy rewards high scores on real data and low scores on generated data.
Only the discriminator's parameters change during this update. Generated samples are detached from the generator's graph.
Why the judge cannot be static
The loss is a two-player game
The original GAN objective is written as a minimax game. The discriminator maximizes the value by separating real from fake. The generator minimizes it by making generated samples impossible to reject.
That equation is elegant but the literal generator term can provide a weak gradient early in training. When the discriminator easily rejects every fake, the practical non-saturating loss gives the generator a much clearer correction.
The equilibrium is unchanged. Only the strength of the generator's learning signal is improved.
What should the generator optimize?
Move the score toward zero to simulate a strong discriminator at the start of training.
minimax signal
-0.083
useful gradient
12.5x
The practical non-saturating loss is -log D(G(z)) = 2.526. It gives the generator a strong correction when the judge confidently rejects its early samples.
Alternate the updates carefully
A GAN is not optimized as one ordinary loss in one ordinary backward pass. The two networks take turns. Each update changes the problem the other network sees next.
Alternating optimization
Draw a real batch x and a noise batch z.
During the D step
φ. Otherwise the generator receives gradients while the judge is still learning how to judge.During the G step
θ.Balance matters. If the discriminator becomes perfect too quickly, the generator gets an unhelpful signal. If it is too weak, the generator learns cheap artifacts instead of the real distribution. Learning rates, update ratios, normalization, and architecture all influence that balance.
Good samples can still hide a bad model
GAN training can look successful while missing the actual goal. A few sharp images do not prove that the generator learned the full data distribution.
Coverage of the real distribution
Generated samples cover every visible mode of the real distribution.
Oscillation
Discriminator imbalance
Evaluation needs both quality and diversity. Visual inspection is useful, but metrics such as FID are used to compare feature distributions across many samples. They are diagnostics, not proof that a model is safe or useful.
The whole system is one feedback loop. Noise gives the generator room to vary, the discriminator turns real data into a lesson, and alternating optimization keeps both players learning from the latest opponent.
Check your understanding
Self-contained GAN course. Equations render with KaTeX. Interactive diagrams stay on this page.