Fatal Error ClubFatal Error
Extra resources

The Forger and the Judge

A working course on generative adversarial networks: how two neural networks turn noise into data, why their objective is a game, and what makes that game fail.

adversarial training, livethe judge sees the fakes
01

One model forges. One model judges.

A generative adversarial network contains two networks with opposing jobs. The generator turns random noise into synthetic samples. The discriminator inspects real and generated samples and tries to tell which is which.

Neither network receives a pixel-by-pixel target for what a generated sample should look like. The generator learns from the discriminator's criticism. The discriminator improves because the generator keeps inventing harder negatives.

One training round at a time

Generator

noise

sample fidelity

vs

Discriminator

98%

classification accuracy

UntrainedEasy win for D

The destination

At the ideal equilibrium, generated samples follow the same distribution as real data. The best discriminator can do is output 0.5, the score of a coin flip.
02

The generator learns a distribution

Start with a vector z sampled from a simple distribution, often a Gaussian. The generator Gθ maps that vector into the shape of the training data: an image, waveform, or another structured object.

z∼p(z),x^=Gθ(z)\mathbf{z} \sim p(\mathbf{z}), \qquad \hat{\mathbf{x}} = G_\theta(\mathbf{z})

Different noise vectors must produce different plausible outputs. Training is therefore not about reproducing one correct answer. It is about pushing an entire generated distribution p_g toward the real distribution p_data.

What the latent vector controls

Position

Nearby z values often produce nearby outputs.

Variation

Moving through z changes features the model learned.

Coverage

The full noise space should reach every real mode.

Important distinction

A GAN generator does not memorize a lookup table from noise to training examples. Its weights define a continuous transformation that can produce samples not present in the dataset.
03

The discriminator supplies the lesson

The discriminator Dφ is a binary classifier. It receives a sample and returns a score between zero and one. Higher means more likely to be real.

Dϕ(x)∈(0,1)D_\phi(\mathbf{x}) \in (0,1)

Its training batch mixes real examples x ~ p_data with generated examples G(z). Binary cross-entropy rewards high scores on real data and low scores on generated data.

LD=− Ex∼pdata ⁣[log⁡D(x)]−Ez∼p(z) ⁣[log⁡ ⁣(1−D(G(z)))]\mathcal{L}_D = -\,\mathbb{E}_{\mathbf{x}\sim p_{\mathrm{data}}}\!\left[\log D(\mathbf{x})\right] - \mathbb{E}_{\mathbf{z}\sim p(\mathbf{z})}\!\left[\log\!\left(1-D(G(\mathbf{z}))\right)\right]

Only the discriminator's parameters change during this update. Generated samples are detached from the generator's graph.

Why the judge cannot be static

A frozen classifier would teach the generator one fixed exploit. A trained discriminator keeps closing those loopholes, forcing the generator to model more of the real data structure.
04

The loss is a two-player game

The original GAN objective is written as a minimax game. The discriminator maximizes the value by separating real from fake. The generator minimizes it by making generated samples impossible to reject.

min⁡Gmax⁡D  V(D,G)=Ex∼pdata ⁣[log⁡D(x)]+Ez∼p(z) ⁣[log⁡ ⁣(1−D(G(z)))]\min_G \max_D \; V(D,G) = \mathbb{E}_{\mathbf{x}\sim p_{\mathrm{data}}}\!\left[\log D(\mathbf{x})\right] + \mathbb{E}_{\mathbf{z}\sim p(\mathbf{z})}\!\left[\log\!\left(1-D(G(\mathbf{z}))\right)\right]

That equation is elegant but the literal generator term can provide a weak gradient early in training. When the discriminator easily rejects every fake, the practical non-saturating loss gives the generator a much clearer correction.

LGNS=− Ez∼p(z) ⁣[log⁡D(G(z))]\mathcal{L}_G^{\mathrm{NS}} = -\,\mathbb{E}_{\mathbf{z}\sim p(\mathbf{z})}\!\left[\log D(G(\mathbf{z}))\right]

The equilibrium is unchanged. Only the strength of the generator's learning signal is improved.

What should the generator optimize?

0.08

Move the score toward zero to simulate a strong discriminator at the start of training.

minimax signal

-0.083

useful gradient

12.5x

The practical non-saturating loss is -log D(G(z)) = 2.526. It gives the generator a strong correction when the judge confidently rejects its early samples.

05

Alternate the updates carefully

A GAN is not optimized as one ordinary loss in one ordinary backward pass. The two networks take turns. Each update changes the problem the other network sees next.

Alternating optimization

Draw a real batch x and a noise batch z.

During the D step

Generate a fake batch, detach it, and update only φ. Otherwise the generator receives gradients while the judge is still learning how to judge.

During the G step

Freeze the discriminator's parameters but keep it in the computation graph. Its gradient must pass through to θ.

Balance matters. If the discriminator becomes perfect too quickly, the generator gets an unhelpful signal. If it is too weak, the generator learns cheap artifacts instead of the real distribution. Learning rates, update ratios, normalization, and architecture all influence that balance.

06

Good samples can still hide a bad model

GAN training can look successful while missing the actual goal. A few sharp images do not prove that the generator learned the full data distribution.

Coverage of the real distribution

Generated samples cover every visible mode of the real distribution.

Oscillation

The networks circle each other without settling. Losses move, but neither sample quality nor coverage improves consistently.

Discriminator imbalance

One player dominates. A near-perfect discriminator can starve the generator, while a weak one rewards obvious artifacts.

Evaluation needs both quality and diversity. Visual inspection is useful, but metrics such as FID are used to compare feature distributions across many samples. They are diagnostics, not proof that a model is safe or useful.

z→ G x^→ D learning signal for G\mathbf{z} \xrightarrow{\,G\,} \hat{\mathbf{x}} \xrightarrow{\,D\,} \text{learning signal for } G

The whole system is one feedback loop. Noise gives the generator room to vary, the discriminator turns real data into a lesson, and alternating optimization keeps both players learning from the latest opponent.

Check your understanding

1. What does the generator receive as its input?
2. What is the discriminator trained to estimate?
3. Why is the practical generator loss often -log D(G(z))?
4. What is mode collapse?
5. During a generator update, what happens to the discriminator?

Self-contained GAN course. Equations render with KaTeX. Interactive diagrams stay on this page.