Standard GAN loss functions for G and D

What's actually being asked

State the minimax value function, split it into what D maximizes and what G minimizes (or maximizes, in the practical variant), and explain why this pairing is what makes the "adversarial" training actually work.

Answer

The minimax value function

min_G max_D V(D,G) = 𝔼_{x∼p_data}[log D(x)] + 𝔼_{z∼p_z}[log(1 − D(G(z)))]

Symbols: D(x) — discriminator's estimated probability that x is real; G(z) — generator's fake sample produced from noise z; p_data — the real-data distribution; p_z — the noise prior (usually N(0,I)).

Discriminator loss

D maximizes: log D(x) + log(1 − D(G(z))). In practice this is implemented as binary cross-entropy — label real samples 1, fake samples 0 — minimizing L_D = −[log D(x) + log(1 − D(G(z)))]. D wants D(x) → 1 on real data and D(G(z)) → 0 on fakes: a straightforward binary classifier.

Generator loss

Two standard forms:

• Saturating (original minimax): G minimizes log(1 − D(G(z))).
• Non-saturating (used in practice): G maximizes log D(G(z)), i.e. minimizes −log D(G(z)).

Why the non-saturating form is preferred: early in training D easily tells real from fake, so D(G(z)) ≈ 0. At that point log(1 − D(G(z))) sits in a flat region of its curve — tiny gradient, G barely learns. −log D(G(z)) is steep exactly when D(G(z)) is small, so it gives G a strong learning signal early on, when it needs it most.

How these losses drive the adversarial process

Matlab, training is a two-player game played with alternating gradient steps:

• D is trained to be the best possible classifier against the current G.
• G is trained to best fool the current D.
• D's gradient tells G which directions in data space currently look "obviously fake" — G nudges its output distribution p_g to erase exactly those tells.
• As this repeats, p_g is pushed toward p_data. At the theoretical equilibrium (Nash equilibrium of the minimax game), p_g = p_data and D can do no better than random guessing — D(x) = 1/2 everywhere, for both real and fake inputs.
• The key coupling: each network's loss is a direct function of the other network's current output. Neither can improve its loss in isolation without implicitly forcing a response from the other — that mutual dependency is the whole mechanism behind "adversarial" training, as opposed to two networks trained independently.

Exam tip

Always write both the saturating and non-saturating Generator loss, and explicitly justify which one is used in practice and why (the flat-gradient region of log(1−d) near d≈0). Stating only one form, or stating both without the justification, is a common way to lose marks here.