Three major GAN training challenges

What's actually being asked

Explain three well-known GAN failure modes, and for each one tie it to a concrete loss-function-level fix, not just a description of the symptom.

Answer

1. Mode collapse

G finds a small set of outputs (sometimes just one) that reliably fools the current D, and keeps producing variations of only those — it stops exploring the rest of p_data's diversity. Matlab, G ne ek "trick" dhoond liya jo kaam kar raha hai, aur ussi pe atak gaya.

Cause: G only has to fool a single snapshot of D each step; finding one easy mode is cheaper than covering the whole distribution, and the standard loss has no explicit term rewarding diversity across samples.

Loss-level fixes: the Wasserstein objective (WGAN) gives a smoother, everywhere-informative distance between distributions, reducing the incentive to hide in one mode; minibatch discrimination feeds D batch-level statistics so it can directly penalize low-diversity batches; unrolled GANs let G anticipate several future steps of D's response before updating, discouraging it from exploiting a short-lived blind spot.

2. Vanishing gradients

With the saturating G loss log(1 − D(G(z))), once D becomes a confident classifier (D(G(z)) ≈ 0), the gradient reaching G's parameters becomes vanishingly small — G stops learning even though it is nowhere close to done.

Cause: D's sigmoid output saturates at the extremes (0 or 1), and log(1−d) is flat there.

Loss-level fixes: switch to the non-saturating G loss −log D(G(z)), which is steep exactly where the saturating loss is flat; or move to an objective whose critic/discriminator isn't a saturating sigmoid classifier at all — e.g. the Wasserstein critic, which stays informative even when it confidently separates real from fake.

3. Training instability (oscillation / non-convergence)

G and D losses oscillate instead of settling — D improves, G catches up, D improves again, sometimes diverging rather than converging toward equilibrium.

Cause: G and D are optimized by alternating gradient steps on a non-convex, moving-target objective; unlike a convex-concave game there's no guarantee of a stable fixed point under simultaneous/alternating updates.

Loss-level fixes: Wasserstein loss with a Lipschitz-constrained critic (weight clipping, or the gradient-penalty variant WGAN-GP) produces a loss value that actually correlates with sample quality and moves smoothly instead of oscillating wildly; LSGAN's least-squares objective similarly penalizes samples by their distance from the decision boundary rather than a binary cross-entropy, giving smoother gradients and fewer wild swings.

Exam tip

For each challenge, structure the answer as: name → one-line cause → a named loss or architectural fix, and close the loop by saying explicitly why that fix addresses that specific cause. Listing symptoms without the mechanism linking cause to fix is the most common way marks are lost on this question.