Colorizing photos with a GAN-based pix2pix approach

What's actually being asked

Map "black-and-white → colour" onto a conditional GAN image-to-image translation pipeline, and specify the generator, discriminator, training loop, and expected output precisely.

Answer

Framing as image-to-image translation

Domain A = grayscale photo, domain B = its colour version. This is a conditional GAN problem, condition y = grayscale input: min_G max_D 𝔼[log D(x|y)] + 𝔼[log(1 − D(G(z|y)|y))], where x is the real colour image. A nice property of colorization specifically: paired training data is trivial to construct — take any existing colour photo, desaturate it to get the grayscale/colour pair, no manual labelling needed.

Generator

A U-Net encoder–decoder (as in the diagram): the encoder downsamples the grayscale input, extracting multi-scale structural features (edges, object boundaries, textures); the decoder upsamples back up, producing colour output. Skip connections carry the encoder's fine spatial detail straight to the matching decoder layer, so edges stay sharp and don't get "smeared" as colour is added — crucial, since colour should fill in regions without blurring their boundaries. Output is a full colour image (or just the missing chrominance channels on top of the given luminance, depending on the formulation).

Discriminator

A PatchGAN, conditioned on the grayscale input: it classifies small overlapping patches of the pair (grayscale, colour) as real or fake, rather than scoring the whole image once. Real pairs are (grayscale, its true original colour image); fake pairs are (grayscale, G's colorized output). Judging locally makes it sensitive to implausible local colour cues — colour bleeding across edges, flat/desaturated patches — which a whole-image classifier would miss.

Training process

Alternate the usual two steps:

1. Train D: maximize log D(real pair) + log(1 − D(fake pair)) — push it to separate genuine colourizations from generated ones.
2. Train G: maximize log D(fake pair) (non-saturating form, for stronger early gradients), plus minimize an L1 term λ·‖x − G(y)‖₁ against the true colour image. The adversarial term pushes G toward sharp, confident, realistic colour choices; the L1 term anchors it to the right overall colours and prevents it drifting into a plausible-but-wrong palette.

Expected transformation

Input: a grayscale photograph. Output: a plausible, realistic colour version with exactly the same structure/edges as the input (since the generator never alters spatial content, only adds colour), with colours inferred from learned priors about what typically occupies a scene (sky tends blue, grass tends green, skin tends warm tones).

Exam tip

Note the trap: the model produces a plausible colorization, not necessarily the photo's true original colour — luminance alone genuinely doesn't contain enough information to recover the exact original colour of an ambiguous object (e.g. a shirt that could have been any colour). Examiners like this nuance.