Generative Adversarial Networks: A Two-player game

Introduced in 2014 by Goodfellow. et al, Generative Adversarial Networks (GANs) revolutionized the field of Generative modeling. They proposed a new framework that generated very realistic synthetic data trained through a minimax two-player game.

With GANs, we don't explicitly learn the distribution of the data $p_{\text{data}}$, but we can still sample from it. Like VAEs, GANs also have two networks: a Generator and a Discriminator that are trained simultaneously.

A latent variable is sampled from a prior $\mathbf{z}\sim p(\mathbf{z})$ and passed through the Generator to obtain a fake sample $\mathbf{x}=G(\mathbf{z})$. Then the discriminator performs an image classification task that takes in all samples and classifies it as real ($1$) or fake ($0$). These are trained together such that while competing with each other, they make each other stronger at the same time.

To summarize,

• A discriminator $D$ estimates the probability of a given sample coming from the real dataset. It works as a critic and is optimized to tell the fake samples from the real ones.

• A generator $G$ outputs synthetic samples given a noise variable input $\mathbf{z}$. It is trained to capture the real data distribution (want $p_G\approx p_{\text{data}}$) so that its generative samples can be as real as possible, or in other words, can trick the discriminator to output higher probabilities.

Training Objective

We want to make sure that the Discriminator's decision over the real data is accurate, i.e. $D(\mathbf{x})=1$. This can be achieved by maximizing the log-likelihood over the data. $$\begin{array}{}\text{(1)}& \underset{D}{max}{\mathbb{E}}_{\mathbf{x}\sim p_{\text{data}}}[\log D(\mathbf{x})]\end{array}$$

The Discriminator should also assign low probabilities to the fake samples generated by the Generator, i.e. $D(G(\mathbf{z}))=0$. In other words, minimize the log-likelihood of the fake data. $$\begin{array}{}\text{(2.1)}& \underset{D}{min}{\mathbb{E}}_{\mathbf{z}\sim p(z)}[\log (D(G(z)))]\end{array}$$ which can be re-written as, $$\begin{array}{}\text{(2.2)}& & \underset{D}{max}{\mathbb{E}}_{\mathbf{z}\sim p(z)}[\log (1-D(G(z)))]\end{array}$$

On the other hand, the Generator wants to fool the Discriminator such that it classifies fake samples as real, i.e. $D(G(\mathbf{z}))=1$. This can be obtained by maximizing the log-likelihood over the fake data. $$\begin{array}{}\text{(3.1)}& \underset{G}{max}{\mathbb{E}}_{\mathbf{z}\sim p(z)}[\log (D(G(z)))]\end{array}$$ which can be re-written as, $$\begin{array}{}\text{(3.2)}& \underset{G}{min}{\mathbb{E}}_{\mathbf{z}\sim p(z)}[\log (1-D(G(z)))]\end{array}$$

The full objective is given by,

$$\begin{array}{}\text{(4)}& \underset{G}{min}\underset{D}{max}{\mathbb{E}}_{\mathbf{x}\sim p_{\text{data}}}[\log D(\mathbf{x})]+{\mathbb{E}}_{\mathbf{z}\sim p(z)}[\log (1-D(G(z)))]\end{array}$$

The objective of training GANs is therefore a minimax game: the generator $G$ is trying hard to trick the discriminator, while the critic model $D$ is trying hard not to be cheated. In this process, both networks force each other to improve their functionalities. And both are trained together with alternating gradient updates (gradient ascent on discriminator and gradient descent on generator).

However, at the start of training, the generator is really bad due to which $D(G(\mathbf{z}))$ is close to zero, creating a vanishing gradients problem for the Generator (as loss = 0). Therefore, instead of $\underset{G}{min}{\mathbb{E}}_{\mathbf{z}\sim p(z)}[\log (1-D(G(z)))]$, we use $\underset{G}{min}{\mathbb{E}}_{\mathbf{z}\sim p(z)}-[\log (D(G(z)))]$ or just $\underset{G}{max}{\mathbb{E}}_{\mathbf{z}\sim p(z)}[\log (D(G(z)))]$ (Equation 3.2) as an objective for the Generator.

Training Algorithm

Optimality

Let's write Equation 4 with a change of variables (ignoring the improvement of Generator for now), $$\begin{array}{rl}& \underset{G}{min}\underset{D}{max}{\mathbb{E}}_{\mathbf{x}\sim p_{\text{data}}}[\log D(\mathbf{x})]+{\mathbb{E}}_{\mathbf{z}\sim p(z)}[\log (1-D(G(z)))]\\ & =\underset{G}{min}\underset{D}{max}{\mathbb{E}}_{\mathbf{x}\sim p_{\text{data}}}[\log D(\mathbf{x})]+{\mathbb{E}}_{\mathbf{x}\sim p_G}[\log (1-D(\mathbf{x}))]\\ & =\underset{G}{min}{\int }_{\mathbf{x}}\underset{D}{max}(p_{\text{data}}(\mathbf{x})\log D(\mathbf{x})+p_G(\mathbf{x})\log (1-D(\mathbf{x})))\\ \\ & \text{For the max function, we take the derivative wrt }D\text{ and assign it to zero.}\\ & \frac{p_{\text{data}}(\mathbf{x})}{D(\mathbf{x})}-\frac{p_G(\mathbf{x})}{(1-D(\mathbf{x}))}=0\Rightarrow D_{max}(\mathbf{x})=\frac{p_{\text{data}}(\mathbf{x})}{p_{\text{data}}(\mathbf{x})+p_G(\mathbf{x})}\text{ [Optimal Discriminator]}\\ \\ & =\underset{G}{min}{\int }_{\mathbf{x}}(p_{\text{data}}(\mathbf{x})\log \frac{p_{\text{data}}(\mathbf{x})}{p_{\text{data}}(\mathbf{x})+p_G(\mathbf{x})}+p_G(\mathbf{x})\log \frac{p_G(\mathbf{x})}{p_{\text{data}}(\mathbf{x})+p_G(\mathbf{x})})\\ & =\underset{G}{min}({\mathbf{E}}_{\mathbf{x}\sim p_{\text{data}}}[\log \frac{p_{\text{data}}(\mathbf{x})}{p_{\text{data}}(\mathbf{x})+p_G(\mathbf{x})}]+{\mathbf{E}}_{\mathbf{x}\sim p_G}[\log \frac{p_G(\mathbf{x})}{p_{\text{data}}(\mathbf{x})+p_G(\mathbf{x})}])\\ \\ & \text{Multiply and divide by 2}\\ & =\underset{G}{min}({\mathbf{E}}_{\mathbf{x}\sim p_{\text{data}}}[\log \frac{2p_{\text{data}}(\mathbf{x})}{p_{\text{data}}(\mathbf{x})+p_G(\mathbf{x})}]+{\mathbf{E}}_{\mathbf{x}\sim p_G}[\log \frac{2p_G(\mathbf{x})}{p_{\text{data}}(\mathbf{x})+p_G(\mathbf{x})}]-\log 4)\\ \\ & \text{From the definition of KL-Divergence: }{\mathbf{D}}_{KL}(p||q)={\mathbf{E}}_{x\sim p}[\log \frac{p(x)}{q(x)}]\\ & =\underset{G}{min}({\mathbf{D}}_{KL}\left[p_{\text{data}}||\frac{p_{\text{data}}+p_G}{2}\right]+{\mathbf{D}}_{KL}\left[p_G||\frac{p_{\text{data}}+p_G}{2}\right]-\log 4)\\ \\ & \text{From Jensen-Shannon divergence: }{\mathbf{D}}_{JS}(p||q)=\frac{1}{2}{\mathbf{D}}_{KL}(p||\frac{p+q}{2})+\frac{1}{2}{\mathbf{D}}_{KL}(q||\frac{p+q}{2})\\ & =\underset{G}{min}(2{\mathbf{D}}_{JS}(p_{\text{data}}||p_G)-\log 4)\end{array}$$

JS divergence is always non-negative and is zero only when the two distributions are equal. Hence, training a GAN with this objective, minimizes the JS divergence to zero, obtaining a global minimum when $p_{\text{data}}=p_G$.

The global minimum of the minimax game happens when, $$\begin{array}{rl}\text{Optimal Generator: }& p_{\text{data}}=p_G\\ \text{Optimal Discriminator: }& D_G^{\ast }(\mathbf{x})=\frac{p_{\text{data}}(\mathbf{x})}{p_{\text{data}}(\mathbf{x})+p_G(\mathbf{x})}=\frac{1}{2}\end{array}$$

Some prominent papers,

1. Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks (DCGAN): Architectures of Generator and Discriminator created with CNNs.


2. Wasserstein GAN (WGAN): Improvement over traditional GAN training.


3. Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks (CycleGAN): For image-to-image translation