This tutorial assumes that you are comfortable with these prerequisites.<\/p>\n\n\n\n
Introduction to GANs<\/h2>\n\n\n\n
GANs, introduced by Ian Goodfellow and his colleagues in 2014, consist of two neural networks: the Generator and the Discriminator. These two networks compete against each other in a game-theoretic framework. The Generator tries to create realistic data, while the Discriminator attempts to distinguish between real and fake data. Over time, both networks improve, resulting in a Generator capable of producing highly realistic data.<\/p>\n\n\n\n
Basic Structure of GANs<\/h3>\n\n\n\n\n- Generator<\/strong>: Takes random noise as input and generates data.<\/li>\n\n\n\n
- Discriminator<\/strong>: Takes data (either real or generated) as input and outputs a probability that the data is real.<\/li>\n<\/ol>\n\n\n\n
The Generator and Discriminator are trained together in a two-player minimax game:<\/p>\n\n\n\n
\n- The Discriminator is trained to maximize the probability of assigning the correct label to both real and generated data.<\/li>\n\n\n\n
- The Generator is trained to minimize the probability that the Discriminator assigns the correct label to the generated data.<\/li>\n<\/ul>\n\n\n\n
Mathematically, the objective can be expressed as:<\/p>\n\n\n
$latex \\min_G \\max_D V(D, G) = \\mathbb{E}{x \\sim p{\\text{data}}(x)} [\\log D(x)] + \\\\
\n\\mathbb{E}_{z \\sim p_z(z)} [\\log(1 – D(G(z)))]&s=2$<\/p>\n\n\n\n
Where:<\/p>\n\n\n\n
\n![\"G\"](\"https:\/\/s0.wp.com\/latex.php?latex=G&bg=ffffff&fg=000&s=2&c=20201002\")
is the Generator.<\/li>\n\n\n\n![\"D\"](\"https:\/\/s0.wp.com\/latex.php?latex=D&bg=ffffff&fg=000&s=2&c=20201002\")
is the Discriminator.<\/li>\n\n\n\n![\"p_{\text{data}}(x)\"](\"https:\/\/s0.wp.com\/latex.php?latex=p_%7B%5Ctext%7Bdata%7D%7D%28x%29&bg=ffffff&fg=000&s=2&c=20201002\")
is the distribution of real data.<\/li>\n\n\n\n![\"p_z(z)\"](\"https:\/\/s0.wp.com\/latex.php?latex=p_z%28z%29&bg=ffffff&fg=000&s=2&c=20201002\")
is the distribution of the noise input to the Generator.<\/li>\n<\/ul>\n\n\n\nStep-by-Step Implementation<\/h2>\n\n\n\n
We will implement a simple GAN to generate handwritten digits similar to those in the MNIST dataset. The MNIST dataset contains 28×28 grayscale images of handwritten digits (0-9).<\/p>\n\n\n\n
Step 1: Setup and Dependencies<\/h3>\n\n\n\n
First, we need to install and import the necessary libraries. We will use TensorFlow for building and training our GAN.<\/p>\n\n\n
import<\/span> tensorflow as<\/span> tf\nfrom<\/span> tensorflow.keras import<\/span> layers\nimport<\/span> numpy as<\/span> np\nimport<\/span> matplotlib.pyplot as<\/span> plt\n\n# Ensure we are using TensorFlow 2.x<\/span>\nassert<\/span> tf.__version__.startswith('2'<\/span>)\n\n# Set random seed for reproducibility<\/span>\nnp.random.seed(42<\/span>)\ntf.random.set_seed(42<\/span>)<\/code><\/span>Code language:<\/span> Python<\/span> (<\/span>python<\/span>)<\/span><\/small><\/pre>\n\n\nStep 2: Data Preparation<\/h3>\n\n\n\n
We will use the MNIST dataset, which is available in TensorFlow Datasets. The dataset will be normalized to have values between -1 and 1, which is a common practice when training GANs.<\/p>\n\n\n
# Load and preprocess the MNIST dataset<\/span>\n(train_images, _), (_, _) = tf.keras.datasets.mnist.load_data()\ntrain_images = train_images.reshape(train_images.shape[0<\/span>], 28<\/span>, 28<\/span>, 1<\/span>).astype('float32'<\/span>)\ntrain_images = (train_images - 127.5<\/span>) \/ 127.5<\/span> # Normalize the images to [-1, 1]<\/span>\n\n# Batch and shuffle the data<\/span>\nBUFFER_SIZE = 60000<\/span>\nBATCH_SIZE = 256<\/span>\n\ntrain_dataset = tf.data.Dataset.from_tensor_slices(train_images).shuffle(BUFFER_SIZE).batch(BATCH_SIZE)<\/code><\/span>Code language:<\/span> Python<\/span> (<\/span>python<\/span>)<\/span><\/small><\/pre>\n\n\nStep 3: Build the Generator<\/h3>\n\n\n\n
The Generator takes random noise as input and produces an image. We’ll use a simple neural network with Dense and Conv2DTranspose layers to generate 28×28 images.<\/p>\n\n\n
def<\/span> make_generator_model<\/span>()<\/span>:<\/span>\n model = tf.keras.Sequential()\n model.add(layers.Dense(7<\/span>*7<\/span>*256<\/span>, use_bias=False<\/span>, input_shape=(100<\/span>,)))\n model.add(layers.BatchNormalization())\n model.add(layers.LeakyReLU())\n\n model.add(layers.Reshape((7<\/span>, 7<\/span>, 256<\/span>)))\n assert<\/span> model.output_shape == (None<\/span>, 7<\/span>, 7<\/span>, 256<\/span>) # Note: None is the batch size<\/span>\n\n model.add(layers.Conv2DTranspose(128<\/span>, (5<\/span>, 5<\/span>), strides=(1<\/span>, 1<\/span>), padding='same'<\/span>, use_bias=False<\/span>))\n assert<\/span> model.output_shape == (None<\/span>, 7<\/span>, 7<\/span>, 128<\/span>)\n model.add(layers.BatchNormalization())\n model.add(layers.LeakyReLU())\n\n model.add(layers.Conv2DTranspose(64<\/span>, (5<\/span>, 5<\/span>), strides=(2<\/span>, 2<\/span>), padding='same'<\/span>, use_bias=False<\/span>))\n assert<\/span> model.output_shape == (None<\/span>, 14<\/span>, 14<\/span>, 64<\/span>)\n model.add(layers.BatchNormalization())\n model.add(layers.LeakyReLU())\n\n model.add(layers.Conv2DTranspose(1<\/span>, (5<\/span>, 5<\/span>), strides=(2<\/span>, 2<\/span>), padding='same'<\/span>, use_bias=False<\/span>, activation='tanh'<\/span>))\n assert<\/span> model.output_shape == (None<\/span>, 28<\/span>, 28<\/span>, 1<\/span>)\n\n return<\/span> model\n\ngenerator = make_generator_model()<\/code><\/span>Code language:<\/span> Python<\/span> (<\/span>python<\/span>)<\/span><\/small><\/pre>\n\n\nStep 4: Build the Discriminator<\/h3>\n\n\n\n
The Discriminator takes an image as input and outputs a scalar indicating whether the image is real or fake. We’ll use a convolutional neural network for this task.<\/p>\n\n\n
def<\/span> make_discriminator_model<\/span>()<\/span>:<\/span>\n model = tf.keras.Sequential()\n model.add(layers.Conv2D(64<\/span>, (5<\/span>, 5<\/span>), strides=(2<\/span>, 2<\/span>), padding='same'<\/span>, input_shape=[28<\/span>, 28<\/span>, 1<\/span>]))\n model.add(layers.LeakyReLU())\n model.add(layers.Dropout(0.3<\/span>))\n\n model.add(layers.Conv2D(128<\/span>, (5<\/span>, 5<\/span>), strides=(2<\/span>, 2<\/span>), padding='same'<\/span>))\n model.add(layers.LeakyReLU())\n model.add(layers.Dropout(0.3<\/span>))\n\n model.add(layers.Flatten())\n model.add(layers.Dense(1<\/span>))\n\n return<\/span> model\n\ndiscriminator = make_discriminator_model()<\/code><\/span>Code language:<\/span> Python<\/span> (<\/span>python<\/span>)<\/span><\/small><\/pre>\n\n\nStep 5: Define the Loss and Optimizers<\/h3>\n\n\n\n
The loss functions for the Generator and Discriminator are crucial for the training process. We will use binary cross-entropy loss for both. The Generator tries to minimize -log(D(G(z)))<\/code>, while the Discriminator tries to maximize log(D(x)) + log(1 - D(G(z)))<\/code>.<\/p>\n\n\ncross_entropy = tf.keras.losses.BinaryCrossentropy(from_logits=True<\/span>)\n\ndef<\/span> discriminator_loss<\/span>(real_output, fake_output)<\/span>:<\/span>\n real_loss = cross_entropy(tf.ones_like(real_output), real_output)\n fake_loss = cross_entropy(tf.zeros_like(fake_output), fake_output)\n total_loss = real_loss + fake_loss\n return<\/span> total_loss\n\ndef<\/span> generator_loss<\/span>(fake_output)<\/span>:<\/span>\n return<\/span> cross_entropy(tf.ones_like(fake_output), fake_output)\n\ngenerator_optimizer = tf.keras.optimizers.Adam(1e-4<\/span>)\ndiscriminator_optimizer = tf.keras.optimizers.Adam(1e-4<\/span>)<\/code><\/span>Code language:<\/span> Python<\/span> (<\/span>python<\/span>)<\/span><\/small><\/pre>\n\n\nStep 6: Define the Training Loop<\/h3>\n\n\n\n
The training loop involves generating random noise, using it to produce fake images with the Generator, and then training both the Generator and Discriminator. We will also periodically generate and save images to monitor the progress of the training.<\/p>\n\n\n
EPOCHS = 50<\/span>\nnoise_dim = 100<\/span>\nnum_examples_to_generate = 16<\/span>\n\n# We will reuse this seed over time (so it's easier to visualize progress in the GIF)<\/span>\nseed = tf.random.normal([num_examples_to_generate, noise_dim])\n\n# Notice the use of `tf.function`<\/span>\n# This annotation causes the function to be \"compiled\".<\/span>\n
The Generator and Discriminator are trained together in a two-player minimax game:<\/p>\n\n\n\n
- \n
- The Discriminator is trained to maximize the probability of assigning the correct label to both real and generated data.<\/li>\n\n\n\n
- The Generator is trained to minimize the probability that the Discriminator assigns the correct label to the generated data.<\/li>\n<\/ul>\n\n\n\n
Mathematically, the objective can be expressed as:<\/p>\n\n\n
$latex \\min_G \\max_D V(D, G) = \\mathbb{E}{x \\sim p{\\text{data}}(x)} [\\log D(x)] + \\\\
\n\\mathbb{E}_{z \\sim p_z(z)} [\\log(1 – D(G(z)))]&s=2$<\/p>\n\n\n\nWhere:<\/p>\n\n\n\n
- \n
Step-by-Step Implementation<\/h2>\n\n\n\n
We will implement a simple GAN to generate handwritten digits similar to those in the MNIST dataset. The MNIST dataset contains 28×28 grayscale images of handwritten digits (0-9).<\/p>\n\n\n\n
Step 1: Setup and Dependencies<\/h3>\n\n\n\n
First, we need to install and import the necessary libraries. We will use TensorFlow for building and training our GAN.<\/p>\n\n\n
import<\/span> tensorflow as<\/span> tf\nfrom<\/span> tensorflow.keras import<\/span> layers\nimport<\/span> numpy as<\/span> np\nimport<\/span> matplotlib.pyplot as<\/span> plt\n\n# Ensure we are using TensorFlow 2.x<\/span>\nassert<\/span> tf.__version__.startswith('2'<\/span>)\n\n# Set random seed for reproducibility<\/span>\nnp.random.seed(42<\/span>)\ntf.random.set_seed(42<\/span>)<\/code><\/span>Code language:<\/span> Python<\/span> (<\/span>python<\/span>)<\/span><\/small><\/pre>\n\n\nStep 2: Data Preparation<\/h3>\n\n\n\n
We will use the MNIST dataset, which is available in TensorFlow Datasets. The dataset will be normalized to have values between -1 and 1, which is a common practice when training GANs.<\/p>\n\n\n
# Load and preprocess the MNIST dataset<\/span>\n(train_images, _), (_, _) = tf.keras.datasets.mnist.load_data()\ntrain_images = train_images.reshape(train_images.shape[0<\/span>], 28<\/span>, 28<\/span>, 1<\/span>).astype('float32'<\/span>)\ntrain_images = (train_images - 127.5<\/span>) \/ 127.5<\/span> # Normalize the images to [-1, 1]<\/span>\n\n# Batch and shuffle the data<\/span>\nBUFFER_SIZE = 60000<\/span>\nBATCH_SIZE = 256<\/span>\n\ntrain_dataset = tf.data.Dataset.from_tensor_slices(train_images).shuffle(BUFFER_SIZE).batch(BATCH_SIZE)<\/code><\/span>Code language:<\/span> Python<\/span> (<\/span>python<\/span>)<\/span><\/small><\/pre>\n\n\nStep 3: Build the Generator<\/h3>\n\n\n\n
The Generator takes random noise as input and produces an image. We’ll use a simple neural network with Dense and Conv2DTranspose layers to generate 28×28 images.<\/p>\n\n\n
def<\/span> make_generator_model<\/span>()<\/span>:<\/span>\n model = tf.keras.Sequential()\n model.add(layers.Dense(7<\/span>*7<\/span>*256<\/span>, use_bias=False<\/span>, input_shape=(100<\/span>,)))\n model.add(layers.BatchNormalization())\n model.add(layers.LeakyReLU())\n\n model.add(layers.Reshape((7<\/span>, 7<\/span>, 256<\/span>)))\n assert<\/span> model.output_shape == (None<\/span>, 7<\/span>, 7<\/span>, 256<\/span>) # Note: None is the batch size<\/span>\n\n model.add(layers.Conv2DTranspose(128<\/span>, (5<\/span>, 5<\/span>), strides=(1<\/span>, 1<\/span>), padding='same'<\/span>, use_bias=False<\/span>))\n assert<\/span> model.output_shape == (None<\/span>, 7<\/span>, 7<\/span>, 128<\/span>)\n model.add(layers.BatchNormalization())\n model.add(layers.LeakyReLU())\n\n model.add(layers.Conv2DTranspose(64<\/span>, (5<\/span>, 5<\/span>), strides=(2<\/span>, 2<\/span>), padding='same'<\/span>, use_bias=False<\/span>))\n assert<\/span> model.output_shape == (None<\/span>, 14<\/span>, 14<\/span>, 64<\/span>)\n model.add(layers.BatchNormalization())\n model.add(layers.LeakyReLU())\n\n model.add(layers.Conv2DTranspose(1<\/span>, (5<\/span>, 5<\/span>), strides=(2<\/span>, 2<\/span>), padding='same'<\/span>, use_bias=False<\/span>, activation='tanh'<\/span>))\n assert<\/span> model.output_shape == (None<\/span>, 28<\/span>, 28<\/span>, 1<\/span>)\n\n return<\/span> model\n\ngenerator = make_generator_model()<\/code><\/span>Code language:<\/span> Python<\/span> (<\/span>python<\/span>)<\/span><\/small><\/pre>\n\n\nStep 4: Build the Discriminator<\/h3>\n\n\n\n
The Discriminator takes an image as input and outputs a scalar indicating whether the image is real or fake. We’ll use a convolutional neural network for this task.<\/p>\n\n\n
def<\/span> make_discriminator_model<\/span>()<\/span>:<\/span>\n model = tf.keras.Sequential()\n model.add(layers.Conv2D(64<\/span>, (5<\/span>, 5<\/span>), strides=(2<\/span>, 2<\/span>), padding='same'<\/span>, input_shape=[28<\/span>, 28<\/span>, 1<\/span>]))\n model.add(layers.LeakyReLU())\n model.add(layers.Dropout(0.3<\/span>))\n\n model.add(layers.Conv2D(128<\/span>, (5<\/span>, 5<\/span>), strides=(2<\/span>, 2<\/span>), padding='same'<\/span>))\n model.add(layers.LeakyReLU())\n model.add(layers.Dropout(0.3<\/span>))\n\n model.add(layers.Flatten())\n model.add(layers.Dense(1<\/span>))\n\n return<\/span> model\n\ndiscriminator = make_discriminator_model()<\/code><\/span>Code language:<\/span> Python<\/span> (<\/span>python<\/span>)<\/span><\/small><\/pre>\n\n\nStep 5: Define the Loss and Optimizers<\/h3>\n\n\n\n
The loss functions for the Generator and Discriminator are crucial for the training process. We will use binary cross-entropy loss for both. The Generator tries to minimize
-log(D(G(z)))<\/code>, while the Discriminator tries to maximizelog(D(x)) + log(1 - D(G(z)))<\/code>.<\/p>\n\n\ncross_entropy = tf.keras.losses.BinaryCrossentropy(from_logits=True<\/span>)\n\ndef<\/span> discriminator_loss<\/span>(real_output, fake_output)<\/span>:<\/span>\n real_loss = cross_entropy(tf.ones_like(real_output), real_output)\n fake_loss = cross_entropy(tf.zeros_like(fake_output), fake_output)\n total_loss = real_loss + fake_loss\n return<\/span> total_loss\n\ndef<\/span> generator_loss<\/span>(fake_output)<\/span>:<\/span>\n return<\/span> cross_entropy(tf.ones_like(fake_output), fake_output)\n\ngenerator_optimizer = tf.keras.optimizers.Adam(1e-4<\/span>)\ndiscriminator_optimizer = tf.keras.optimizers.Adam(1e-4<\/span>)<\/code><\/span>Code language:<\/span> Python<\/span> (<\/span>python<\/span>)<\/span><\/small><\/pre>\n\n\nStep 6: Define the Training Loop<\/h3>\n\n\n\n
The training loop involves generating random noise, using it to produce fake images with the Generator, and then training both the Generator and Discriminator. We will also periodically generate and save images to monitor the progress of the training.<\/p>\n\n\n
EPOCHS = 50<\/span>\nnoise_dim = 100<\/span>\nnum_examples_to_generate = 16<\/span>\n\n# We will reuse this seed over time (so it's easier to visualize progress in the GIF)<\/span>\nseed = tf.random.normal([num_examples_to_generate, noise_dim])\n\n# Notice the use of `tf.function`<\/span>\n# This annotation causes the function to be \"compiled\".<\/span>\n