Understanding VAEs

0:00 Hey guys in this video we're going to learn how to implement VAE and train a VAE model. VAE is variational autoencoder which has huge impact in image generation model like stable diffusion and DALIN. Let's first take a look at like how VAE works and then we'll go to implement it and then train it. So to understand the VAE we need to first understand the autoencoder as it is the base of it. So autoencoder is an unsupervised learning neural network model.

0:30 It is primarily used for dimensionality reduction so let's suppose if we want to train our stable diffusion models we cannot feed the 4k images to train the model which approaches like 8 million pixels to train the neural network which will be very computationally expensive. To solve this problem we will need a model which actually reduces the dimensionality of this 8 billion pixels into some fixed small vector with high rich features of those images. That's where the autoencoder comes in the picture which helps us to encode the image into a latent.

1:35 So latent is a fixed point over the latent space so that's it our problem solved but that is not because autoencoder has some limitations when it comes to generations. So basically it is essentially designed to compress the images into latent and to reconstruct back to the original image so it lacks the ability to generate new and varied outputs. That's where the VAE comes into the picture unlike autoencoder rather than memorizing the patterns. It generates variations of images so the major technical improvement in VAE it does not directly generates the latent.

2:04 First it generates the probability distribution and then converts it into the latent. So by learning and predicting the probability distribution over the latent space the VAE encodes each data points to not a fixed vector but as a range defined by mean and variance. So let's suppose if VAE is trained on 100 digits so the 7 will be in one pool and the 2 will be in another pool. So every digit have a unique pools in the latent space rather than a fixed vector or fixed point it will it will generate a pool of it and then as there are pools it has the possibility to generate the variations of it.

2:36 But unlike in autoencoders where you have only a fixed vector for a specific thing and it is exactly based on what model has seen in the training examples. Suppose if we want to generate the number 7 it will going to generate the same close same as close to as the original one what has seen in the training images. The VAE actually learns this by predicting and learning the probability distribution and in the loss function it actually uses the reconstruction loss and the KL divergence which we're going to look at when we're going to implement this.

3:03 So that's exactly