Long lost twin!

Overview of GAN - Generative Adversarial Network

GANs as the name suggest are generative ML models used to generate artificial content. They’re popularly used to generate deep fakes, which are equivalent to real pictures. “Adversarial Networks” because under the hood they employ two deep neural networks. One network is used to generate deep fakes and the other network (the adversary) is responsible for detecting deep fakes from real images.

As one can contemplate, to create a good GAN model, both deep neural network for generating deep fakes as well as the adversarial neural network to detect deep fakes should be equally good. If the adversary is not good, then the deep fake generating model won’t know if it’s really able to generate indistinguishable fakes or it’s the adversary’s lack of detecting which is letting it perform well. So, a well trained GAN results in being able to detect half fakes as reals and half reals as fakes.

Getting hands dirty

Similar to every AI/ML model, best way to get up and running is to leverage existing pre-trained models. We leverage NVIDIA’s StyleGAN2 model as our GAN structure and use a bunch of pre-trained models to quickly generate deep fakes.

Setup

Devil’s in the details, but this blog is not meant to be that. So, anyone interested can try to run this colab notebook to see how to use pre-existing GAN models and project one’s own image to a deep fake space.

Below are some interesting points:

• You need NVIDIA GPUs to run the model, so Google’s Colab is our friend here. You can choose a free GPU based machine to run your ML models.
• Most StyleGAN2 models are trained using faces which have eyes centered and photo cropped in a specific manner. To get good results one needs to crop their own real image in the same manner.
  • To aid with this, we leverage another (simpler) deep neural network to identify location of eyes and use that to crop one’s image in the right dimensions.
• We are using pre-trained painting faces model which expects a 1024x1024 input image. Our cropping function in colab notebook should help with this. Depending on the model you might need to change the input resolution.

Results

In line with expectations

Standard Face Dataset - Long lost twin?!

Painting Art Style - Bearable results!

• Note to self: Don’t try to deep fake shades with old artistic paintings. You’ll end up getting dark circles.
• Another note to self: Tame your beard man! You are no longer in the olden ages! :P

Epic fails

Big Brain Time!

• Lack of helmet in the training data set forced the GAN to pickup a shiny pink head!

What’s this monstrosity!

• Shows what can happen if you don’t crop your image in the right proportions.
• So, centre your shit!

Next Up

• We will try to create interpolation between different latent space representation of myself. Should create interesting animation videos.
• Maybe try to create an anime version of myself in one of the available pre-trained models.
• Training my own GAN model.
• One idea could also be to hack way into style transfer. Project real image into latent space, then try to project a new art style from that latent space.
• We can also try to train on our own images by starting from an existing image to create a hybrid model. Follow the youtube link here
• We can also use lucid for style transfer where the style can come from a GAN video of generative art forms. Forming a fluid transformation between different art styles on original pictures.