Deepfakes vs. Faceswaps: A Deep Dive under the Hood

If you’ve been keeping up with my #demystifyAI series, you might recall our earlier discussion on AI avatars versus faceswaps. If you haven’t, go read the article, ‘AI Avatars vs. Faceswaps: Creating Digital Souls, Not Just Masks’, where we explored how AI avatars transcend mere face manipulation, embodying what I like to call a "digital soul."

In that post, I briefly touched upon the term “deepfake.” This term is quiet confusing for people and often used synonymously with faceswap. After all, both involve replacing one face with another! But, peel back the layers, and you’ll find significant differences in their methodology, purpose, and impact. Today, we’re diving headfirst into the distinction between these two concepts. Fair warning: things are going to get really technical from here on! But bear with me if you want to understand this in and out!

So, the term "deepfake" first emerged in 2017 on Reddit, coined by a user who shared AI-generated videos swapping celebrity faces. It combined “deep learning” (a machine learning method) and “fake.” While simple face manipulation tools had existed in some form before, deepfakes brought an entirely new level of realism and sophistication. This technology’s popularity soared as it became easier to access. Open-source tools like FakeApp and later platforms like DeepFaceLab allowed users to generate eerily convincing face swaps. What began as a playful experiment quickly spiraled into controversy, as deepfakes were weaponized for misinformation, non-consensual content, and fraud. Today, “deepfake” is a term loaded with both fascination and caution.

Now with the origin story done and dusted, let's explore how they differ from faceswaps! While faceswaps and deepfakes might share a common goal, i.e., face replacement, the paths they take to get there are fundamentally different. Faceswaps, powered by hubs like Rope and FaceFusion, rely on efficient algorithms designed to detect key facial landmarks, such as the eyes, nose, and mouth, and overlay a different face onto them. These tools use a static image as input, mapping the new face onto the target with impressive precision. They excel in achieving smooth blending and maintaining the integrity of the swapped face’s features, often in real-time. These apps are far more robust than casual filters we witness on Snapchat and Instagram, showcasing the potential of AI in creating high-quality faceswaps for creative or professional use. In simple terms, no training is required! You give them an image of a face and point at a target, and they execute the faceswap right away! Easy as that!

Deepfakes, on the other hand, operate on an entirely different level of complexity. Powered by advanced neural networks, often utilizing frameworks like Generative Adversarial Networks (GANs), deepfakes require a substantial dataset of images and videos of both the source and target faces. This data is fed into a pre-trained model, which learns the facial structure, expressions, and movements of the faces over time. Once trained, the model can generate new, highly realistic content without additional input images. Unlike faceswaps, deepfakes are capable of capturing intricate details, such as micro-expressions and dynamic lighting, making them nearly indistinguishable from reality. Put simply, they kind of do a better job at faceswaps, however prior to the execution phase, they need to be trained on hundreds if not thousands of images of the subject. Execution feels just like doing inference on a pre-trained AI model (you don’t need to provide any more source images during execution), however they consume significant time and compute during training!

And so, the distinction lies not just in the tools but in their underlying methodology and purpose. Faceswaps like those created by Rope and FaceFusion are designed to deliver quality results with minimal input and computational overhead, making them accessible and user-friendly. Deepfakes, however, involve a labor-intensive training process that emphasizes hyper-realism. While faceswaps can be likened to applying a high-quality mask, deepfakes delve into creating a fully realized digital double, capable of mimicking the original face’s nuances with extraordinary accuracy.

Now that you’re up to speed with the difference between the two, let’s get to busting some associated myths:

Is deepfake an AI model? Yes! Unlike a faceswap, a deepfake is indeed an AI model! It is an ‘AI model of the face’ built on a different architecture than the Flux-based text and image-to-image models that you’re used to seeing me working on. Deepfake models, commonly referred to as DFL models due to the popularity of tools like DeepFaceLab and DeepFaceLive, leverage advanced training techniques to produce highly realistic results.

Is deepfake always better than faceswap? Umm, I know I said earlier in the article that they’re better and trust me, they are good! But, nowadays faceswaps are catching up really, really fast! With advancements like pixel boost and the ability to swap multiple angles, FaceFusion is making impressive progress. Rope is also catching up quickly, and both are open source! You will remember me mentioning in one of my posts earlier why I support open source, but there are other significant close-source apps and websites that are doing quite a good job as well!

Is deepfake going to steal your identity? To be honest, a faceswap is more likely to steal your identity in the current age and time, just because it is so easy to do, without requiring any prior planning or loss of time, labour and compute. That’s primarily the reason why deepfakes are kind of losing users to faceswap. Not only do they take significant time and compute to make, but also require significant manual handling of thousands of images and effort to train.

Is deepfake going to take over the world? The answer is a resounding no. However, they have still faced considerable backlash from the public, hosting platforms, and governments alike, largely due to their infamy shaped by controversial media narratives and perceptions of being a technology ahead of its time (when really, these people were just unprepared to handle the ethical dilemmas and misuse that followed). This has led its users to operate in a ‘more closed’ and ‘inconsistent’ manner. You won’t find as many tutorials in the open as compared to other things AI. There are closed repositories and the development is stalled. Information is seemingly available but in a disorganized manner, with forks and dead-ends all over the place. Add to that, there are ‘cooler/ newer’ AI developments that now engage people more, like the text and image-to-video AI models, LLM trainings, AI chatbots, etc. All these ‘cool’ things existed before in some form, however, they’re now gaining significant prominence due to how accessible and user-friendly certain platforms have made them for the end user.

Anyway, I believe understanding this difference is essential not only for appreciating the technology but also for recognizing its implications. By unraveling the nuances of terms like “deepfakes” and “faceswaps,” I aim to empower you with knowledge that sparks curiosity and informed conversations. The journey of understanding AI is as much about the technology as it is about its impact on our lives. Stay tuned as I continue to demystify AI!