Snapchat’s On-Device AI Model Transforms User’s Background and Clothing in Real-Time
Snapchat’s upcoming on-device AI model will revolutionize the way users interact with the app by transforming their backgrounds and clothing in real-time. This new technology, which was unveiled in a recent announcement, will allow users to apply custom lenses based on text prompts, resulting in wild and creative looks to share with friends.
Snapchat AI Turns Prompts Into New Lens
Snapchat’s on-device artificial intelligence (AI) model is meticulously crafted to facilitate the transformation of a user’s surroundings through the integration of augmented reality (AR). This innovative model empowers content creators with the ability to convert textual prompts into customized lenses, thereby opening up a world of possibilities for imaginative and visually captivating appearances. In the upcoming months, users will have the opportunity to experience lenses that leverage this groundbreaking model, while content creators will be able to commence the creation of lenses utilizing the model by the end of the current calendar year.
The GIF below demonstrates how this feature works. It shows a person’s clothing and background transforming in real-time based on the prompt “50s sci-fi film.”
How AI Snaps Work
To use the new AI-powered lenses, users simply take a Snap and select a prompt. The app’s AI model then generates an image based on the chosen prompt, which can take up to 20 seconds. The resulting image is then overlaid on the user’s real-world background, creating a seamless and visually impressive AR experience.
Advanced Augmented Reality
In addition to the on-device AI model, Snapchat is also rolling out a suite of new AI tools for creators to make custom augmented reality (AR) effects. These tools, which are now available with the latest Lens Studio update, include new face effects that let creators write a prompt or upload an image to create a custom lens that completely transforms a user’s face.
One of the most exciting new features is called “Immersive ML,” which applies a realistic transformation over the user’s face, body, and surroundings in real-time. This technology allows for more sophisticated and whimsical AR experiences, as demonstrated in the image below using the prompt “Matisse Style Painting.”
Additional artificial intelligence tools integrated with Lens Studio empower lens creators to generate three-dimensional assets based on textual or visual prompts, craft facial coverings and textures, and construct three-dimensional character heads that mirror a user’s emotional expression. These advanced functionalities will enable creators to craft more captivating and interactive augmented reality experiences for Snapchat users.
Snapchat’s on-device AI model and new AI tools for creators represent a significant step forward in the world of augmented reality. By leveraging the power of AI, Snapchat is creating new and exciting ways for users to interact with the app and express themselves creatively. With these new features, Snapchat is positioning itself as a leader in the AR space and setting the stage for even more innovative and engaging experiences in the future.
