Pixar Used AI to Stoke the Flames in Elemental
by Tanya Kalinina |
Aug 16, 2024 3:21 PM
Aug 16, 2024 3:21 PM
Get a FREE quote
Let us understand what exactly do you need to provide the best service
You'll get a response from the Banano team within 24-hours.
The Challenge of Animating Fire
The idea of animating elemental characters like fire and water presented unique challenges for Pixar’s animators. While the concept of a sentient mound of dirt or a body made of water could be resolved with creative animation and suspension of disbelief, fire proved particularly tricky. Ember, one of the film’s main characters, needed to look fiery yet human, with expressive features and an engaging personality.
Pixar’s traditional fire simulations, while realistic, didn’t translate well to character design. Early attempts made Ember look like a ghostly or demonic figure—a terrifying mix of Calcifer from Howl’s Moving Castle and Nicolas Cage’s Ghost Rider. “It can look horrifying if it’s too realistic, like you actually have a human figure made of real pyro,” explained Paul Kanyuk, Pixar’s crowds technical supervisor.
Adding to the challenge, fire is inherently dynamic and unpredictable. Slowing it down for animation risked making it appear more like plasma than fire, according to director Peter Sohn. And with every shot requiring detailed effects passes, creating a believable and aesthetically pleasing fire character seemed both time-consuming and expensive.
Pixar’s traditional fire simulations, while realistic, didn’t translate well to character design. Early attempts made Ember look like a ghostly or demonic figure—a terrifying mix of Calcifer from Howl’s Moving Castle and Nicolas Cage’s Ghost Rider. “It can look horrifying if it’s too realistic, like you actually have a human figure made of real pyro,” explained Paul Kanyuk, Pixar’s crowds technical supervisor.
Adding to the challenge, fire is inherently dynamic and unpredictable. Slowing it down for animation risked making it appear more like plasma than fire, according to director Peter Sohn. And with every shot requiring detailed effects passes, creating a believable and aesthetically pleasing fire character seemed both time-consuming and expensive.
The AI Breakthrough
Paul Kanyuk, a longtime Pixar animator, had an idea. Drawing on research from the Association for Computing Machinery’s Siggraph—a community dedicated to advancing computer graphics—he proposed using neural style transfer (NST) to solve the flame conundrum. NST is a form of AI that combines two visual styles, often used to make photos resemble famous artworks. For Elemental, Kanyuk saw the potential to combine naturalistic fire simulations with artist-designed flames to create a unique look for Ember.
Kanyuk pitched his idea to Sohn, warning that there was only a 50% chance it would work. Sohn’s response? “Let’s do all of them,” he said.
Kanyuk enlisted the help of Disney Research Studios, which specializes in AI and machine learning. Together, they created a solution: Jonathan Hoffman, a Pixar artist, designed a set of stylized, swirling flames dubbed the “fleur-de-lis.” Using NST, these artistic flames were combined with Pixar’s natural fire simulations. The result was a character that moved and flickered like fire but retained Pixar’s signature artistic touch.
“Once you apply a style transfer to naturalistic fire, you can actually start to direct its style and bring the artist’s hand into something otherwise untouchable,” said Jeremie Talbot, co-character supervisor. This method allowed animators to harmonize Ember’s expressive features with the dynamic qualities of fire.
Kanyuk pitched his idea to Sohn, warning that there was only a 50% chance it would work. Sohn’s response? “Let’s do all of them,” he said.
Kanyuk enlisted the help of Disney Research Studios, which specializes in AI and machine learning. Together, they created a solution: Jonathan Hoffman, a Pixar artist, designed a set of stylized, swirling flames dubbed the “fleur-de-lis.” Using NST, these artistic flames were combined with Pixar’s natural fire simulations. The result was a character that moved and flickered like fire but retained Pixar’s signature artistic touch.
“Once you apply a style transfer to naturalistic fire, you can actually start to direct its style and bring the artist’s hand into something otherwise untouchable,” said Jeremie Talbot, co-character supervisor. This method allowed animators to harmonize Ember’s expressive features with the dynamic qualities of fire.
Scaling the Solution
Initially, Pixar planned to use NST only for close-up shots of Ember due to its high computational demands. Rendering even a single frame took five minutes, making it impractical for broader use. However, the animators realized they couldn’t limit the technique to Ember alone—other fire characters in the film needed to match her style.
To meet the increased demands, Pixar leveraged the computing power of its in-house GPUs. By virtualizing the GPUs on every Pixar animator’s computer and running processes overnight, the team reduced rendering times dramatically—from five minutes per frame to just one second.
This breakthrough allowed Pixar to apply NST across all 1,600 shots in the film, ensuring a consistent and visually stunning portrayal of its fire characters.
To meet the increased demands, Pixar leveraged the computing power of its in-house GPUs. By virtualizing the GPUs on every Pixar animator’s computer and running processes overnight, the team reduced rendering times dramatically—from five minutes per frame to just one second.
This breakthrough allowed Pixar to apply NST across all 1,600 shots in the film, ensuring a consistent and visually stunning portrayal of its fire characters.
The Result: Art Meets Technology
For Sohn, the process symbolized Pixar’s unique blend of artistry and innovation. By merging AI with traditional animation techniques, the studio created something entirely new while staying true to its storytelling roots. “It’s this coming together of left brain and right brain,” Sohn explained, “using technology as a tool to help express emotions.”
The result was a stylized, emotive Ember who embodied Pixar’s mission to push the boundaries of animation. NST allowed the team to transform fire—a chaotic, unmanageable element—into a character audiences could connect with emotionally.
The result was a stylized, emotive Ember who embodied Pixar’s mission to push the boundaries of animation. NST allowed the team to transform fire—a chaotic, unmanageable element—into a character audiences could connect with emotionally.
A Glimpse Into the Future
While Pixar is still exploring the full potential of neural style transfer, its success in Elemental marks a significant milestone in animation. “We’re scratching the surface of what NST can do,” Kanyuk said, “but I’m very excited that we found a use case on Elemental that elevated the kind of imagery we can create.”
Through the innovative use of AI, Pixar not only solved one of its toughest animation challenges but also demonstrated how technology can enhance storytelling. In Elemental, Ember doesn’t just burn brightly—she sets a new standard for what’s possible in animated filmmaking.
Through the innovative use of AI, Pixar not only solved one of its toughest animation challenges but also demonstrated how technology can enhance storytelling. In Elemental, Ember doesn’t just burn brightly—she sets a new standard for what’s possible in animated filmmaking.