Former Pixar story artist Connie He has directed a new animated short at Google DeepMind using a custom generative AI pipeline trained entirely on her own hand-drawn art. As the animation industry faces widespread job anxiety and fierce backlash over AI adoption, her project demonstrates a method for maintaining artistic control rather than relying on automated text-to-video generation.
Building a bridge with developers
He experienced internal conflict when tools like Midjourney and DALL-E first emerged, but decided ignoring the technology was not an option. "The motivation to start jumping in is because I feel like if this technology is going to keep developing, artists need to stay in the conversation rather than shy away," she said. She drew inspiration from an interview where musician Björk defended electronic music, recalling that a tool only lacks soul if the artist fails to put soul into it. This mindset shaped her collaboration at DeepMind, where she worked directly with software engineers to solve technical hurdles. When the AI generated unwanted flickering, she collaborated with a researcher to export masks from Maya, stabilizing the character's face while intentionally allowing the hair and background to flicker to reflect the character's emotional state.
A blueprint for human-led animation
The resulting short, "Dear Upstairs Neighbor," tells a story about a character named Ada dealing with noisy neighbors. The team rejected text-to-video generation entirely, opting instead for a Generative Video pipeline driven by hand-drawn art and purpose-trained machine learning models, known as LoRAs. He painted abstract acrylics on paper, which were scanned to train the custom models and combined with a hybrid 2D and 3D animation process. She said that while traditional computer graphics struggle to avoid a standard CG look, the machine learning model successfully adopted the specific design rules of her paintings.
He acknowledges the valid ethical anxieties surrounding generative AI and recognizes the broader copyright concerns tied to foundational models. She said she hopes for incoming legal regulation, as the current lack of a legal framework makes it difficult to pursue these creative conversations respectfully.
Why this matters for creatives
For professionals in AI for Creatives fields, He's approach offers a practical alternative to the fear of replacement. By treating generative AI as a new medium rather than an autonomous replacement, artists can dictate the output and maintain their creative voice. "To me, it's like a new, fresh box of paint," she said. "Right now, this new box of paint only has a few colours in it for artists to use. There are a lot of blanks. But when that expands, I'm hoping artists can use them to create stuff that used to not be possible."
Your membership also unlocks:
