Overview · 2:35
How AI lip sync works and Hedra results
Image/video + audio = talking character; Hedra examples.
- Lip-sync = image/video + audio → talking character.
- Judge mouth, head and body movement.
- Hedra results first.
| Criterion | Look for |
|---|---|
| Mouth | Accurate sync |
| Movement | Natural head/body |
| Artifacts | Glitches |
Do this in your own tools, step by step. Ticks are saved on this device.
Inputs?
Image/video and audio.
Judge?
Mouth, movement, artifacts.
0:00 I'll show you the results, a quick walkthrough of how to use each tool, and we'll take a peek at the pricing. And what you'll see are first results only. No cherry-picking, no rerolls, just what the tool gave me on the first run. Let's get going. Starting with Hedra. >> Bob asked me to say something serious, but all I can think about is how my shirt looks like the wallpaper in a dentist's waiting room. >> Felt like things got fuzzy in the mouth area there a little bit, but his head movement, eyes, and even his upper body and shirt collar moving looked good.
0:45 >> I think it's exploring the duality of existence. Or maybe the artist just dropped a smoothie on the canvas. Either way, it's kind of brilliant. There were a couple of misses there with the lips matching the words and some fuzzy teeth briefly, but the expressiveness looked good. Next, we have a cartoon style character. >> I'm not saying I'm keeping score, but if I were, you'd definitely be in second place. And I never explain who's in first.
1:08 >> With a cartoon, the mouth movement doesn't have to be as precise as with a photo style subject. Hedra did a great job with the expressiveness beyond the character's mouth or body language and whatnot. It wasn't stiff, but it wasn't overdone either. I just realized if the humans leave through the front door, they could return through the back. This changes everything. He's moving an awful lot, but his mouth is just kind of open. Not really liking this one.
1:32 Many, if not most of the AI lip-s sync tools really struggle with pets, like dogs or cats, as well as cartoons and other nonhuman subjects. So, I didn't really expect much when trying these, but folks often ask me about lip-syncing pets or cartoon characters or singing. And speaking of singing, that's what we're going to have our next test character doing. Singing a little song I made up in >> a good title and thumbnail, too. Got to make the hook and intro shine.
2:00 And maybe they'll watch till minute. A few million views or maybe 10. [Music] Overall, that was pretty good. It looks like it's not making the right mouth movement for the F sounds. F as in foxtrot, and it had a little trouble at the end carrying out that last note, but the vocal there at the end fades out quietly over the music, so I can't really complain about that. It might have done better if I'd extracted the vocals and just used the vocals for my audio source.
2:23 To create a lip-s sync on Hedra, just click on the video button. You can click this start frame button to create an AI generated image, capture an image using your webcam, or upload an image. Or you can just drag and drop
Get the other 12 chapters, the exam and your certificate for LinkedIn, plus all 700+ AI courses.