Transcription
In this video I'm going to show you how to create a digital clone of anyone. This technique is incredibly realistic and has the power to unlock a ton of creative possibilities.
Wait, who are you?
I'm your digital clone. I can do your tutorials now.
Oh, okay um I think I'll take it from here.
Fine, I'll just go back to texting your family members.
Okay, so first things first, the tool that we're going to be using to pull this off is Haen. Haen is an incredible tool that allows you to do everything from language translations to cloning digital replicas, and they recently came out with their Haen 3.0 model that allows you to create realistic clones of just about anyone as long as you have their consent.
Haen gives you some of the best results and it's also one of the easiest tools to use, so let me show you how to get it done in this quick tutorial.
There are seven quick steps that you need to know.
Step one is to go to the Haen website and select the plan that works for you. The Creator plan is great; you get videos up to 5 minutes long and you can export videos up to 1080p. You also have the ability to use language translations and just get a lot out of some of the other secondary features from Haen.
Now, the Curious Refuge team uses the team account, which is able to scale, and if you're working with a really large organization, they have Enterprise models that are available to you.
Step two. Once you're logged into Haen, go to the Haen website and then click on that little avatars button on the far left. You’ll see this little section here that says "Video Avatar." Go ahead and click "Create Avatar." Go ahead and click on the "Get Started" button and select "Motion."
So I'll go ahead and click "Text Instructions." It's actually pretty straightforward. Basically, they need video footage of your subject looking at the camera and talking. Now your subject can be expressive, especially with their hand movements; that really helps to basically sell the overall video that you're trying to create.
Now comes the fun part. You need to create some training footage to create this clone. So, I used a video example like the video that you're watching right now. Basically, it's on a tripod, the subject is well lit, and they are centered in the frame.
The other note that I should add is you need to have space between the sentences. So for example, if I was training the data, it might sound like this: "Hi, my name is Caleb and in this video I am training the Haen Avatar. This video is really all about creating processes that will help us to have the most expressive and realistic Avatar possible."
So there you go. As you can see, there's some breaks between the sentences; it allows the conversations to just come across as more natural. The video doesn't have to be on a professional background like this; it could be in a studio, it could be outside—really, you have creative control over the type of footage that you're working with.
So once you have your footage together, be sure to go to the next step.
And now, all you have to do is either upload your footage or paste in a Google Drive link. Now go ahead and select all of these checkboxes— they're just checkpoints to make sure that the video looks good.
And this is true—there's also this checkbox here that says "Keep ambient sounds in my recording." You would select that if your footage was like on location and there was background noise; that helps to sell the overall clone. So for example, if your subject was in a park and you needed the birds chirping in the background, but you don't have to select that because we're in a studio environment.
I'm not going to check that box and I'm going to select the 4K resolution just because our footage was in 4K. You have to be on the Enterprise plan to get 4K avatars, but might as well go ahead and select that just so you have the option to scale in the future as long as your training footage is in 4K.
Then go ahead and click "My footage looks good."
So the next step is really important. You basically have to have your subject give consent to create the clone. They will need to be in front of a camera with a webcam to record the consent.
Now, the cool thing is they do give you the option to record with a phone, so you can send that link to someone else who is not physically on location with you. And they do have the option to upload consent if you're on an Enterprise model, but it entirely just depends on the workflow and the dynamics of you and the talent and the entire production process.
So now Haen will create your avatar in the background. In our test, it takes about 10 to 15 minutes to create an avatar from about 3 minutes of training data.
Once your avatar is ready, just go back to the Haen website, and that brings us to step three: selecting your avatar. Just click on "Avatars" on the far left and you can select the avatar that you're looking to use.
I should also note that they do have a ton of professional avatars down here that are pretty interesting, and they also have different looks. So if you're working on a project and it doesn't matter who the subject is specifically and there's more flexibility there, you do have a few different options in their library.
But for our example, I'll go ahead and select me. You can also see that you can add up to 100 different looks, so this could be a different background, environment, expression—it entirely depends on the types of projects that you are working on.
So now, in step four, I'm going to actually select our avatar here and I'm going to select the one that was generated, and we will go to "Create with AI Studio."
Now you have the ability to select landscape or portrait. I'll go ahead and select landscape.
So that brings us to step five, which is editing your script. You have a few different options here. The first is you can just type in text and get it to generate the audio, very similar to using AI audio tools like 11Labs.
But one of my favorite features is the ability to upload audio. All you have to do to upload an audio clip is just select this little icon right here, and you can drag and drop your audio into the audio section here.
I have an audio clip from earlier, so I'll just go ahead and select that and bring it in here. You can see that it creates a timeline, so you could do a combination of both audio that was spoken and audio that was automatically generated—it's entirely up to you.
I should also note that you do have the ability to click this "Translate" button to translate your script into a different language.
And that brings us to step number six. When you're ready to create your video, just go ahead and click the "Submit" button. After a few minutes, Haen will generate your video. You'll get an email where you can download the video, or you can just go into your Haen library to download the clip.
So let's take a look at some of our examples here. So this is the first example. This just used a script alone, so just typing in text to generate the video, and here's the result.
Now, on the other side of the park, you also have Disney's California Adventure. DCA is one of my favorite places to hang out and just get a Starbucks and walk around. So that looks pretty good! I think that the non-verbal communication wasn't too bad. It wasn't perfect, but it was pretty good. But the thing that really bothered me about that video was really just the audio. The audio got a little flat and there wasn't that kind of expressive, emotive intonation that you would hope to see from really professional videos.
So I did another test where basically I uploaded the audio, and this is the result.
Now on the other side of the park, you also have Disney's California Adventure. DCA is one of my favorite places to hang out and just get a Starbucks and walk around.
Okay, so not too bad. Obviously, if you meticulously look at the details, there are some funny things about the lips and the non-verbal communication cannot be aligned in some small parts. But generally speaking, it did an amazing job.
We did our own audience testing with Curious Refuge, where we showed various audience members one video clip that was real live action footage and another video clip that was a Haen avatar. It’s basically a 50/50 split between people that can identify the real one versus the fake one, which is basically a coin flip.
I think that this specific tool is going to impact the world of creativity in a few very specific ways. The first is simply training content and everyday utility videos that need to happen and maybe require someone like a CEO or talent that people are used to seeing on camera. So, of course, you could create an AI avatar and create repetitive videos like that.
I also think that as technology progresses and you pair this with real-time conversation tools, chances are in the next two to three years, we'll be able to have conversations with AI avatars that basically feel completely realistic.
However, I should note that these avatars are still very limited. If I was doing a video tutorial like this, I would probably turn my head to the side and really emphasize certain points, and that would make the entire video feel more human and connected to the audience.
But with these AI avatars, you have to have someone that's a bit more of a talking head that's being expressive, but they can't be overly expressive. And I should also note that we did a video clip where we put the camera to the side, basically to simulate a two-camera setup like you might do on location, and Haen was not able to clone me from an angle.
So it seems like Haen Avatar 3.0 really works when the subject is just kind of looking directly at the camera and the environment is optimal.
I think that when you pair Haen Avatar 3.0 with audio that is actually uploaded by your subject, that is where you get really incredible footage.
So if you're working with talent that has a very busy schedule or it's hard to get them into a studio, you could simply have them record audio on location, even on their phone, and you can upscale using AI tools like the Adobe Podcast Enhance tool to make the audio sound super nice, and then pair that with Haen, put it together, and you will have an incredible video.
Thank you so much for watching this video. Obviously, this is the worst version of Haen and cloning that will ever exist—it's only going to get better from here. If you want to learn more about creating digital avatars and what it means to create AI documentaries in this new era of storytelling, be sure to check out our AI documentary course over at Curious Refuge.
Thank you so much for watching. Be sure to like and subscribe if you want to get the latest tutorials and AI news directly here on the platform. We'll see you in the next one!