Transcription
Today, I'm going to show you how I was able to automate hours of recording content using Claude Code and Hey Gen. And some of you may have already guessed this, but yeah, I'm an AI clone of Nate. A lot of comments lately have been saying that he uses an AI avatar, but that's actually not true. This is my first ever time appearing on this YouTube channel. So, yeah, that was actually an AI clone of me that I made in like 10 minutes. And I have been testing a ton of different methods. If you go through my Hey Gen here, you will see literally hundreds of different avatars and scripts that I've ran through here and different methods, too.
So, in today's video, I'm going to show you how I'm able to use Claude Code as a complete orchestration layer around three different tools that help me basically go from raw script to finished, edited video in minutes. And it's so, so cool.
So, basically what we're doing here is there's three main pieces. We have Hey Gen, which is what we use for the actual clone, the actual avatar. We have 11 Labs, which is what we're using for that voice cloning. And then we have Remotion, which is what we're using to edit everything together and give us professional-looking motion graphics.
So, I'm going to show you guys exactly how I set the system up. But first, let me show you an output. So, I'm going to play some clips right up here, and just let me know what you think.
Lesson 5.0. The audit is how you become a partner. Up to this point in the course, everything you've learned has been about getting in the door, finding clients, starting conversations, running discovery calls that surface real problems, delivering a project that proves you can do the work. And if you've been following along, you may have already done some of this. Maybe you've landed a client through Upwork and delivered your first automation. Maybe you've reached out to your warm network, had a few conversations, and booked a discovery call. You send the invoice, the work wraps up, and you're back to looking for the next one, or the discovery call surfaces a real problem. And you can see that the business has issues in five different places, but you're only talking about one of them. That gap between I did a good job on one thing and I'm helping this business get better over time is where most freelancers get stuck. You've been building those skills through discovery calls, through learning to diagnose workflows, through mapping processes and identifying bottlenecks. The audit applies those same skills at a broader scale.
So, yeah, there might be a few minor imperfections, but the fact that we have the facecam in the lower right and the fact that the script was still written by me, so that's still my raw and real thoughts on the actual tutorial, I don't really see a problem with it. I'm never going to AI avatar this YouTube channel. I like making the videos, and I'm always going to keep doing this. But think about what this means for short-form content or for course material or for advertisements where really in the grand scheme of things, the best content wins.
So, why is this such a big deal now? It's because Hey Gen has created a new model called Avatar 5. Previously, we had Avatar 3 and Avatar 4, which were decent, but they still had, you know, really bad lip movements or really robotic, you know, just gestures. So, for example, if I click into one of these newer ones that I generated, which was using the new Avatar 5, this is what we get: "Your audit engagement kit is now fully fleshed out with four sections: Pitch scripts, interview guide, report template, and retainer transition script." All of those movements there actually look natural, and it learned from me about, you know, like how my head moves or how my lips move, things like that.
But if I go back to an older one that we did without using Avatar 5, you know, maybe this one used Avatar 3. "Lesson 5.0, the audit is how you become a partner. Up to this point in the course, everything you've learned has been about getting in the door, finding clients, starting conversations." It's really not horrible, but if someone knows me and they look at that, they're going to see that my lips are moving weird and that I just feel more like a robot.
So, the first step here is setting up your avatar clone. And I know that we have a few different things going on in this tech stack. So, at the end, I'm going to cover like the pricing and the usage, but let's just continue on with like how you actually start to do this stuff.
So, if I click on avatar up here, you can see that we're able to do a really quick create, and I can choose some of these other looks over here. But if you go to avatar, this is where you can actually generate one. So, you can see that I have a few different ones. I've tried different ones, like I said, and ultimately I liked this one the most because I ended up using this to train it on way more data. But, if I come up here and click on create avatar, you can clone yourself, right? And so, this is where you can record. And my camera's occupied because I'm filming, but it literally gives you a script and you read it off, and it takes 15 seconds, and then you get an Avatar 5 clone that is really, really good. That's the one that you saw in like the YouTube intro. Or what you could do is you could upload footage. So, what I did is I uploaded footage. I uploaded about 10 GB of footage, so it had a lot more data to train on me, and then I created the one where I'm like sitting down in my chair.
So, anyways, once you do that, in a matter of minutes, you have an avatar. Now, it'll automatically clone your voice, but it's just not very good. And even though you can import voices from 11 Labs, I've imported the voice that I'm going to show you guys today from 11 Labs into Hey Gen, but it still doesn't sound the same. So, like right here, I was able to import my voice from uh 11 Labs. So, I went to switch, and I went to import from third party, and I imported it from 11 Labs right here, as you can see. And this is my professional voice clone that I got from 11 Labs. But just take a listen to this: "Hi, I'm excited you're here. This is your voice clone preview. Take a quick listen." Like, that does not sound very good. But it's weird because in 11 Labs, it sounds phenomenal. So, that's why we're kind of doing this multi-stage step.
So, let me pull up 11 Labs now. And by the way, the links to all these tools will be down in the description. But in 11 Labs, what you're going to do is you're going to go to voices, and you're going to create a new voice. So, right here in my voices, you can see that I've got this one, the Nate voice clone. So, when you go to create a voice, you can do a few different options. So, you can have an instant voice clone, which is really quick, super simple, but it's not going to be as good as the professional voice clone. Like, I ended up putting in 2 hours worth of my voice, and that's how I got this thing really, really good. You probably don't need 2 hours, but it says that you need at least 30 minutes. So, you literally will just either speak into it for 2 minutes or upload, you know, a clip of you talking for 2 minutes or however long you need to do it, and then you'll get a voice clone really quick.
And then when you want to use that voice clone, you basically just click on this button over here, and you can enter in the scripts. And this is the one that you guys saw in the intro of this video. And then you can basically just generate it. And you can play with the speed, the stability, the similarity, the style exaggeration, all this kind of stuff. So, let me just show you guys real quick. If I hit generate: "Today, I'm going to show you how I was able to automate hours of recording content using Claude Code and Hey Gen." Okay, cool. So, you guys can tell that that sounds like me. Obviously, you might want to play with these things a little bit and keep testing. I went through tons and tons of different iterations of the best settings, and I got to a place that I feel like sounds the most like me with my inflection and things like that.
So, then basically what we're doing is we're downloading that file. This is the, you know, 16-second clip of this audio. So, if I download this, I have that 11 Labs audio file right here. And then I'm going to go back into Hey Gen. I'm going to go ahead and go to AI Studio. And this is going to let us basically drop in an audio, and it gets synced up to our avatar. So, I'll click on upload audio. I will drag in that 11 Labs file that we just got right here. And then when I add that, it's going to transcribe it. And then we can generate this video using, like I said, that audio file and then our avatar. And this is where you see that we have Avatar 5. We have three, we have four, but five is the new one, and five is the one that is obviously the absolute best. So, I'm going to go ahead and generate this scene, and we're just going to call this a demo. Now, this is going to take, um, I don't know, maybe 30 seconds to a minute. Obviously, it depends on how long the video is. So, I'll check back in with you guys when this is done.
All right, so that just finished up. And one thing you'll notice real quick is this one was 16 seconds, and this one was 19, even though they were the same. So, that just kind of shows that the 11 Labs model still is going to output something a little bit different each time. But anyways, let's click into this and give it a quick watch. "Today, I'm going to show you how I was able to automate hours of recording content using Claude Code and Hey Gen. And some of you may have already guessed this, but yeah, I'm an AI clone of Nate." A lot of, I mean, like, you can see my eyes dart around a little bit. I like swallowed in the clip. I just think it looks so much more real. There's obviously a few things, like I said, that aren't perfect. Sometimes it's like extra exaggeration. And sometimes I've noticed if I lift up my arm, there will be like a little red triangle down there 'cause it doesn't actually know what's behind me. So, little things like that, but those can be worked out. And especially when this gets zoomed down to a face cam rather than being full screen, it's very hard to tell. But the voice is like the most important thing, and that is why you have to go over to 11 Labs to get that voice clone.
So, you guys may have also noticed if I keep scrolling through my dashboard here that all of these clips are just about a minute. And the reason I did this is because there's a few limitations. The first one being Hey Gen doesn't let you, oh my gosh, look at this guy. He is intense. But Hey Gen makes your Avatar 5 uh generations capped at 3 minutes. So, if you have a script that's 6 minutes long, you'd have to chunk that up. But the other reason is in 11 Labs, if you let these run too long, I mean, obviously you've got a character limit here of 5,000, but once you push past about a minute of audio, it starts to degrade, and it starts to sound less and less like you as that audio file gets longer. So, I basically found that 45 seconds to a minute is the sweet spot where it still sounds like me throughout.
So, then I started to realize, okay, hm, so I've got like all of these scripts that I need to record that are about 10 minutes long, and how do I actually do this? This is going to be super, super manual for me to paste in chunks of my script, get the audio, bring it over to Hey Gen, and just do this all manually, then stitch it together. So, I just started talking to Claude Code, and I was like, "Hey, I've kind of got this idea of, you know, using Hey Gen and using 11 Labs. Can you help me do this?" And so, it did research on the API documentation, and it found that it could do all of this automated.
So, now what happens is I come into Claude Code, and I say, "Hey, go look through my Google Drive, go find these scripts, and then just start processing them for me." And so, now it will take a script, and if you come over here and you see the um output, you can see we have the scripts, and it's chunking them up. So, for Lesson Five, it had to chunk up part one, part two, part three, part four, and it chunks them up to basically equal about 45 to 60 seconds of me speaking. And then it takes those chunked-up scripts, shoots those over to Hey Gen. And then we get back all these part one, part two, part three, part four videos, and then these are what we feed into the next step of ffmpeg and Remotion to edit everything together and add those motion graphics.
So, like, I know that sounds complicated, right? But we're taking a super, super manual process of me copying and pasting things and moving things around, and we're just orchestrating all of that together in Claude Code. So, like, literally last night, I said, "Hey, go process lessons 5.0 through 5.4," and I went to bed, and I woke up, and they were all just done.
Now, there is something interesting that I want to show you guys, and that is the fact that right now you can't actually generate a video with Avatar 5 via API. You have to do it with Avatar 3 or Avatar 4. So, we have to get a little creative here, right? Because I obviously want to use Avatar 5. So, what I did is I have the script generating all of these videos in Avatar 4, and then I have a Playwright script that opens up my Hey Gen dashboard and just changes it. So, let's say this one was just generated by the automation with Avatar 4. Then Playwright would click into it. It would click into new revision in AI studio. It would change this avatar over here once this loads up. It would change this from Avatar 4 to Avatar 5, and then just generate it, and then it would download the finished version. So, that's probably not a solution I'm always going to need, but it works right now because once they have Avatar 5 come out via API, I will just update the script to do that and get rid of the Playwright step. But, like, I just think that that is so cool.
And then I have a different project right here, which is called Remotion. This is where I gave it like this background image. This is where it was able to figure out the styling for me of the different motion graphics. And what that does is it transcribes the video. It stitches everything together, and then it puts it into this local host browser of Remotion, and it starts to just add on the text at the right time because when it transcribes everything, it's able to look at what time did he say this. Okay, he said this at, you know, 44 seconds. So, at 44 seconds, I'm going to start this animation where we have, you know, these little elements pop in, and that's how it's able to sync up the audio with my, you know, motion graphics that it's making.
And the other key thing there is when you are back in this step over here of in the Hey Gen studio and you're creating those scripts, you have to make sure that it's only cutting the script or sorry, it's only chunking each section at the end of a sentence because obviously you don't want it to stop in the middle of a sentence because now I get to this place where I basically rendered this entire thing, and in between each clip, you can't even tell that it was a new clip. So, it's very awesome.
Now, once again, I did have to play with this stuff for quite a bit to get it to work. As you guys can see, I have generated over a 100, maybe even 200 of these clips, and I've tweaked things, and I've went back and forth. And that is why I have one separate project for um the Hey Gen Studio, and I have one separate project for the Remotion editor. But now that I've proved the concept, I'm basically going to come into Remotion, and I'm going to copy all of these, you know, folders over to my Hey Gen Studio project, and I'm going to turn it into a skill where I genuinely just drop in a script, and I say, "Give me a full video," and it just does it all in one clean shot.
All right, so now let's talk about some of the other stuff. Um, this document will be attached for completely free in my free school community. The link for that is down in the description. And I'm also going to attach this Hey Gen Studio project in my free school community as well. So, you can download this, look through it, basically be able to replicate it based on what I've done here. So, both of those will be in my free school community. Link is in the description.
So, let's first talk about the three different shifts. So, the first one is that the avatar has crossed the uncanny valley. Hey Gen Avatar 5 is trained on 10 million plus data points for facial expressions, and it creates you a digital twin from just 15 seconds of a webcam clip. So, why this is awesome is because now you can put yourself in different settings, and you don't have to worry about if the sun is good or if your lighting is good or if the neighbors upstairs are being loud or if you're in my case, the fire trucks that go down the street every single two seconds. Seriously, when I'm recording, I have to stop and just wait like multiple times during my sessions because there's a fire truck. But anyways, like, I don't have to wait for the the environment to be perfect now because I could do something like this because it's not just lip-syncing. It's actually learning how you gesture and how you move, and it feels way more like you.
The next one is that AI can orchestrate the entire production pipeline. Like I showed you, Claude Code doesn't just generate one thing, and then you have to do the rest. I said, "Hey, I want you to connect all of these different tools together so I don't have to click on anything. I just tell you what I want, and you do it." So, it's getting the voice clone. It's chunking the audio. It's feeding it to Hey Gen. It's stitching it together, and it's creating those motion graphics for me. It's an AI agent coordinating a full production workflow that used to require three to four different types of roles. Maybe a camera guy, maybe an AV guy, a video editor, a person to actually read the script off. And every single time you run this thing, it gets smarter and smarter, which is why I did so much testing. But it basically turns a 5-hour pipeline into an overnight job that I didn't even need to be awake for.
And what else is cool is that the bottleneck has moved because the bottleneck in content creation was basically always production and, you know, post-production. So, recording, editing, motion graphics. But now that bottleneck is essentially solved. The new bottleneck is what we should be doing, which is thinking, scripting, the strategy, and the ideas. The human stays in the loop where it matters most, which is once again the ideas and the content itself.
So, I'm not going to read all these off, but I've got some stats here. Like, 91% of businesses use video as a marketing tool. That's huge. 67% of non-video marketers plan to start this year. And another one, the barriers to videos. 24% say that it's too expensive because, like I said, you need equipment, you need a nice studio, you need editing, you need all this stuff. But now, we're going to cover the cost in a little bit. It's so, so much cheaper. And not only cheaper, but think about the time you're getting back.
All right, so the fun stuff. These are three objections that I just knew I was going to get in the comments, and I'm sure I'm going to get more, but I wanted to address these three. The first one is that that's fake. That's not authentic. And to an extent, I definitely agree. Like I said, I never planned to have my YouTube channel be an AI avatar. Um, maybe on certain plays if I wanted to have like a Tik Tok channel that was just news, and it was just me reading off news. I think that's fine. I think most of the times people don't care if I'm actually saying it. They just want to know if it's my real thoughts, and they want to know that I'm not just feeding them a ChatGPT script. And I do enjoy making YouTube videos, and I want to keep them feeling personal. The script is yours, the voice is yours, the face is yours. The only thing missing is that you don't have to sit in a chair, or you don't have to wait for the stars to align, right? If you're on vacation and you realize you need to get something out, you already have your avatar of you in your home office.
The next one is, won't this flood the internet with garbage content? AI slop. I mean, I don't really tend to think so. And I think that people that think that way are just very negative because AI writing tools already exist. If you think that there's not already millions and millions every single day of AI slop being put out on LinkedIn or X or wherever, then you're wrong. But what happens is the best content is still going to win in a world where everything is written or enhanced by AI in some form. So, removing the production bottleneck means there's going to be more content, but the quality filter is still the idea. And that even makes it more competitive where only the best will actually make it to your feed or make it, you know, to the top. Bad content with a good avatar is still bad content.
And then the last one, isn't this going to kill video editor jobs? It changes the job. I mean, like all of these things that we're doing with video editing or automation, it's not just going to kill these jobs. I mean, there will be some jobs that are fundamentally changed and eliminated, but those people just need to be the ones that apply their subject matter expertise to it. I couldn't go make the world's best AI automation for SEO because I have basically no subject matter expertise about SEO, but the people that do, they'll be the ones that are able to make some killer SEO AI systems.
So, anyways, now let's talk about what this actually costs. And before I get into this, you guys might be thinking this is expensive, and it it it's definitely not cheap to make these videos, but think about it when you actually think about the investment. Same way I like to think about Claude Code. I pay 200 bucks a month for it, and that's probably my most expensive subscription, one of them, but it's the most valuable one I have right now.
So, the production stack on a creator plan of Hey Gen, it's going to cost you about 30 bucks a month, and that is going to give you only a limited amount of Avatar 5 generations, but it's more than enough to start on. Then you also got the 11 Labs creator plan. The 22 bucks a month plan is going to give you roughly 100 minutes of audio, which is a lot of audio. And then for Claude Code, you know, 20 to 200 bucks a month, which orchestrates the entire pipeline. You could also do this in something like co-work if you put the right connectors in.
And then we do have a different cost for the Hey Gen API. Um, it's going to treat it differently than just your your dashboard. So, let me show you a quick example. If I go into Hey Gen and I go into my settings and I go to my usage and history, you can see that I have premium credits right here, right? So, I've used up about 502 out of my 2,000 credits. And then if I go to my API though, here you can see I've spent a lot of money, right? Like I said, I've tested a lot. And if I go to my usage, you can see my spend, my error rate, my total requests. And you can see that each of these 1-minute clips are taking about four bucks. So, if you do a 10-minute video, that's probably going to run you almost $50.
Now, yes, a little bit expensive, right? But if you just think about it as far as like, okay, so how much am I actually paying to get back a couple hours of my time? Is that worth it? For some people, the answer is yes. For some people, the answer is no. For most businesses and for how much you're going to be able to scale your content, which is top of funnel, it basically funnels everything downstream and affects your bottom line in a great way. I think that that investment is a thousand% worth it. If this is the unlock that a business needs to start consistently creating content, then yeah, it's worth it. And that's why I talk about right here, what is the human alternative, right? A freelance video editor could be 35 to 75 bucks an hour. So, a 10-minute YouTube video could take up to 300 bucks. And then if you want to get a professional voice over or if you want to have a studio to record in, all of this stuff could add up to a few thousand potentially just to get out a few course material videos or a few advertisements. So, the right way to think about it is that it's not cheap in absolute terms. But if you think about the investment, it is worth the investment. So, if the pipeline gives you back 10 hours a week, that's 40 hours a month. And at 251 bucks a month, you're paying about six bucks an hour to buy your own time back. And according to the buyback matrix, that's probably a good deal.
So, anyways, I am going to do a full breakdown on Remotion. I know that you guys have noticed some of my other videos have been edited by it, but I've been just been playing around with it a lot, and I'm trying to figure out the best way to package up a video. So, that will be coming soon. I will let you guys know when that is out. But that is going to do it for this one. Just remember that you can download this Hey Gen Studio and this Word document that we were looking at if you are interested in kind of diving in. But yeah, otherwise I would go ahead and say go to Hey Gen, create your avatar, go to 11 Labs, create your clone, and just start playing around with this stuff. So, if you guys enjoyed the video or you learned something new, please give it a like. It helps me out a ton. And as always, I appreciate you guys making it to the end of the video, and I'll see you in the next one. Thanks, everyone.