Transcription
Here's how to create realistic, high-quality music videos completely using AI. Most tutorials for how to make an AI music video might show some basic lip sync tools with a pretty static character, but that's boring and there's way more that AI can make for you. Now, and in this tutorial, you'll learn how to create a character that sings with the absolute most expressive voices and animations, like singing while strumming a guitar at the same time, for example, and incredibly eye-catching effects that can truly bring your music to life.
[Music]
So, first we need a song. If you already have your own music, you can skip ahead to the section where I talk about how to make the AI visuals. I'll put chapters below. Otherwise, we need to go ahead and generate the music. We'll be using an app called Suno AI to create the AI songs. This is the best damn music generator that I found. It's pretty simple to get started with this. You can add in your own lyrics or have it automatically generate the lyrics. I'll enter a description of what I want the topic to be about. Let's say it's a rebellious teenage song, for example. And then below there's this style section, which is where we'll enter the genre we want the AI song to be. For example, rock and roll, '90s teens. Now, let's create the song.
[Music]
So, of course, you can spend a lot of time fine-tuning these prompts, writing your own lyrics, also combining a bunch of different styles together. Like, I could add in Latin, uh, I don't know what this Pentecostal is going to sound like, and really make it customized to the specific style of music you want. You can also look through a huge library of songs that other users have generated. I made quite a few different versions of this and ended up with this one which I really liked.
[Music]
Really impressive what the AI can do. Now I've downloaded the audio file for the song. Now what I need to do is to create the AI visuals that we'll be turning into a music video. Starting with this photo of my female singer. And we could just stick with this image, but for a complete music video, I want more visuals that show her playing different instruments, for example, with some more dynamic effects, like singing into a microphone with some fire in the background. I'd also like to see some different camera angles. Here's a pretty common one where we're above her and tilting downwards. This is the type of thing you'd expect to see in an actual music video.
So, how do we make these visuals? There's a lot of different ways of doing this, but one of the more beginner-friendly ways I found is using this platform, Code Design. So, you can see that I went ahead and created quite a few different visuals of her in different situations like playing on a drum set. So, the way we'll do this starting from the homepage is creating a new project. Uh, let's give it a name, uh, like music video. We can set the aspect ratio to a widescreen image. Let's make a project. And then we'll be using this instant storyboard tool. So what this tool will do is let us upload an image of our character and then create more images of that same character doing different things like playing an instrument. So I'll upload that photo of the singer and drop her inside here. And now we can describe a new image of her. So in the original character image, she's just standing there. It's kind of a boring photo. Let's say we want to have her hold a guitar. Inside the prompts, I can put something like, "Create a photo of this woman playing an electric guitar." And then I also describe the style I want the guitar to look like. So it's grimy and gnarly in a dystopian style, which is the vibe of the music video I'm going with. Uh, let's also increase the output quality to 1080p and then make sure that it's generating a 16:9 horizontal image. So basically, it's taking that same exact woman from the original character reference photo, except it's added a guitar that she's holding. And something like this is going to be perfect for the music video.
Using the same method, I went ahead and created a bunch of different visuals of this woman. Some of them she's singing into a microphone. There's, uh, burning buildings behind her. And later on, when we actually go and animate the lip sync for the characters where she sings along with the song, you'll notice that a lot of these special effects in the background, like the burning buildings, get animated as well. So a good tip if you want more exciting videos is make sure there's background elements like these flames and smoke that could actually be animated in the AI video itself. So, I have all the AI visuals ready for the music video, and it does a really solid job for the most part of preserving the way the character looks. Between these two, it does look like it is the same exact character.
Next step is to add a lip sync to the characters where they're actually singing the song. Now, a lot of the tutorials out there that I've seen show how to do some basic lip sync on pretty static characters like this one where she's just standing still and mouthing the words to the song. Something like this is okay. We want to create something that's a lot more exciting. Something that looks like this where you can see how expressive she is singing to the microphone. You can really feel the emotion she's going through. You see the smoking buildings in the background. Even the little details of the wind blowing on her shirt. Here's another example where she's strumming the guitar and singing at the same time. And to get really cool lip sync results like this, it's really important to make sure that the original visuals that we created in the beginning have the dynamic elements already included inside. Adding a microphone is a hack that really improves the expressiveness of the lip sync to the music.
So to do the lip sync, I'll stay inside Code Design and go to this lip sync button. From here, we can pick an image of the character. So I have a bunch of them already uploaded or we can just drop them on here. But let's go with her singing into the microphone to begin with. And it automatically will detect the face for you to select. But you can also manually drag the selection box over the area that you want the lip sync to be added in. Now we need to upload the audio for the song in this "Pick a Voice" section. So the thing is, the audio files need to be under 30 seconds, which means you'll need to break down the song into separate parts first before doing this. And so I've taken a clip out of the full song and I'll put that in here. By the way, if you're going to be using Code Design for this, you want to make sure you leave a little bit of space at the beginning because in the first couple frames, sometimes the lip sync isn't animated correctly. And then I recommend you using the Pro mode. What's going to get you the super expressive results that I've showed so far. And this does take a little bit of time, 20 to 30 minutes, but I think the results really are worth it. And let's see what it looks like.
"Lonely hearts drift away, searching for dawn at the edge of the day."
You can really see how lifelike and dynamic it is. It's not perfect. You'll see it a little bit out of sync with a few of the words here and there, but anybody can make something like this. And if you look at the entirety of the video, the expressiveness of the character, the background animations of the city burning down, the wind blowing on her shirt and her hair, this is such an impressive result. For a complete music video, we'll want a good variety of different scenes with this character. So, I also added the lip sync to the version of her that's playing the guitar.
"...for the edge of the day."
[Music]
Here's another lip sync version where we're looking at it from a different camera angle. This is probably my favorite shot that I generated.
So, now that the lip sync is done, there's still a lot more visuals and creativity that we can add to the music video where we can create different effects for the character. For example, if she vanishes into dust or just have her playing on a drum set. To animate the rest of the videos, I use this AI video generator where we can put in an image of the character and then describe any special effects we want to happen. So, let's say if I want her to walk forwards a bit and disintegrate into dust and fade away. I can use a prompt like, "She disintegrates into burning dust that blows away into the wind." There's a pretty good variety of video models you could pick from here. For most of these, I used either the Cling 2.1 model or the C-Dance model, which I found gave me the best results. So, for more VFX-style of effects, like the one I'm going for here, I actually like using the Cling model a bit better. I find that it tends to do the best job of following the idea I have in my prompt and actually animating it. But for other human motions that are a bit more basic, like her playing on a drum set, I found that the C-Dance model actually does the best job of creating the most realistic-looking animations. Both of them are solid choices.
Finally, we can also take things a step further and enhance the quality of the videos even more using an AI video upscaler to really get that high-definition look. And I like using a tool called Topaz AI for this, which I think is probably the best AI video upscaler. So, if we dropped this lip sync video of her in here, we have some different options for upscaling. So, I can choose between different upscaling amounts. Let's say I just go for a full HD upscale. Then, you can choose between a bunch of different video models. Proteus is the default. That tends to work pretty well. There's this model, which is specialized for enhancing the faces. But let's see how it does if we just stick with Proteus. And then I'll just need to export the video. If we compare the before and after, the upscaled version of the video is just going to have that little bit of extra sharpness and details to it. Although this is an optional step, I think the results are already pretty good even without the video upscaling.
And after putting everything together, here's what the final music video looks like.
"Stay away. Searching for dawn at the edge of the day. Let the star shine as a beacon of light guiding us homeless."
And I think that's really impressive for something that you can put together yourself without a camera crew, without all the recording equipment. Is it perfect? No. For some of the clips, the lips don't match the lyrics exactly, but the fact that you can do this completely on your own is pretty amazing. I showed you how to do the lip sync using Code Design, but there are a bunch of other lip sync tools out there like HeyGen AI, Higgins, for example. If you want a deep dive comparison between all the different lip sync tools to see which one will work best for the specific type of character you're trying to animate, go watch this tutorial right here.
[Music]