How to Write PERFECT Prompts for AI Video

The secret to achieving high-quality professional videos consistently that looks and sounds realistic is probably not what you think. If your AI videos look generic, awkward, or nothing like the professional examples you see online, >> [music] >> the problem probably isn't your AI. It's your prompting system.

You see, 95% of people treat AI video like it's a magic box. They type something vague like "Make a video about marketing" and hope the AI figures out the rest. But AI models [music] don't work that way. The quality of your video depends almost entirely on how you clearly instruct the model in a way that the AI actually understands.

A good prompt for AI only needs these four things to generate amazing results. So, in this video, I'm going to walk you through the prompting framework behind great AI videos using HeyGen's Video Agent. You'll learn how to structure prompts that generate clever scripts, how to provide appropriate context for better delivery, and the secret sauce behind every prompt that generates videos that actually look professionally produced.

And if you want a simple reference for the full prompting framework, HeyGen has a prompt guide that breaks everything down step-by-step. And I'll link that below so you can follow along. But for now, let's dive in.

So, here's the thing that most people don't realize when it comes to creating AI videos. The difference between a beginner AI video that feels generic and a professional AI video that actually makes an impact isn't complicated prompts. It's having a system for consistency.

And when it comes to prompting Video Agent, there are three levels that separate a good AI video from a great one. And at HeyGen, we've developed a simple prompting formula that you can use to get repeatable results no matter the industry or type of video you're creating. And it's called the CODE formula. We'll unpack each aspect of this formula throughout this video step-by-step.

All right, so let's start with the first part of the CODE formula. The easiest way to think about this is: more context equals better videos.

I like to think of it like talking to a person. If I ask someone to make me a video, but I barely explain what I want, they're probably going to give me something vague or completely off. But the more I tell them who it's for, what it should sound like, what it should include, and what outcome I want, the better they understand the assignment.

And so, AI works in a very similar way. The difference is: AI has access to an enormous amount of information. So, if your prompt is too broad, it has too many directions it can go. That's why you start getting generic results, or worse, things that feel off-brand, inaccurate, or just plain random.

Now, before we even write a single word, we need to make a few selections here inside of HeyGen. But first, what exactly is Video Agent?

Video Agent is a powerful AI creative partner that helps you turn an idea into a complete production-ready video. And honestly, it's my new best friend for creating banging videos using the system I'll be showing you in this video.

But it's not just me. Creators across the globe are using Video Agent to generate thousands of videos every day for social media, product explainers, online courses, and UGC ads. And the best part? It saves hours of work while producing videos that look professionally made. And you can, too. I've added a link in the description below so you can create an account and start experimenting with Video Agent yourself.

So, first, let's navigate over to Video Agent here in the sidebar. First stop is avatar selection. You can choose a specific avatar if you already know who you want presenting the video. Or, you can leave it on auto, and HeyGen will select one that fits your content. Now, a quick pro tip. If you don't want an avatar at all and prefer a voice-over-only video, you can absolutely do that. Just make sure you say "No avatar" directly in your prompt so the agent knows what you want.

So, our job is to narrow the field and give the model enough context to generate something useful. So, let me show you what this looks like in practice. For this example, let's say I'm creating a video on the state of the creator economy. Here's what might be considered a weak prompt example. Suppose we type in a prompt like "Create a video about the creator economy." Now, technically, this is a prompt, but it's missing almost everything Video Agent needs. It doesn't say who the video is for, what angle to take, what kind of information to include, or what kind of output I'm expecting. So, yes, it can generate something, but the result will probably feel generic.

Consider a stronger prompt example like "Create a video for knowledge creators providing an overview of the current creator economy. Explain the major shifts happening right now and the trends creators need to adapt to. Use real statistics from credible sources like Forbes and other trusted publications." Now, this sounds a lot better. But why? It's because I'm giving the agent direction. I'm telling it who the video is for, what the video is about, what specific angle to take, and even the type of sources I want to draw in from. So, if you want better AI videos, don't start by trying to sound clever. Start by being clear.

Next, let's work on selecting our duration. To do so, you're going to navigate over to the options icon here, and then click on this clock icon right here. You can let the agent decide automatically based on your prompt, or you can choose something specific like 30 seconds, 1 minute, 2 minutes, or even 3 minutes. Just keep in mind that the agent still follows your prompt or script length. So, this isn't a hard limit. It's more of a guide. This setting helps point Video Agent in the right direction before it even starts generating.

And finally, we have the aspect ratio. This lets you decide if the video should be portrait for vertical platforms like TikTok or Instagram, or landscape for YouTube and more traditional formats. And of course, you can leave this on auto, as well.

Now, once you've established context, the next level highlights the single biggest upgrade most people miss. Well, for you watching, now we're stepping into the pro level. Next in the CODE formula is: Use your script directly.

While most creators simply prompt the AI to generate an entire script, you can actually paste your full video script directly into the prompt. This is possible because Video Agent is capable of following a script scene by scene, while automatically improving things like the flow, the pacing, the visuals, and timing. Now, this approach requires a little more thinking up front, but by doing this, you've just elevated yourself a tier above the rest. You're essentially stepping into the role of a screenwriter and director. By providing your own script, you guide the entire narrative, which ensures the AI produces the exact type of content you're aiming for. Not to mention, you avoid running the risk of including things that you might not actually need.

For my creator economy example we mentioned before, that might look something like this. Okay, so what I want you to pay attention here is: notice how I'm structuring the script with sections like A-roll, B-roll, and motion graphics. Now, one thing to keep in mind is: Video Agent may still make small edits to things like grammar, pacing, or phrasing to improve the overall clarity and the flow of the video. But the structure and direction will follow the script that you provide.

Now that we've established our script, the next step to elevate and enhance your video is one of the most undervalued features in Video Agent. But it has the potential to take your video from "This was made with AI" to "This was intentional and professionally produced." And yes, my friend, the secret is attachments.

This is where you start giving Video Agent reference material to work with. This is what the pros are implementing in their videos, because the more context you give the AI through supporting material, the better it can reference, visualize, and structure your content.

Video Agent also allows you to upload several types of files. For example, you can upload images and videos. I'm talking about things like product screenshots, diagrams, iPhone clips, and existing media assets. And you can even upload a photo of yourself and ask the agent to turn it into a talking avatar.

But it doesn't stop there. But wait, there's more. You can also upload PDFs and documents. Things like training materials, research papers, or product documentation. And the most impressive part? Video Agent will actually read through those files and extract key information to use inside the video.

The secret to unlocking Agent's full potential is: once you upload your references, tell the agent how you want them to be used. For example, you could say things like "Use the attached product screenshots as B-roll when discussing features." Or you could also say "Reference the attached PDF for accurate technical specifications." This level of context allows Video Agent to create videos that are far more intentional, accurate, and professional.

All right, so now we're stepping into the advanced level of prompting. This is where we really kick things up a notch, because this is the stage where the difference between an amateur AI video prompt and an expert-level prompt become very clear. And trust me, the results speak for themselves.

So, what actually makes a prompt advanced? Well, it comes down to the finer details. Specifically, your visual style and your colors. Video Agent is capable of executing your style requirements consistently across the entire video. So, by using style descriptors in your prompt, you're essentially directing the visual identity of the video from start to finish.

To make this easier, we've included a list of example style descriptors in the article. You can pause the video here to take a look, or you can check the link in the description to review them later.

This is what allows you to dictate the look and feel of your video. Maybe you want something minimalistic and clean, or maybe something more bold and vibrant for social media, or even something more cinematic with dramatic lighting.

You can go even further and specify things like exact color codes and font families for consistent branding. Prefer sans-serif fonts over serif? No problem. Want to use your brand's exact color? Just state it in the prompt.

And here's why this matters. When you define visual styles, Video Agent can maintain consistent design language across every scene. Without that guidance, sometimes the visuals can feel a little off from clip to clip.

Now, another advanced prompting skill is understanding what type of visual media works best for each situation. Things like motion graphics, AI-generated visuals, or stock footage. Video Agent can make smart decisions automatically, but when you guide it with specific instructions, you're far more likely to get the exact output that you want.

For example, motion graphics work great for explaining concepts of highly detailed information. An example prompt where you might use this is: "Use motion graphics to display the five key benefits as animated bullet points appearing one by one while the avatar speaks."

Now, when it comes to AI-generated images and videos, they work really well for abstract ideas or concepts that don't exist yet. For instance, you might use an example like "Generate an AI image showing a futuristic office where humans and AI work together. Use this as B-roll for the future of work section."

And when you want something grounded in the real world, stock footage is often the best option. An example prompt for stock footage might be, "Use stock footage of a busy corporate office for B-roll when discussing workplace productivity."

But when you combine clear visual direction, branding guidelines, and media instructions, you give Video Agent everything it needs to produce videos that look intentional, polished, and professionally directed.

Now we've covered the basic, pro, and advanced level of prompting, let's bring it all together with a scene-by-scene prompt example.

Okay, so we're inside Video Agent, and as you can see, I've already selected my avatar. I'm using the voice on auto, and the style is what I will be configuring inside of my text. I've added my assets like the HeyGen logo, as well as other accommodating material, and I've established a global style using the appropriate colors and typography that I've established here.

Now you'll see in scene 1, 2, 3, 4, and 5, I have also indicated in a certain format the visuals, the voiceover, as well as the duration. Notice I use VO for voiceover, which Agent is able to adapt and understand. I've also added the visuals, which I clearly describe in this fashion here.

So one key thing to note is that once you're in Video Agent, you have the option of clicking on chat or generate. So you can automatically generate the video from this area.

Now we can just click generate and give Video Agent a moment to render the video.

Now that Video Agent has established a blueprint, we can see everything configured here, like the orientation, duration, the language, the captions, and the number of credits that will be used for this video.

And so once the video is ready, we can review it to see how the scenes, visuals, and pacing come together.

The creator economy is evolving faster than ever, and the creators who learn to use AI will be the ones shaping what comes next. And with tools like AI video generation, creators can now turn ideas into polished videos in minutes instead of hours.

Great. So now if you want to tweak something, you can simply prompt the AI to make adjustments, just like you would in any normal AI chat. And if you want to take the reins again, you can always open the project directly in AI Studio mode by clicking this option in the interface. This will let you take full control over the scenes, visuals, and timing if you want to fine-tune the video even further.

Okay, so let's do a quick recap.

First, we talked about context. The clearer you are about what you want the video to be about, who it's for, and what it should include, the better the AI can understand the assignment.

Then we moved into the pro level where you can paste your own script directly. That's where you really start guiding the story of letting Video Agent turn it into a polished video.

And then next, we added attachments, giving the agent reference materials like screenshots, visuals, logos, or documents, so that it has even more context to work with.

And then finally, at the advanced level, we define things like visual style, colors, and branding, so the entire video stays visually consistent.

Now you put all of that together, and you simply move from generating AI videos to actually directing them.

Now if you'd like to try this for yourself, I've added a link in the description where you can start experimenting with Video Agent today. And if you create something cool, feel free to share with us. We'd love to see what you build.

Now if this video was helpful, be sure to like and subscribe for more tutorials on AI video creation. And if you want to keep learning, check out this video right here, where we dive even deeper into creating videos with AI.