Transcription
Hey there, I'm Nico and today we have some exciting stuff to talk about. Yesterday we launched image to video for Gen 4.5. As always, we put together various sizzles to showcase what the model can do.
Now I want to show you how you can create a sizzle for yourself right inside of Runway. Image to video well needs images. So, let's get started there. We recently introduced Nanab Banana Pro into Runway and we'll be using it to create all of our shots.
We start with this image that we had laying around as it could be a great reference for a sci-fi mining scene. A few things to shout out are that I am generating at a 21x9 aspect ratio to make our images even more cinematic. I'm also rendering them at 2K to have nice and crispy details.
To have consistency, we pull our hero shot into the image reference container near the prompt box. With our world set, we now prompt for a close-up of the outfit, a wide shot standing next to a mining truck, walking shots on the factory grounds. There's no need for fancy prompting here as a lot of details are defined by our reference image.
Here we see our first results. We can see how the reference image made sure that the detail stayed intact, enabling us to really weave that consistency through this scene. Here we see the three selected hero shots that we will use as the basis of our scene without any additional prompting. We also got some nice extra shots which we will use in the final sequence.
Now it's time to animate our images. We head over to the video tab and make sure that Gen 4.5 is selected in the model picker. The process here is relatively straightforward. Often you don't even need to use a prompt to get great results. I pulled in some of the images we generated. The videos you see here didn't have a prompt. It was simply the model interpreting the image and deciding on what motion fits best.
For two shots, however, we want to control the action a bit more precisely. For this close-up shot of the harness, we want the character to move like they're checking that everything is tightly fit. We can introduce that nuance motion by adding this simple text prompt. We can now see an increased sense of weight in the final video. Small details like this can make the big difference.
Something that Gen 4.5 is very good at is sequential prompting. We're going to use that technique for our advantage with this shot. With sequential prompting, we can control the sequence of specific events that a generation should render in a shot. In this scene, we want the character to walk forward a bit. Then after a delay, have the truck explode, which in return startles the character and makes them run. We can actually use those exact instructions as our prompt. And voila, we get a scene which follows that order of events precisely. This is a great technique for directors to really make their vision come to life.
All that's left to do is add some sound design, after which we end up with something like this. And that's it. Gen 4.5 is truly a special model. It has the kind of quality which renders movement in a beautifully different but intentional way. This update will not just help creatives tell brand new stories, but it will also guarantee that those stories feel different. We can't wait to see what you create next. From the entire team here at Runway, thank you for generating with us.