Transcription
Let's push the limits of the Cling AI video generator, which is absolutely incredible at turning images of your characters into videos. When it comes to generating different human motions, Cling has the best prompt control I've seen so far.
I'll show you how to prompt for different movements so you can control your characters like puppets. Let's use a clean image-to-video generator for the first example. I'll upload this image of a woman sitting on her sofa.
The key is to prompt for what actions you want to happen in the video and not just describe the image itself. I could go with a prompt like "a woman in her living room," but that doesn't actually describe any movements. Instead, let's use a prompt like "a husky puppy runs onto the woman's lap." In the generated video, the woman turns her head and looks at the dog, which jumps onto her. That's pretty much exactly how I'd expect a dog to behave.
What's amazing about Cling is the variety of actions that are possible. I use prompts of her reading a book, which appears at the bottom of the screen, or drinking tea. Cling's really good at animating hand and arm motions while keeping the body structure intact.
If you introduce a lot of motion, for example, when she lies down on the couch, the face starts to blur a bit and loses its shape. Now, you may have noticed Cling added in this professional mode in the settings, which costs more credits and increases the visual quality of the videos generated. However, if you turn this on, Cling actually has a harder time adding tons of motion into the video.
Here's a comparison with a prompt: "the woman lies down on the couch" in standard mode and professional mode. In the professional mode video, the plants shake in the background a bit like there's an earthquake, but she doesn't lie down.
There is a way to force Cling to obey your prompts more by turning up the relevance and creativity slider. Sliding this all the way up forces Cling to follow the prompt as much as possible. Now, even in professional mode, she lies down like we asked for, although you'll notice there's blurring on her face at the end of the clip, and the colors also get messed up and become brighter around her.
The increase in the brightness of the colors does happen when there's a lot of motion animated. Although there are some limitations, the level of control you have is absolutely amazing.
Let's take a closer look at the creativity and relevance slider. We'll animate this photo of a man standing still. On the left are a with the slider set to the middle, and on the right, the slider is turned all the way up. In all these cases, the AI follows a prompt way more closely when I turn the slider to the max setting.
This was my favorite animation: there are two motions of kneeling and praying, and the AI does both simultaneously. There's no blurring at all. The details on the clothing do get slightly smoother towards the end of the video, but the movement is perfect.
You can control head movements by asking your subjects to look in different directions. Look at how the shadows on the wall move to match the different head motions. Now, they're not entirely accurate; in the clip where she turns her head to face us, the shadow looks kind of deformed. You can still see her nose on there, but it's cool that they're actually matching the head movements.
Like I said, Cling's good with hands and fingers pointing. I prompted for the man to point his finger excitedly in the video on the right, and you'll see his finger popping into the video with the same textures and color as his face.
Get creative and experiment with different body movements. Arms, hands, and full body motions are all possible in Cling, and for most of these, it took me at most two or three tries to get an accurate movement.
One thing Cling does struggle with is animating different camera movements in image-to-video. I tried out a bunch of images but was unable to get Cling to follow my prompts for different camera motions like zooming in or panning in different directions. Let me know if you found any solutions to that; I'd love to know.
I've been super impressed with Cling so far. If you're interested in learning about the different visual styles you can animate in Cling, go and check out this video tutorial I made over here. [Music]