Transcription
It was a Bonkers week in the world of AI filmmaking. The world's first AI feature film is here! Runway just announced the ability to control actor performance directly inside of their tool, there's a brand new AI video tool that basically connects all of the AI video generators together, and Anthropic released the ability for computers to control your mouse. Good gravy, there's a lot to cover! This is your AI film news of the week.
The first big piece of news that we're going to talk about is the big news from Runway, which kind of feels like they have big announcements every single week. The feature that we are, of course, talking about is Runway Act One. Essentially, this brand new tool allows you to control actor performance directly inside of Runway.
Previously, you would have to use a tool like Live Portrait to take a video of an actor performing and then put that on an AI video or image. Runway is basically creating their own tool that looks like a modified version of Live Portrait and putting it directly in their platform, so you don't have to daisy chain all of these third-party applications together. They released a few demo videos showcasing the tool.
Let's take a look:
“You're going to fire me 'cause there’s hot milk everywhere on the floor of the coffee shop. I mean it's one latte; it's not that big of a deal. Everything's fine; everything's totally great. This is going to be the best day ever. Smash my phone the other night, and you know how much I love that phone. Okay, all right, just breathe. Okay, maybe you don't breathe that hard.”
So as you can see, the videos that are generated generally require the subject to be looking at the camera or at least oriented closer to the camera. So you're not going to get the control over an actor who's, let's say, facing to the left or the right. You still have to have them generally looking at the camera if you want to get the best result.
One of the more impressive examples that came from Runway is this diner scene that kind of feels like it lives in the Breaking Bad universe. Let's take a quick look:
“You really think you can just walk away from all this?”
“I have to; there’s no other choice.”
“There’s always a choice; you just don't like the alternatives.”
“What would you have me do? Stay? Just watch it all crumble around me? That's what you signed up for; it's what we all signed up for.”
Okay, so as you can see, this is kind of the beginning of being able to use AI to have conversational scenes that have emotional depth. Now obviously, in the example, even inside of that diner scene that looks pretty darn good. That's one of the more performative and interesting AI videos that we've seen up to this point.
If you look in the background with the guy talking, there are like slow-motion characters that just ebb and flow and just kind of fade into the distance. So with that being said, do I think that this feature will be Silver Screen ready? Probably not. But for online distribution or putting things on YouTube or social media, it seems like this tool actually could do a really good job.
Now, I should also note that in the video with the older man, the lip syncing doesn't quite 100% match up. I'd say it's like 98% of the way there, which is fantastic. But if you're looking for that extra refinement to be used inside of a professional project, you still are going to have to go through the process or use some higher-end tools to make it happen. But the fact that it's going to be so easy to do this inside of Runway is a huge step forward.
From the demo that Runway put together, it does seem like this kind of only works whenever you are using a character that is either human or humanlike. You probably aren't going to get dynamic mouth movements from animated characters that have snout movement and things like that.
Now unfortunately, you don't have access just yet, but Runway is actually really good at announcing new features and then releasing it just a few weeks later, so I imagine that we'll have access to this tool within the next two weeks.
There was some pretty cool news from the people behind Korea. Now, Korea is an incredible tool that has been used to generate images to do Uprising video. Uprising, but they actually just came out with a new feature that is basically an API integration with most of the major AI video tools that are out there. Essentially now you have the ability to generate a video from virtually any platform directly inside of Korea.
This includes Minx, Runway, Paa Luma, and more. To use this feature, all you have to do is go to the Korea website and select the video icon from the homepage. From here, all you have to do is type in your prompt. For our example, we'll just type in “a cinematic live action shot of a baby duckling sitting in a porcelain teacup,” and in the bottom left here, you can see that you can select the menu and essentially change it into any of the video models that are listed there.
So we have Luma, Runway, Cing, Minax, which goes by Hu, and Pika. When you're ready, all you have to do is hit generate. I went ahead and generated results for all of the video models. Let's take a look:
We have this shot from Runway of this duck. Not as much movement, but you know I like the camera shake. It does come across as a little realistic although I think it's a little too contrasted and kind of is coming across more as concept art as opposed to realistic.
We have this shot from Cing, and I think the duck just like shattered that cup 'cause his butt is sticking out the end of it. So that’s like a very strong little duckling there.
Here's Minax, and yeah it looks pretty good. The feathers look a little extra sharp. They're kind of coming across more as hair as opposed to like duck fuzz, but honestly a pretty good generation.
Finally, we have the crush feature in Pika, and I should note that this one is actually a rubber duck, so no harm, no foul there.
So the cool thing is all of those models can be used directly inside of Korea. You don't have to go to all of the individual tools to generate the video, and the cool part is, with your subscription to Korea, you actually can select any of those models. So again, you don't have to have a bunch of different subscriptions to a bunch of different tools.
So text to video is cool, but what's even better is image to video because when you're working on a film project, you'll quickly realize that image to video allows you to have everything from character consistency to style consistency and it just gives you more control.
So using Korea, you actually have the ability to do first frame and last frame if you're using Luma or Cing, and you have the ability to do first frame if you're using Runway. So let's look at a quick example.
So I have two shots here. This is the first shot of this kid holding an ice cream cone, and then we have a wide shot basically of the same kid, and we want to interpolate and animate between those two frames. So we'll run this through Luma and Cing, and here is our result from Luma. You can see the kid just standing there, and yeah, it did a pretty good job.
There’s kind of some warping around the hand, and obviously the people in the background, it’s a little odd. It’s not 100% there, but not too bad.
And here is the same result from Cing, and this one was actually pretty good. I think there's some weight behind the steps; the overall physics look better, which is very characteristic of using Cing, so I think they did a pretty good job.
I should note that you do have the ability to adjust the camera movement inside of Korea if you are using Cing, but you don't have that ability using Luma. You have to just go back to prompting in the normal way that you would prompt camera movement.
I think news like this is really important because it really just showcases that AI video tools are becoming increasingly user-friendly.
Now the one question that you probably have is: how much does this cost? Well, if you go to the Korea plans page and scrub down, you will find the section that says Video Beta, and they basically break down how much each one of their plans allows you to generate per month.
So their most expensive version is $60 a month, and essentially that allows you to create anywhere from 163 videos all the way up to 571 videos. It just depends on the model you're using and ultimately how much they charge Korea to connect to the API.
If you compare that amount to how much they actually charge you if you were to use the tool directly, like if you were to go to Runway or go to Luma, you're essentially paying about 50% more to use Korea. But to be fully honest, to just have a single subscription and be able to use these tools is pretty powerful.
I still think that the unlimited feature inside of Runway is probably your best bet if you want to get maximum bang for your buck. But using multiple video models is pretty normal. If you talk to somebody who creates AI videos professionally, they will tell you that they just use certain models for different shots.
So you may use one model for one specific shot, and if it's not working, you'll go to a different one and pick and choose the right result for you. So it's cool to see that Korea is basically taking that experience and putting it in one place.
Now, you're not going to get the very best model from all of these tools directly inside of Korea either, so I recommend checking out their page to see some of the limitations. For example, if you're using Cing, the generations can take up to 10 minutes long, which is just kind of slow compared to using the actual platform itself.
The team at Midjourney also came out with a new feature that you definitely should know about. Now, up to this point, if you were working with Midjourney, you could use images as style references, image references, or character references, but you didn't have the ability to just directly upload your image and edit it using Midjourney.
Well, all of that has changed today. Now you have the ability to use a brand new editor inside of their platform. To use it, all you have to do is go to the Midjourney website, try not to get distracted by all of the incredible Community generations, and go over to the Edit button on the left.
Here you have the ability to upload an image from a URL or you can just upload the image directly. I have this shot of this beach scene here. We'll go ahead and import that directly into Midjourney, and similar to using their inpainting feature whenever you're actually editing a Midjourney image using their image generator, basically you have the ability to define where you want to add in your new information.
So I want to basically upload a photo, let's say of me at the beach here. So what I'm going to do is I'm just going to erase the area where the character would be standing and maybe a little bit of the ground here just to have some room for shadows or any other details that we might want to add in.
We will prompt in and say “a man at the beach.” Easy peasy! The cool thing is you also have the ability to do style reference, character reference, and image reference all directly using this new tool. So we can select character reference.
I have this picture of me wearing a sweater, and what I'm going to do is I'm going to do “--cw 0” for character weight and say zero because I shouldn’t be wearing a sweater at a beach. I should definitely be wearing like a t-shirt.
You do have the ability to go in here, change the aspect ratio, do all sorts of really interesting things. And you know what? I’m going to add in a little bit more information to the bottom here just to expand it so there's just more room.
Midjourney is always better whenever you give more room for it to inpaint just because it adds in more creative data. I’m going to say “a man in a t-shirt at the beach facing the camera.” There we go!
We’ll go ahead and click submit edit. Okay, let's take a look at our results. So we have result number one. I think my head might be a little too big, but you know, not bad.
We have this one here. Yeah, that one's pretty good. This one here, I don’t know if the lighting is quite right. And then finally we have this one here, which does kind of look like a selfie. I will say the fidelity on the character's face isn't perfect, but it did a pretty good job.
Now this tool can do a wide variety of different things. For example, we also have the ability to extend the edges of images. So if we wanted to change the aspect ratio, we can just basically grab the edges like this and move it.
We also have the ability to move and resize the image if we wanted to make it larger. So let's just say we wanted to generate it with this new composition here. We can click submit edit and it will basically add in new information into the frame.
Okay, cool, so it did a good job at expanding the frame there, and there's not those hard seams on the edges that can come across as basically unrealistic. It did a really good job at smoothing that out. So great job for Midjourney there!
Now I should also note that Midjourney has another feature called retexture that basically allows you to change the overall style of an uploaded image. So for example, we can retexture this image here, and let's say that we want to change it to a completely different style.
I’m going to say that the new style we want to do is like a line drawing. So I'm going to say, “a line drawing of a man at the beach.” There we go!
We'll get rid of the character weight code because we don't need that anymore, and you also have the ability to upload a reference if you want. I'm not going to do that for this one. We’ll go ahead and click submit retexture.
Okay, let's take a look at our results here. So we have shot number one here. Yeah, it did a pretty good job. I do think that it broke down some of the character style here; like it seems like that might be a different person compared to the original image.
So you know, it’s really adding in a little bit more stylization there. If you wanted to turn that down, you can just turn down stylization a bit.
We have shot number two here, which completely got rid of the face, which is great. We have shot number three completely removing the face there. The composition is the same but, you know, generally, it changed the face.
Then we have the final shot here, which again, the background is slightly different, but the overall composition is pretty close. I think having the ability to use AI tools to edit images in a wide variety of applications is really important and really helpful whenever you are in the creative process. So it’s cool to see Midjourney coming out with these new features.
The team at Kyber also had a major update recently called Super Studio. It’s a really interesting pivot for the company because you're probably familiar with Kyber because of all of those crazy hallucinogenic videos over the years.
Well, they are basically trying to be some sort of mixture between a whiteboard and some of the functionality that we talked about in Korea earlier where you basically have this creative space to throw a bunch of ideas together and organize them in a single spot.
To use the new studio, all you have to do is go to the Kyber website and go ahead and click create. Now, the default canvas that they have inside of Super Studio is actually really confusing and hard to use, so I don't recommend using that default one. You can just go to the side menu, click the plus icon, or click on a new canvas to create a new canvas.
So we're inside of our canvas here, and it's pretty interesting. It works by going to this little menu at the top that says flow menu. You can go ahead and click that button, and there are basically these flows, which are kind of like nodes that allow you to perform a certain creative task.
For example, we have Flux for generating an image. In Flux, we have Luma video, Runway video, and then all sorts of other Kyber-related tools, like image upscaling and video restyling, things that you may be used to if you're used to working with Kyber.
I'm going to ignore those for now, so let's focus on generating our image first. So let's select Flux, and you see here we have the ability to type in a subject. I’m going to say “a cinematic still of a dog on a patio.” Real creative, right?
You have the ability to adjust the aspect ratio, width, height, all that good stuff, and when you're ready to generate, all you have to do is select the little gummy guy here, and I love him! He kind of looks like a grape-flavored dot. You remember those old candies that are like super hard to get out of your teeth? He looks exactly like one of those with a smiley face, which is just delightful.
And you can see we generated an image of a dog on a patio here. Looks pretty good! You can, of course, just download that directly to your machine if you wanted to.
Now, the other thing that makes this pretty interesting is having the ability to bring in a node that allows you to use Runway or Luma. So for example, if we wanted to use Luma, we can just select Luma here, and we can drag and drop this image into the start keyframe.
You can see that it loaded it into the start keyframe here, which is pretty cool! And now you can basically just type in a prompt like you would if you were prompting inside of Luma. So we can say “a dog looks around on a patio, handheld footage.”
You do have the ability to select Loop video if you wanted to, and you also can select an end keyframe if you wanted to as well. We're not going to do that, and we will go ahead and select the gummy guy here to generate our video.
And our video has been generated! Let's take a look here. We have our dog on the patio; he’s kind of looking around. Yeah, not too bad!
I also have a few other examples I want to show you real quick. I wanted to test out the dual keyframe feature inside of Kyber. So we have shot number one here and shot number two. We want to interpolate between those shots, and we have this scene here which looks pretty good.
The particle dynamics? They’re not bad; they’re not perfect, but it did a pretty good job there. And we also have this shot here of this ghost character, which is perfect for a Halloween film, and there's actually quite a bit of fidelity there too. So not too bad!
And we also tried generating a video using Kyber's Video Lab module, and this is the result we got. So not great. It is interesting that it can generate a clip up to 17 seconds long just from my prompt alone, but the quality is just not great, so I don't really recommend using it.
It's cool to see different tools using different approaches to the creative process. While I personally don't think that we will recommend using Kyber at Curious Refuge just because it can kind of get disorganized really quickly, I think having the ability to create images and videos and have just this whiteboard creative space is an interesting way to go about the creative process.
So I recommend checking out the tool. If you like it, you can use it; if you don't like it, you totally don't have to use it, and you can go back to just traditional folder structures for organizing your content.
In the world of Hollywood, there was some pretty interesting news that the production company Blumhouse has partnered up with Meta to give artists access to the new Meta Movie Gene tool. As part of the announcement, Meta released an AI short film called “I Hate AI” that was created by Anish Chaganty.
The film is pretty interesting because it takes an old film that he created whenever he was 10 and basically creates visual effects that create an entire world that the film can live in. Blumhouse said that they will integrate Movie Gene into their productions in 2025.
This is, of course, really important news because Blumhouse is not the first Hollywood studio to widely integrate AI into their productions. Obviously, at Curious Refuge, we've trained thousands of people in the industry to use AI in their day-to-day work. But there was also news from Lionsgate just a month ago where they're working with Runway to train their own AI model.
Very soon, AI will be completely normal in the Hollywood production process, and artists like yourself that know how to use the tools to maximum creative effect will have many opportunities available to them.
In voice news, the team at 11 Labs has released the ability to type in a prompt and generate a completely new voice. So obviously, they have a voice library that has a ton of voices that you can use on your creative projects, but this one goes one step further where you can actually type in a prompt and hear a voice.
To use this feature, all you have to do is go to the 11 Labs website and click “Voices” on the left side of the screen and click “Add a New Voice.” From here, just click the Voice Design button, and you can type in a prompt.
For our prompt, we’ll say, “an angry old pirate with a British raspy voice,” and the text to preview will just be, you know, just a line here, and we will go ahead and generate that voice.
And here is our result:
“I’ll have none of your mutinous talk, ye scurvy rats! You’ve got two choices: go back to your chores or find yourself a place at the bottom of Davey Jones’s Locker!”
Not too bad honestly! And we can take that and integrate it with Runway lip sync (that's not Runway Act One just yet). Then we generate this video:
“I’ll have none of your mutinous talk, ye scurvy rats! You’ve got two choices: go back to your chores or find yourself a place at the bottom of Davey Jones’s Locker!”
Amazing! So you can see that having the ability to have fine-tuned control and basically combine different voices together to create a completely new voice is pretty interesting.
I should also note that they announced the API will be available in about a week, so you'll be able to do some real-time generation of brand new voices and basically prompt again and again to create characters in an entertaining experience.
Now that’s kind of the beginning of what I can imagine will be prompt to entertainment, where you want to watch a story about a raspy British pirate, and the voice that will be powering that story will be automatically generated.
We’re probably a couple of years away from that being in real-time, but we are probably only 6 months away from actually being able to prompt and watch something that is pretty interesting using technology like this.
The team at Comfy UI also announced that they are releasing a Mac version of their application that you will be able to use directly on Mac machines. The reason why this is important is that, up to this point, it’s been very challenging to run Comfy UI locally on a Mac machine.
You've basically had to use a cloud instance of the tool. The tool will come preconfigured with the Python environment and really everything you need to get going with Comfy UI. You can click the link below this video to join the waitlist, and of course, you will be the first to know when it goes live directly here on Curious Refuge.
And there is some wild news from the team at Anthropic this last week. Essentially, they announced that they are releasing AI agents that have the ability to control the mouse and take actions on your computer.
So you can type in a prompt, and the system will basically control the mouse and do whatever you ask it to do. There are some pretty interesting examples. For example, in the announcement video, Anthropic basically had the system create an entire website from scratch.
There's also a really funny example of somebody using the feature to skip YouTube ads, which is just delightful! This is basically the beginning of having AI digital agents. You can ask them to do tasks, and they will go off and do the task for you.
Now it’s not perfect at this point, and there's a lot of room for this to go wrong. So I imagine that if you're working with sensitive information, or if the system goes crazy and just deletes a bunch of files on accident, that could be problematic.
But I can imagine that very soon, especially because OpenAI is already working on agents to do automated tasks, we will have AI systems that can do a lot of the grunt work that we do whenever we’re working on our computer.
And on that note, I want to remind you that enrollment for the November session at Curious Refuge opens on October 30th. That is next Wednesday at 11:00 a.m. Pacific time.
This is the very first session where we are going to offer AI animation. We partnered up with Storybook Studios to create the most cutting-edge AI animation course in the world. Not only will you learn how to create animations from scratch, including how to create environments and custom characters, but you'll also learn essential techniques like rigging, character tracking, and scene development.
This is the essential course if you want to learn how to create animated films but also sell them because we are going to show you how to train custom models that allow you to own the final pixels on screen.
Be sure to go over to the Curious Refuge website if you want to learn more. But I should also note that we will be opening up enrollment for AI filmmaking, AI advertising, and AI documentary. So if you're interested in joining any one of those four courses, we would love to have you inside of the program!
Voting has also closed for our second annual AI horror film competition. We had around 500 horror films uploaded for this competition, and our judges are reviewing these submissions. The winner will receive $3,000 in prize money, thanks to the folks at Infinity AI, and of course, you will also get a cursed potato, which is one of the best awards I have ever seen!
We will announce the winners of this competition early next week. There are also a ton of AI meetups happening around the world. I want to give a shoutout to the Curious Refuge Meetup that happened in Stockholm just a few days ago.
We also have a meetup in Los Angeles on November 6th, so come by if you'd like to say hello! There's a Curious Refuge Meetup in West Palm Beach on November 7th, and we are hosting our first Meetup in Chicago on November 14th.
So if you happen to be in any one of those areas, be sure to stop by and meet some new friends in the world of AI!
And that brings us to our AI films of the week. The first film that I want to highlight is truly a moment in the world of AI filmmaking, and I am of course talking about “Where the Robots Grow” by AI Mas Studios.
This is, from what we can tell, the very first AI animated feature film. The team basically put together the film using a combination of different technologies. From what we've seen so far, it looks like it was a combination of Wonder Dynamics with simple 3D environments and probably some other AI tools to just add texturing and generally create the world in which the characters live.
They actually said they put the project together with a team of 10 people, and they even released it for free over on YouTube, really as a marketing play to get studios interested in their studio, which might be a pretty good strategy because it's had a lot of people talking about it.
I highly recommend checking out the film; I think they did an amazing job. The next film that I want to highlight is called “Those Who Remain” by Cavin the Kid. Cavin is one of the best AI curators out there.
The project was put together using Cing. It basically has zombies and robots and ninjas in this epic battle world, and I also think it's a wonderful example of fantastic sound design. Sound goes a long way in enhancing your AI visuals and just creating the larger world in which your project lives, and Cav really has a strong grasp of sound editing.
The last film that I want to highlight is called “Last Stop” by Mean Orange Cat. Mean Orange Cat is created by Don Teal, who created some bonus lessons inside of our AI filmmaking course. The project was created for the Culver Cup competition, and it really just showcases the mixture of having a really good story and script in conjunction with really sound AI technical skills.
Thank you so much for watching this week's episode of AI Film News! Of course, if you want to subscribe here on YouTube to get AI Film News and tutorials directly here on the platform, be sure to like and subscribe.
You can of course get AI Film News sent directly to your inbox. Not only will you get AI Film News sent to your email inbox each week, but you'll also get free access to an Intro to AI Filmmaking course.
And if you ever have a project that you would like to submit for our AI films of the week, we have a submit button on our AI Film Gallery page. If you ever have news that the community should know about, be sure to hit us up at hello@curiousrefuge.com.