Placeholder Image

Subtitles section Play video

  • So guys, just yesterday we saw what is perhaps the largest competition to OpenAI's incredible Sora video generator model yet.

  • A lot of you probably have already seen videos about it, but I want to bring you some new and interesting use cases and value propositions that you might not see in other places.

  • So for those of you who don't know, this is Gen 3, which is produced by RunwayML up here.

  • Now, RunwayML is special in the AI video space because they were the first ones to make the actual first commercial video generation model.

  • And now they have Gen 3 Alpha, so the third iteration.

  • It's a step towards building general world models.

  • And yeah, right off the bat, you can see this thing is damn impressive.

  • I mean, really, really good.

  • Edges close to Sora.

  • I think in the motion department, it does struggle a tad.

  • But I mean, all of these examples are really, really quite impressive.

  • Take a look at this one, for example.

  • I think it doesn't have necessarily the same fidelity that Sora has, but it looks really, really cool and the motion is super impressive.

  • Or even something like this.

  • This has a little bit better detail in my mind, and it's really cool to see how temporally consistent the buildings are as they pass by the camera.

  • And I mean, you can do very special effect-esque style things as well.

  • The possibilities, as always with AI video, are truly endless.

  • So apparently, it's been trained with highly descriptive, temporally dense captions, enabling imaginative transitions like, you know, a street being flooded with water.

  • Not something you're going to see every day.

  • Or, you know, we have like this drone shot that moves through a castle.

  • This looks pretty cool.

  • Looks like real GoPro footage.

  • I mean, that is really impressive.

  • That almost looks legit.

  • Like, I would believe that that is an actual video, not some AI generated thing.

  • Here we can see some water glistening on a window through a train as it passes by.

  • Again, wouldn't really be able to tell right off the bat that that is AI.

  • Here is another one that people found really, really impressive.

  • This is clearly, you know, some more GoPro footage of someone walking through some tunnels, and then it pans over and you see a runway.

  • Look at the effect of the flashlight as it moves over the graffiti on the runway.

  • I mean, you can tell it's a different texture there.

  • It's really impressive stuff.

  • I mean, no doubt in my mind, all of these examples blow me away, and I cannot wait until we get access.

  • That's the thing.

  • We still don't have access to this, but it appears that access is going to be coming to Gen 3 very, very soon.

  • And I can imagine people are going to be willing to pay a lot for access to this thing because it is the closest thing to Sora that we've seen so far in terms of like prompt following, coherency, and temporal stability.

  • It is something else.

  • Photorealistic humans, that's obviously a really big deal when you're trying to tell stories.

  • Humans are most often a part of a lot of stories that are told in filmmaking, TV shows, etc.

  • So there's some really great-looking humans in here, and obviously that is some bias in the training data there.

  • They purposefully set out to make sure that it's going to be able to produce some pretty realistic-looking people.

  • And they absolutely have done that.

  • I mean, look at how cinematic this scene looks, and look at the glare of the sun and everything.

  • So impressive.

  • One thing I will say that I am seeing a lot is pretty much every one of these videos, no matter what it is, if it's a person or like this over here, everything looks like it's being shot in slow motion, which isn't necessarily a downside because, think about it, you can always speed up a video.

  • If for some reason it produces videos that look like they're in slow motion, well, we can always just speed up the video.

  • Like, that totally looks like it's in slow motion, no?

  • Maybe they purposefully trained it on slow motion video?

  • Like, we could just download these videos and speed them up and they would look completely normal.

  • I mean, shockingly realistic, but I just find that really intriguing.

  • I mean, there we have like a monster walking around.

  • That looks like a shot from a movie legitimately.

  • I mean, it's so impressive.

  • The way that the light bounces and reflects off of the fur, it is so darn cool.

  • Here's like some, I don't know, rock person walking through the woods.

  • Like, this thing has an understanding of physics and the world in a way that reminds me of the Sora demos.

  • This is a true competitor to Sora, I think, more so than anything else we've seen.

  • And that's really, I think, what is causing all of the stir around Gen 3.

  • I keep seeing people talk about this.

  • I mean, yeah, check that out.

  • I mean, that's some, that's a pretty good understanding of physics and paper and how it's going to interact in this alleyway with the wind tunnel.

  • It's super cool.

  • Here's another one that again looks like it's in slow motion.

  • I mean, why does it look like it's so in slow motion?

  • I don't get that part about it.

  • But the way he like moves his thumb and everything, that just looks so legit.

  • And these are obviously going to be cherry picked here.

  • Detailed close-up of bacteria.

  • That's so cool.

  • I mean, that looks so legit, man.

  • That's just some crazy, crazy concepts.

  • It's exciting.

  • It is exciting, no doubt.

  • See different styles.

  • Here's like an anime, animated film art style.

  • Really cool stuff.

  • They've really gotten close to solving motion at this point.

  • I mean, it's just, again, here's another one that looks like it's in slow motion.

  • Super cool though.

  • I mean, what would you guys create if you had access to this?

  • It seems like access is going to be coming.

  • At least it was hinted in like a few days.

  • We're going to be able to get access to this.

  • So I'd love to do like a live stream, testing it out, hanging out with you guys, doing some prompts.

  • I don't mind paying for credits to use something like this because it's groundbreaking technology.

  • Like I said, biggest Sora competitor that we've seen so far, at least in my mind.

  • Anyways, that's sort of the TLDR on Gen 3.

  • Now let's get into some of the more interesting use cases.

  • Some of the bigger value propositions that you might not see in other places.

  • So Cristobal here on Twitter brings up that there's a lot of unexplored latent horror to discover.

  • So the applications for horror movies or horror genres are very, very much already alive, as we can see by this terrifying generation.

  • He does a few other generations.

  • Like we've got a monkey here who is learning to play guitar on the street.

  • And again, it's just a really nice smooth shot.

  • Looks like it's in slow motion.

  • I don't know what it is with runway and the slow-mo stuff, but it's super weird.

  • Oh yeah, he works for runway, so that's why he's got access to it.

  • And you can see drink water over here.

  • So the text is also really impressive.

  • It's able to do like these animated things with text popping up on the screen.

  • I mean, that's really darn cool.

  • Some more examples of this.

  • Look at the reflections on the stones, though.

  • That's not something that is easy to replicate in a 3D animation environment.

  • That's really, really crazy.

  • And that's one of like the impressive things about AI is that those more complex things that you would normally do in 3D animation are actually a lot easier in AI.

  • And the simple stuff in animation is the most difficult part in the AI.

  • And here we go.

  • We've got like some animation with snow in the mountains and text comes through and says, ice, ice, baby.

  • And yeah, I mean, he is really, really showing off the fact that you can produce really good looking text.

  • Can't sleep coming down there in sand.

  • And here we have another example where it just says, Timmy.

  • And it's like this blasting dirt flying into view.

  • It's really awesome stuff.

  • I would love to try some MatVidPro ones.

  • I mean, most of these are pretty simple words that he's trying in here.

  • So I'd like to see how it does with more complex stuff like a username.

  • And here you can see it does text in novel situations, not just where you would expect to see it.

  • So we have a giant maybe on top of, you know, an exit sign on the highway or something.

  • It looks like there's some interesting car accidents and really, really weird craziness going on in the corner here.

  • So these models, keep in mind, are by no means perfect.

  • But this absolutely competes with the best of the best, which is Sora.

  • And the list kind of just goes on for you guys.

  • I mean, you can see on top of a train, giant danger sign.

  • It obviously looks edited, but it gets the point across.

  • This was incredible.

  • I mean, this is clearly like a bowl of soup or something.

  • And then you can see these little letters drop in.

  • And the way that the physics and the letters move with like the liquid, it just looks so realistic.

  • And it looks like it's not been created in a machine or a physics environment.

  • It looks like it was created or actually filmed in real life.

  • And that's really one of the more interesting parts about AI generated video that you're going to notice is that since it is trained on real footage, it gets a lot more close to realism in the physics department than you'll see in like actual 3D animation, because it's something that is really hard to replicate.

  • Again, we have more of that latent horrors type of thing.

  • We got Pytorch written on a wall here, but it's someone who is walking through the woods and then comes across something.

  • Here we've got coffee pouring into a mug.

  • This is by Emily Golden, another member of the runway staff.

  • So it seems like they all have access to it, but there's just so many crazy things that can go on.

  • Here, folks, is yet another impressive display of Gen 3's knowledge capabilities.

  • We take this ice cube and we drop it on a searing hot pan, and immediately it starts to, well, melt into water, which shows that the model has a very intelligent and nuanced understanding of how physics work and interact with our natural world.

  • So the ice isn't necessarily melt, but it definitely produces the water.

  • We're getting a lot closer to a model that can generalize and think about physical interactions in our world very well.

  • So again, coming straight from runway ML here, we do have some specs.

  • So it takes about 90 seconds to get a 10-second video, which is not too shabby at all.

  • That's very quick generation, can generate multiple videos at once.

  • They also are going to be adding motion brush, advanced camera controls, and director mode to this.

  • And they're also going to have more fine-grained control over structure, style, and motion.

  • I mean, again, we didn't really get this kind of announcement with Sora.

  • Sora is very much kept under wraps as an AI video generator, and we're already getting all of these crazy other generators cropping up, and Gen 3 seems to be the most fully-fledged of the bunch.

  • Here is a nice example of some 3D animation of an anime girl.

  • This one's a little bit creepy.

  • The eyes and the face are very consistent, and the hair and everything, it looks good, but it's creepy.

  • It looks like a doll or something.

  • Here we've got some footage of some fire burning in the distance.

  • Again, this looks super real, almost scary real.

  • People might think that this is a genuine actual video that was recorded.

  • Lots of really great cinematic stuff as well.

  • Again, some of it does delve into the more creepy side of things, like we've got giant ears on someone.

  • This just looks weird.

  • I don't know.

  • It's just a strange generation.

  • It freaks me out a little bit.

  • Here they kept also showing off this example.

  • You might have seen this floating around, where we've got slow-motion video of a wig and glasses dropping perfectly on someone's head, and they did this with multiple different people.

  • It's clear that they have a pretty precise ability to at least manipulate a good seed, which is nice to see.

  • Here, that is the one that kind of went viral here, where it's this really surprised face of a guy, and the wig and glasses drops right on his head.

  • This one's really cool.

  • So a hidden civilization pops out in the clouds, and it looks so legit.

  • I mean, again, scenes from movies.

  • That's what a lot of this looks like.

  • I have no doubt in my mind that it was trained on cinematic prompts.

  • The training data has got to be really good, just like Sora's was.

  • Their runway is like right behind OpenAI.

  • Right behind OpenAI, I would say.

  • This is by always editing on Twitter here.

  • This shows a little bit of sound design to go along with a video and really show how you can make this AI-generated content come to life in a way that is no BS.

  • You know, it's going to be usable much sooner than you think.

  • When you're ready, gently open your eyes.

  • Returning to your surroundings.

  • Now carrying this light.

  • So you can see a little bit of sound design goes quite a long way.

  • I really think that creative minds are going to have a field day with this kind of technology.

  • I mean, it is going to be this incredible blossoming of expression in all kinds of forms now available to people who previously never had access to the money, the tools, the knowledge required to create such a thing.

  • You can also see always editing here.

  • Another runway employee says access coming to everyone soon.

  • AmoebaGPT on Twitter also did a little bit of a side-by-side comparison for us that I think is very telling about the quality difference between Gen 3 and the Luma AI generator.

  • Of course, Dream Machine, which we just talked about on this channel.

  • So the prompt for this astronaut running through an alley in Rio de Janeiro.

  • And you can see that there is just no contest really here.

  • The runway ML generation looks way better.

  • It still makes sense for Luma Labs and the motion isn't bad and everything, but it's just not nearly as coherent.

  • It doesn't look nearly as realistic.

  • So I think it's just so obvious that runway ML is way, way better.

  • Dragon Toucan walking through the Serengeti.

  • Again, I mean, the Luma AI Dream Machine prompt is not bad, but the runway ML has a much more realistic looking bird.

  • And the way that the bird is walking as well.

  • I mean, it's just, it's a little bit more, I don't know, birdie.

  • Does that make sense?

  • It looks more like a real wild animal.

  • And I'll also note that the detail in the background is a lot better on the runway prompt as well.

  • To be fair though, these are probably all cherry-picked runway generation.

  • So you're going to want to keep that in mind as we go through these.

  • Subtle reflections of a woman on a window of a train moving at hyperspeed in a Japanese city.

  • Again, you know, there are pretty much no subtle reflections, I would say, in either of these.

  • I don't know if that's just because of the compression on Twitter here, but it's a more realistic image of a woman's face overall.

  • This one also, you can tell.

  • First-person view flying through a colorful coral-lined street of underwater suburban neighborhood.

  • I mean, this is a pretty complex prompt.

  • It has to combine the underwater style with flying through a neighborhood.

  • And clearly, Luma doesn't have the smarts, let's say, to produce this, where Runways Gen 3 absolutely does.

  • I mean, this looks like an underwater coral-infested suburban neighborhood.

  • The other one just looks like a regular plain old suburban neighborhood, pretty much.

  • Handheld tracking shot following a dirty blue balloon floating above an old European street.

  • And I mean, both of these do adhere to the prompt pretty well, but the Runway Gen 3 video looks a little bit more cinematic.

  • It's a nicer shot.

  • And it is also a tracking shot where this one up top is not a tracking shot.

  • Again, though, could all be cherry-picked.

  • Probably all is definitely cherry-picked.

  • So that's just something you're going to want to keep in mind.

  • So guys, let's ponder here on what this means for the broader AI video generation spectrum.

  • I think it's quite clear by now that 2024 is the year of AI-generated video really, really picking up.

  • There's a lot of competition in the space now.

  • We've got, obviously, OpenAI with the king Sora AI video, supposed to be releasing this year.

  • We've got Gen 3, which we just saw, really competitive with Sora, really impressive, about to release publicly.

  • We've also got the Chinese Cling AI video generator that, again, kind of looks like it competes with Sora, maybe a little bit worse than Gen 3, but made in China and is available to, I guess, a set of test users right now, some alpha users.

  • We'll eventually be public in the form of an app.

  • We've also got the LumaLabs Dream Machine, which, again, maybe not as good as the others, but it's publicly available right now.

  • You can use it today.

  • You have limited free generations, and it's pretty darn cool, much more impressive than the likes of any previous video generation model we've seen.

  • It's making leaps and bounds in not years, but months, and sometimes probably even weeks, which shows you the rate of development of this AI technology is not halting at all.

  • It's getting faster and faster, and it's both exciting and maybe a little bit worrisome in some areas.

  • I do think, though, that now that there are three really competitive third-party generators, third party, OpenAI is going to now have to reconsider its strategy.

  • I think it wanted to release Sora probably more around December, but to stay relevant, I mean, Sora's... no one's going to really care too much about Sora if we already have access to Gen 3, I think, unless Sora releases with better quality than OpenAI showed us.

  • I don't know.

  • This is definitely, I think, hopefully going to force OpenAI's hand, but we don't really know.

  • They might not really care too much about these smaller companies.

  • Maybe they're working on something that we don't know about yet.

  • Maybe GPT-5 is a bigger deal than we think.

  • And this leads me into maybe a few other pieces of news that I want to share with you, just to kind of help keep you guys in the loop.

  • ComfyUI just started Comfy.org.

  • Essentially, they're going to take ComfyUI to the next level.

  • If you don't know what ComfyUI is, essentially it was this easy node-based platform for developing with Stable Diffusion, etc.

  • Stable Diffusion 3 control net is also here, and it looks pretty freaking incredible.

  • The ability to adjust images here, I mean, that's a whole different level of control.

  • Really impressive.

  • Also a really, really strange tweet from the one and only Jimmy Apples here, who oftentimes has really accurate insight into open AI.

  • Are you ready to ignite some innovation?

  • It's all the rage in boardrooms across the country.

  • With its smooth, sassy flavor, you'll be leaving everyone craving for more.

  • Rolling out in coming weeks.

  • And it's an AI-generated image that says Marlboro, Sam Altman, obviously supposed to be cigarettes here.

  • Looks like we're going to get some new form of image generation.

  • This is probably the GPT-4.0 image generation capability, maybe rolling out in the few coming weeks.

  • I think that's what Jimmy Apples is hinting at here.

  • Also, GPT-4.0 Omni got a silent update that allows it to generate multiple images in one prompt and also generate a lot more text at once.

  • Haven't confirmed this myself, but apparently this is a new thing.

  • So let me know if you guys have also noticed this.

  • Oh, and also don't forget that Luma AI with their Dream Machine is going to be adding more fine-tuned controls that allow you to explore more concepts in like a very native way.

  • Essentially, they teased that coming soon, you're going to be able to more or less in-paint or edit your videos.

  • Like we're keeping the same little kid all throughout all of this, but we're changing the background quite consistently.

  • So I don't know.

  • There's a lot more going on than meets the eye, even with Luma Labs Dream Machine.

  • Much more precise edits are going to be possible, like changing that little boy into like a more fantasy character, something like that.

  • I mean, there's a lot going on.

  • So yeah, folks, I hope this video served as like a nice recap of everything that's kind of going on in the AI space.

  • Obviously, Gen 3 is like the biggest announcement.

  • This is essentially like an actual Sora, real, real Sora clone that we can have access to, hopefully in a few days.

  • Trying to get more information on that, but I have only heard whispers.

  • Thank you so much for watching, folks.

  • I will see you in the next video and goodbye.

So guys, just yesterday we saw what is perhaps the largest competition to OpenAI's incredible Sora video generator model yet.

Subtitles and vocabulary

Click the word to look it up Click the word to find further inforamtion about it