Placeholder Image

Subtitles section Play video

  • Ladies and gentlemen, I have a very special guest, but could I ask everybody to sit down?

  • We're about to get started.

  • My next guest, I am so impressed by this person, three reasons.

  • First reason is there are only a handful of entrepreneurs, founders that started a company that literally touched the lives of billions of people around the world as part of the social fabric, invented services, and a state-of-the-art computing company.

  • Two, very few entrepreneurs, founders founded the company and led it to over a trillion dollars of value.

  • And three, a college dropout.

  • All three things simultaneously true.

  • Ladies and gentlemen, please help me welcome Mark Zuckerberg.

  • Mark, welcome to your first SIGGRAPH.

  • Can you believe this?

  • One of the pioneers of computing, a driver of modern computing, and I had to invite him to SIGGRAPH.

  • So, anyways, Mark, sit down.

  • It's great to have you here.

  • Welcome.

  • Thanks for flying down.

  • Yeah, no, this will be fun.

  • I hear you've been going for like five hours already or something.

  • Well, yeah, sure.

  • This is SIGGRAPH, you know, these 90% PhDs.

  • And so the thing that's really great about SIGGRAPH, as you know, this is the show of computer graphics, image processing, artificial intelligence, and robotics combined.

  • And some of the companies that over the years have demonstrated and revealed amazing things here from Disney, Pixar, Adobe, Epic Games, and of course, you know, NVIDIA.

  • We've done a lot of work here.

  • This year we introduced 20 papers at the intersection of artificial intelligence and simulation.

  • So we're using artificial intelligence to help simulation be way larger scale, way faster.

  • For example, differentiable physics.

  • We're using simulation to create simulation environments for synthetic data generation for artificial intelligence.

  • And so these two areas are really coming together.

  • Really proud of the work that we've done here at Meta.

  • You guys have done amazing AI work.

  • I mean, one of the things that I find amusing is when the press writes about how Meta's jumped into AI this last couple of years.

  • As if, you know, the work that FAIR has done, remember, we all use PyTorch.

  • That comes out of Meta.

  • The work that you do in computer vision, the work in language models, real-time translation, groundbreaking work.

  • I guess my first question for you is, how do you see the advances of generative AI at

  • Meta today, and how do you apply it to either enhance your operations or introduce new capabilities that you're offering?

  • Yeah.

  • So, a lot to unpack there.

  • First of all, really happy to be here.

  • Meta has done a lot of work and has been at SIGGRAPH for eight years.

  • So, we're noobs compared to you guys.

  • But I think it was back in 2018...

  • You're dressed right, but this is my hood.

  • I just wanted to...

  • I mean, well, thank you for welcoming me to your hood.

  • I think it was back in 2018, we showed some of the early hand-tracking work for our VR and mixed reality headsets.

  • I think we've talked a bunch about the progress that we're making on codec avatars, the photorealistic avatars that we want to be able to drive from consumer headsets, which we're getting closer and closer to.

  • So, pretty excited about that.

  • And also, a lot of the display systems work that we've done.

  • So, some of the future prototypes and research for getting the mixed reality headsets to be able to be really thin, with just pretty advanced optical stacks and display systems, the integrated system.

  • That's been stuff that we've typically shown here first.

  • So, excited to be here this year, not just talking about the metaverse stuff, but also all the AI pieces, which, as you said, we started FAIR, the AI research center.

  • Back then it was Facebook, now Meta, before we started Reality Labs.

  • We've been at this for a while.

  • All the stuff around Gen AI, it's an interesting revolution.

  • And I think that it's going to end up making all of the different products that we do different in interesting ways.

  • You can look at the big product lines that we have already, so things like the feed and recommendation systems and Instagram and Facebook.

  • We've kind of been on this journey where that's gone from just being about connecting with your friends.

  • And the ranking was always important, because even when you were just following friends, if someone did something really important, like your cousin had a baby or something, it's like, you want that at the top, you'd be pretty angry at us if it was buried somewhere down in your feed.

  • So, the ranking was important, but now, over the last few years, it's gotten to a point where more of that stuff is just different public content that's out there.

  • The recommendation systems are super important, because now, instead of just a few hundred or thousand potential candidate posts from friends, there's millions of pieces of content.

  • And that turns into a really interesting recommendation problem.

  • And with generative AI, I think we're going to quickly move into the zone where not only is the majority of the content that you see today on Instagram just recommended to you from stuff that's out there in the world that matches your interests and whether or not you follow the people.

  • I think in the future, a lot of this stuff is going to be created with these tools, too.

  • Some of that is going to be creators using the tools to create new content.

  • Some of it, I think, eventually is going to be content that's either created on the fly for you or pulled together and synthesized through different things that are out there.

  • So I think that's just one example of how the core part of what we're doing is just going to evolve.

  • And it's been evolving for 20 years already.

  • Well, very few people realize that one of the largest computing systems the world has ever conceived of is a recommender system.

  • Yeah.

  • I mean, it's this whole different path, right?

  • It's not quite the kind of gen AI hotness that people talk about, but I think it's all the transformer architectures and it's a similar thing of just building up more and more general models.

  • Embedding unstructured data into features.

  • Yeah.

  • One of the big things that just drives quality improvements is it used to be that you'd have a different model for each type of content, right?

  • So a recent example is we had one model for ranking and recommending reels and another model for ranking and recommending more long form videos.

  • And then it takes some product work to basically make it so that the system can display anything in line.

  • But the more you just create more general recommendation models that can span everything, it just gets better and better.

  • So, I mean, part of it I think is just like economics and liquidity of content and the broader of a pool that you can pull from.

  • You're just not having these weird inefficiencies of pulling from different pools.

  • But yeah, I mean, as the models get bigger and more general, that gets better and better.

  • So I kind of dream of one day, like you can almost imagine all of Facebook or Instagram being like a single AI model that has unified all these different content types and systems together that actually have different objectives over different timeframes, right?

  • Because some of it is just showing you, you know, what's the interesting content that you're going to be, that you want to see today.

  • But some of it is helping you build out your network over the long term, right?

  • People you may know or accounts you might want to follow.

  • These multimodal models tend to be much better at recognizing patterns, weak signals and such.

  • And so one of the things that people always, you know, it's so interesting that AI has been so deep in your company.

  • You've been building GPU infrastructure, running these large recommender systems for a long time.

  • Now you're, now you're...

  • I'm a little slow on it, actually.

  • Getting to GPUs.

  • Yeah.

  • I was trying to be nice.

  • I know.

  • Well, you know, too nice.

  • I was trying to be nice, you know, you're my guest.

  • When I was backstage before I came on here, you were talking about like owning your mistakes or something.

  • Right?

  • So...

  • You don't have to volunteer it out of the blue.

  • I think this one has been well tried.

  • Yeah.

  • It's like I got raked over the shoulder for a while, you know?

  • But as soon as you got into it, as soon as you got into it, you got into it strong.

  • Let's just put it...

  • There you go.

  • There you go.

  • Now, the thing that's really cool about generative AI is these days when I use WhatsApp, I feel like I'm collaborating with WhatsApp.

  • I love imagine.

  • I'm sitting here typing and it's generating the images as I'm going.

  • I go back and I change my words, it's generating other images, you know?

  • And so, the one that old Chinese guy enjoying a glass of whiskey at sundown with three dogs, golden retriever, golden doodle, and a Bernese mountain dog.

  • And it generates, you know, a pretty good looking picture.

  • Yeah.

  • Yeah.

  • We're getting there.

  • That's me.

  • Every month.

  • Yeah, yeah.

  • And then now you could actually load my picture in there and it'll actually be me.

  • Yeah.

  • That's as of last week.

  • Yeah.

  • That's me.

  • I know.

  • I'm spending a lot of time with my daughters imagining them as mermaids and things over the last week.

  • It's been a lot of fun.

  • Yeah.

  • But yeah, I mean, that's the other half of it.

  • I mean, a lot of the gen AI stuff is going to...

  • On the one hand, it's, I think, going to just be this big upgrade for all of the workflows and products that we've had for a long time.

  • But on the other hand, there's going to be all these completely new things that can now get created.

  • So, meta AI, you know, the idea of having, you know, just an AI assistant that can help you with different tasks in our world is going to be, you know, very creatively oriented, like you're saying.

  • But, I mean, they're very general, so you don't need to just constrain it to that.

  • It'll be able to answer any question.

  • Over time, I think, you know, when we move from, like, the LLAMA3 class of models to

  • LLAMA4 and beyond, it's going to, I think, feel less like a chat bot where it's like, you give it a prompt and it just responds, and then you give it a prompt and it responds, and it's just, like, back and forth.

  • I think it's going to pretty quickly evolve to, you give it an intent, and it actually can go away on multiple timeframes, and, I mean, it probably should acknowledge that you gave it an intent up front, but, I mean, you know, some of the stuff, I think, will end up, you know, it'll spin up, you know, compute jobs that take, you know, weeks or months or something, and then just come back to you when, like, something happens in the world, and I think that that's going to be really powerful.

  • So, I mean, I'm quite...

  • Today's AI, as you know, is kind of turn-based.

  • You say something, it says something back to you, but, obviously, when we think, when we're given a mission or we're given a problem, you know, we'll contemplate multiple options or maybe we come up with a, you know, a tree of options, a decision tree, and we walk down to the decision tree simulating in our mind, you know, what are the different outcomes of each decision that we could potentially make, and so we're doing planning, and so in the future, AIs will kind of do the same.

  • One of the things that I was super excited about, when you talked about your vision of creator AI, I just think that's a home-run idea, frankly.

  • Tell everybody about the creator AI and AI studio that's going to enable you to do that.

  • Yeah, so we actually, I mean, this is something that we're, you know, we've talked about it a bit, but we're rolling it out a lot wider today.

  • You know, a lot of our vision is that...

  • I don't think that there's just going to be, like, one AI model, right?

  • I mean, this is something that some of the other companies in the industry, they're, like, you know, it's like they're building, like, one central agent, and yeah, we'll have the meta AI assistant that you can use, but a lot of our vision is that we want to empower all the people who use our products to basically create agents for themselves.

  • So whether that's, you know, all the many, many millions of creators that are on the platform or, you know, hundreds of millions of small businesses, we eventually want to just be able to pull in all your content and very quickly stand up a business agent and be able to interact with your customers and, you know, do sales and customer support and all that.

  • So the one that we're just starting to roll out more now is, we call it AI studio, and it basically is a set of tools that eventually is going to make it so that every creator can build sort of an AI version of themselves as sort of an agent or an assistant that their community can interact with.

  • There's kind of a fundamental issue here where there's, there's just not enough hours in the day, right?

  • It's like, if you're a creator, you want to engage more with your community, but you're constrained on time.

  • And similarly, your community wants to engage with you.

  • But it's tough.

  • I mean, there's just, there's limited time to do that.

  • So the next best thing is, is allowing people to basically create these artifacts, right?

  • It's sort of, it's an agent, but it's, you train it to kind of, on your material to represent you in the way that you want.

  • I think it, it's a very kind of creative endeavor, almost like a, like a piece of art or content that you're putting out there.

  • And it's going to be very clear that it's not engaging with the creator themselves, but I think it'll be another interesting way, just like how creators put out content on, on these social systems to be able to have agents that do that.

  • Similarly, I think that there's going to be a thing where people basically create their own agents for all different kinds of uses.

  • Some will be sort of customized utility, things that they're trying to get done that they want to kind of fine tune and, and train an agent for.

  • Some of them will be entertainment.

  • And some of the things that people create are just funny, you know, and just kind of silly in different ways, or, or kind of have a funny attitude about things that, you know, we probably couldn't, we probably wouldn't build into meta AI as an assistant.

  • But I think people, people are kind of pretty interested to see and interact with.

  • And then one of the interesting use cases that we're seeing is people kind of using these agents for support.

  • This was one thing that, that was a little bit surprising to me is one of the top use cases for meta AI already is people basically using it to role play difficult social situations that they're going to be in.

  • So whether it's a professional situation, it's like, all right, I want to ask my manager, like, how do I get a promotion or raise or I'm having this fight with my friend, or I'm having this difficult situation with my girlfriend, like how, like, how can this conversation go and basically having a, like a completely judgment free zone where you can basically role play that and see how the conversation will go and, and get feedback on it.

  • But a lot of people, they don't just want to interact with the same kind of, you know, agent, whether it's meta AI or chat GPT or whatever it is that everyone else is using.

  • They want to kind of create their own thing.

  • So that's, that's roughly where we're going with AI studio, but it's all part of this bigger, I guess, view that we have that there shouldn't just be kind of one big AI that people interact with.

  • We, we, we just think that the world will be better and more interesting if there's a diversity of these different things.

  • I just think it's so cool that if you're an artist and you have a style, you could take your style, all of your body of work.

  • You could fine tune one of your models.

  • And now this becomes an AI model that you can come and you could prompt it.

  • You could ask me to, you know, create something along the lines of the art style that I have.

  • And you might even give me a piece of art as a, maybe a drawing, a sketch as an inspiration, and I can generate something for you.

  • And it's, and you come to my, come to my you know, come to my bot for that, come to my

  • AI for that.

  • It could be, it could be every single, every single restaurant, every single website will probably in the future have these AIs.

  • Yeah.

  • I mean, I kind of think that in the future, just like every business has, you know, an email address and a website and a social media account or several.

  • I think in the future, every business is going to have an AI agent that interfaces with their customers.

  • Right.

  • Some of these things I think have been pretty hard to do historically.

  • Like if you think about any company, it's like, you probably have customer support is just a separate organization from sales.

  • And that's not really how you'd want it to work as CEO.

  • It's just that, okay.

  • They're kind of different skills.

  • You're building up these.

  • I'm your customer support just to be.

  • What's up?

  • I'm your, yeah.

  • Well, apparently I am.

  • Yeah.

  • I mean, whenever Mark needs something, I can't tell whether it's chat bot or it's Mark, but he does.

  • It's just my chat bot, just asking here.

  • Um, no, well, I guess that's kind of, yeah.

  • When you're CEO, you have to do all this stuff.

  • But, but I mean, then when you build the abstraction into your organization, a lot of times, like the, you know, in general, the organizations are separate because they're kind of optimized for different things, but I think like the platonic ideal of this would be that it's kind of one thing, right.

  • As a, you know, as a customer, you don't really care, you know, you don't want to like have a different route when you're trying to buy something versus if you're having an issue with something that you bought, you just want to have a place that you can go and get your questions answered and be able to engage with the business in different ways.

  • And I think that that applies for creators too.

  • I think that the, the kind of personal consumer side of this.

  • And all of that engagement with your customers, especially their complaints, it's going to make your company better.

  • Yeah, totally.

  • Right.

  • The fact that it's all engaging with this AI is going to capture the, the, uh, the institutional knowledge and how to, and all of that can go into analytics, which improves the AI and so on and so forth.

  • Yeah.

  • Yeah.

  • So the business version of this is, um, that I think has a little more integration and we're still in a pretty early alpha with that, but the AI studio making it so that people can kind of create their UGC agents and different things and getting started on this flywheel of having creators create them.

  • I'm pretty excited about that.

  • So can I, can I use AI studio to fine tune with my images, my collection of images?

  • Yeah, you're yeah.

  • We're going to get there.

  • Okay.

  • And then I could, can I give it loaded all the things that I've written?

  • So use it, use it as my rag.

  • Yeah.

  • Yeah.

  • Yeah.

  • Basically.

  • Okay.

  • Yeah.

  • And then every time I come back to it, it loads up its memory again.

  • So it remembers where it left off last time and we carry on our conversation as nothing ever happened.

  • Yeah.

  • And, and, and look, I mean, like any product, it'll get better over time.

  • The tools for training, it will get better.

  • It's not just about what you want it to say.

  • I mean, I think generally creators and businesses have topics that they want to stay away from too.

  • Right.

  • So just getting better at all this stuff.

  • Um, you know, I think the platonic version of this is not just text, right?

  • You, you almost want to just be able to have.

  • And then this is sort of an intersection with some of the codec avatar work that we're doing over time.

  • You want to basically be able to have almost like a video chat with, with the, um, with the, with the agent.

  • And I think we'll get there over time.

  • I don't think that this stuff is that far off, but the, um, the flywheel is spinning really quickly.

  • So it's, it's, it's, it's exciting.

  • Um, there is a lot of new stuff to build.

  • And I think even if the progress on the foundation models kind of stopped now, which I don't think it will, I think we'd have like five years of product innovation for the industry to basically figure out how to most effectively use all the stuff that's gotten built so far.

  • But I actually just think the, the kind of foundation models and the progress on the fundamental research is accelerating.

  • So, um, so that it's, uh, it's a pretty wild time.

  • Your vision, it's all, it's all, um, you know, you kind of made this happen.

  • So, well, thank you.

  • In the last conversation.

  • I thank you.

  • Yeah.

  • You know, you know, we're CEOs, we're, we're delicate flowers.

  • We need a lot of back.

  • Yeah.

  • We're, we're pretty grizzled at this point.

  • I think we're, we're the two kind of longest standing founders in the industry.

  • Right.

  • It's, I mean, it's true.

  • I mean, it's true.

  • It's true.

  • I just.

  • And your hair has gotten gray.

  • Mine has just gotten longer.

  • Mine's gotten gray.

  • Yours gotten curly.

  • What's up?

  • It was always curly.

  • That's why I kept it short.

  • Yeah.

  • I just, if I had known it was going to take so long to succeed.

  • You would never would have started.

  • No, I would have dropped out of college.

  • Just like you get a head start.

  • Well, that's a, that's a good difference between our personalities.

  • I think that these things, you got a 12 year head start.

  • That's pretty good.

  • That's pretty good.

  • You know, you're doing pretty well.

  • Uh, I'm going to, I'm going to be able to carry on.

  • Let me just put it that way.

  • Yeah.

  • So, so, um, I, the thing that I love about, about, um, your vision of that everybody can have an AI that every business can have an AI in our company.

  • I want every engineer and every software developer to have an AI.

  • Yeah.

  • And, um, or many eyes.

  • Uh, the thing that's that, that I love about your vision is you also believe that everybody and every company should be able to make their own AI.

  • So you actually open sourced, uh, when you open source Lama, I thought that was great Lama 2.1, by the way, I, I thought Lama two was probably the biggest event.

  • In AI last year.

  • And the reason for that, I mean, I thought it was the H 100, but you know, it's, uh, it's a chicken or the egg question.

  • That's a chicken.

  • Yeah.

  • Which came first?

  • The H 100.

  • Yeah, well, Lama two, it was, it was actually not the H 100.

  • Yeah.

  • It was a 100.

  • Yeah.

  • Thank you.

  • And so, so, um, uh, but, but the reason why I said it was the biggest event was because when that came out, it activated every company, every enterprise and every industry, all of a sudden, every healthcare company was building AIs.

  • Every company was building AI, every large company, small company startups were, were building AIs.

  • It made it possible for every researcher to be able to re-engage AI again, because they have a starting point to do something with, um, and, uh, and, and then now, uh, 3.1 is out and the excitement, just so you know, uh, you know, we're, we work together to, to, uh, uh, deploy, uh, 3.1, we're taking it out to the world's enterprise and the excitement is just off the charts.

  • And, and I, I think it's going to enable all kinds of applications, but tell, tell me about your, your open source philosophy.

  • Where'd that come from?

  • And, you know, you open source PyTorch and that it is now the framework by which AI is done.

  • And, and, uh, now you've open sourced Lama 3.1 or Lama.

  • Uh, there's a whole ecosystem built around it.

  • And so I think it's horrific, but where did that all come from?

  • Yeah.

  • So there's, there's a bunch of history on, on a lot of this.

  • I mean, we've done a lot of open source work over time.

  • Um, I think part of it, you know, just bluntly is, you know, we got started after some of the other tech companies, right, in building out stuff like the distributed computing infrastructure and the data centers, and, you know, because of that, by the time that we built that stuff, it wasn't a competitive advantage.

  • We're like, all right, we might as well make this open and then we'll benefit from the, from the ecosystem around that.

  • So we, we had a bunch of projects like that.

  • I think the biggest one was probably open compute, where we took our server designs, the network designs, and eventually the data center designs and published all of that.

  • And by having that become somewhat of an industry standard, um, all the supply chains basically got organized around it, which had this benefit of saving money for everyone.

  • So by making it public, um, and open, we basically have saved billions of dollars from doing that.

  • Well, open compute was also what made it possible for Nvidia HGXs that we designed for one data center also works in, yeah, works in every data center.

  • Awesome.

  • Um, so, so we, so that was an awesome experience.

  • And then, you know, we've done it with a bunch of our kind of infrastructure tools, things like React, PyTorch.

  • Um, so I'd say by the time that Llama came around, we were sort of positively predisposed towards doing this, um, for, for AI models specifically, I guess there's a few ways that I look at this.

  • I mean, one is, you know, it's been really fun building stuff over the last 20 years at the company.

  • Um, one of the things that, that has been sort of the most difficult has been kind of having to navigate the fact that we ship our apps through our competitors' mobile platforms.

  • So in the one hand, the mobile platforms have been this huge boon to the industry.

  • That's been awesome.

  • Um, on the other hand, having to deliver your products through your competitors, um, is challenging, right?

  • And I also, you know, I grew up in a time where, you know, the first version of Facebook was on the web and that was open and then, you know, as a transition to mobile, you know, the plus side of that was, you know, now everyone has a computer in their pocket.

  • So that's great.

  • The downside is, okay, we're a lot more restricted in what we can do.

  • So when you look at these generations of computing, there's this big recency bias where everyone just looks at mobile and thinks, okay, because the closed ecosystem, because Apple basically won and set the terms of that and like, yeah, I know that there's more Android phones out there technically, but like Apple basically has the whole market, um, and like all the profits and basically Android is kind of following Apple in terms of the development of it.

  • So I think Apple pretty clearly won this generation.

  • But it's not always like that, right?

  • I mean, if you go back a generation, um, yeah, Apple was doing their, their kind of closed thing.

  • Um, but Microsoft, which was, you know, it's, it obviously isn't like this perfectly open company, but you know, compared to, to, to Apple with Windows running on all the different OEMs and different software, uh, different, different hardware, um, was a much more open ecosystem.

  • And Windows, Windows was the leading ecosystem.

  • It, it, um, you know, it, it basically in the kind of PC generation of things, the open ecosystem won.

  • And I am kind of hopeful that in the next generation of computing, we're going to return to a zone where the open ecosystem wins and is the leading one.

  • Again, there will always be a closed one and an open one.

  • I think that there's reasons to do both.

  • There are benefits to both.

  • I'm not like a zealot on this.

  • I mean, we do closed source stuff.

  • I'm not everything that we, that we publish is open.

  • Um, but I think in general for the computing platforms that the whole industry is building on, there's a lot of value for that if the software especially is open.

  • So that's really shaped my philosophy on this.

  • And, um, for both AI with Llama and with the work that we're doing in AR and VR, where we are basically making the horizon OS that we're building for mixed reality, um, in, in open operating system in the sense of, of kind of what Android or what Windows was and, and basically making it so that, um, like we're going to be able to work with lots of different hardware companies to make all different kinds of, of devices.

  • We basically just want to return the ecosystem to that level where that that's going to be the open one.

  • And, and I, I, I'm pretty optimistic that in the next generation, the open ones are going to win.

  • For, for us specifically.

  • Um, you know, I just want to make sure that we have access to, I mean, this is sort of selfish, but I mean, it's, you know, after building this company for awhile, um, one of my things for the next 10 or 15 years is like,

  • I just want to make sure that we can build the fundamental technology that we're going to be building social experiences on, because there've just been too many things that I've tried to build and then have just been told, nah, you can't really build that by the platform provider that at some level,

  • I'm just like, nah, fuck that for the next generation.

  • Um, like we're going to go build like all, all the way down and, and make sure that, that there goes our broadcast opportunity.

  • I know.

  • Sorry.

  • Um, sorry.

  • Um, as a beep.

  • Yeah.

  • You know, uh, we're doing okay for like 20 minutes, but give me, give me talking about closed platforms and I get angry.

  • Um, so, um, Hey, look, it is great.

  • I think it's a great world where, where, uh, where there are people who are dedicated, uh, to build the best possible AIs, however they build it and they make, they, they offer it to the world, um, you know, as a service and then, but if you want to build your own AI, you could still also build your own AI.

  • So the ability to totally write, to use an AI, you know, there's a lot of stuff.

  • I prefer not to make this jacket myself.

  • I prefer to have this jacket made for me.

  • You know what I'm saying?

  • Yeah.

  • Yeah.

  • But so the fact that, so the fact that leather could be open source is not a useful concept for me.

  • But, but I, I think the, the idea that you could, you could have great services, incredible services, as well as open service, open ability, then, then we basically have the entire spectrum.

  • But the thing that's that, that, that you did with 3.1, that was really great.

  • Was you have four or five B you have 70 B you have eight B you could, you could use it for synthetic data generation, use the larger models to essentially teach the smaller models and although the larger models will be more general.

  • Um, it's less brittle.

  • Uh, you could, you could still build a smaller model that fits in, you know, whatever operating domain or operating costs that you would like to have, uh, you, you, uh, uh, met a guard, I think, uh, llama guard, uh, llama guard for guard railing.

  • Fantastic.

  • Um, and so now, and the way that you built the model, uh, it's built in a transparent way, it's, uh, has you dedicated, you've got a world-class safety team, world-class ethics team.

  • Uh, you could build it in such a way that everybody knows it's built properly.

  • And so I really love that part of it.

  • Yeah.

  • And I mean, just to finish the thought from, from before, uh, before I got,

  • I got sidetracked there for detour.

  • Um, you know, I do think there's this alignment where, and we're building it because we want the thing to exist and we want to knock it cut off from some closed model.

  • Right.

  • And, um, but it, this isn't just like a piece of software that you can build.

  • It's, you know, you need an ecosystem around it.

  • And so it's, it's almost like it, it kind of almost wouldn't even work that well if we didn't open source it, right?

  • It's, it's not, we're not doing this because we're kind of altruistic people.

  • Um, even though I, I think that this is going to be helpful for the ecosystem and we're doing it because we think that this is going to make the thing that we're building the best by, by kind of having a robust ecosystem around how many people contributed to PyTorch ecosystem.

  • Yeah, totally.

  • Mountains of engineering.

  • Yeah.

  • Yeah.

  • Yeah.

  • I mean, Nvidia alone, we probably have a couple of hundred people just dedicated to making PyTorch better and scalable and, you know, more performant and so on and so forth.

  • Yeah.

  • And it's, it's also just when something becomes something of an industry standard, other folks do work around it, right?

  • So like all of the Silicon and the systems will end up being optimized to run this thing really well, which will benefit everyone, but it will also work well with the system that we're building.

  • And that's, I think just one example of how this ends up being, um, just being really effective.

  • So, yeah.

  • I mean, I think that the open source strategy is going to be, it's going to be a good one as a business strategy.

  • I think people still don't quite, we love it so much.

  • We built an ecosystem around it.

  • We built this thing.

  • Yeah.

  • Yeah.

  • Yeah.

  • I mean, you guys have been awesome on this.

  • I mean, every time we're shipping something, you guys are the first to, to release this and optimize it and make it work.

  • And so, I mean, I, I appreciate that, but what can, what can I say?

  • We have good engineers and so, and, and, well, you always just jump on this stuff quickly too, so, you know, I'm a senior citizen, but I'm agile, you know, that's what CEOs have to do.

  • Um, and I recognize an important thing.

  • I recognize an important thing.

  • And, and I, I think the llama is genuinely important.

  • We built this concept to call an AI factory, uh, AI foundry around it, uh, so that we can help everybody build, take, you know, a lot of people, they, they, they have a desire to, um, uh, build AI and it's very important for them to own the AI because once they put that into their, their flywheel, their data flywheel, that's how their company's institutional knowledge is encoded and embedded into an AI.

  • So they can't afford to have the AI flywheel, the data flywheel, that experience flywheel somewhere else.

  • So, and so open source allows them to do that, but they, they don't really know how to turn this whole thing into an AI.

  • And so we created this thing called an AI foundry.

  • We provide the tooling, we provide the expertise, uh, llama, uh, technology.

  • Uh, we have the ability to help them, uh, turn this whole thing, uh, into an AI service and, and then when, when we're done with that, uh, they take it, they own it, we, the output of it's what we call a NIM and this NIM, this, this neural micro NVIDIA inference microservice, uh, they just download it.

  • They take it and they run it anywhere they like, including on-prem.

  • And we have a whole ecosystem of partners, uh, from OEMs that can run the NIMs to, uh, GSIs like Accenture that, that, uh, we've trained and work with to create llama based NIMs and, and, uh, and, uh, pipelines.

  • And, and now we're, we're off helping enterprises all over the world do this.

  • I mean, it's really quite an exciting thing.

  • It's really all triggered off of, uh, the llama open sourcing.

  • Yeah.

  • I think especially the ability to help people distill their own models from the big model is going to be a really valuable new thing because I, there's this, you know, just like we talked about on the product side, how, at least I don't think that there's gonna be like one major AI agent that everyone talks to, it's at the same level.

  • I don't think that there's going to necessarily be one model that everyone uses.

  • We have a chip AI, chip design AI.

  • We have a software coding AI and our software coding AI understands, uh, USD because we code in USD for, for omniverse stuff.

  • Um, uh, we have a software AI that understands Verilog, our Verilog.

  • Um, we have a, we have software AI that understands our bugs database and knows how to help us triage bugs and sends it to the right engineers.

  • And so each one of these AIs are fine tuned off of llama.

  • And, and so we fine tune them.

  • We guard rail them.

  • You know, if we, if we have a, if we have a, an AI design, uh, for, for, uh, for chip design, uh, we're not interested in asking it about politics, you know, and religion and things like that.

  • So we guard rail it.

  • And so, so I think, I think every company will essentially have for every single function that they have, uh, they will likely have AIs that are built for that and they need help to do that.

  • Yeah.

  • I mean, I think it's one of the big questions is going to be in the future to what extent are people just using the kind of the bigger, more sophisticated models versus just training their own models for the uses that they have.

  • And at least I would bet that they're going to be just a, just a vast proliferation of different models.

  • People, we use the largest ones.

  • And the reason for that is because our engineers are, their times are so valuable.

  • And so we get, uh, right now we're getting four or five B, uh, optimized for performance.

  • And as you know, uh, four or five B doesn't fit in any GPU, no matter how big.

  • And so that's why the MV link performance is so important.

  • We have this, every one of our GPUs connected by this non-blocking switch called MV link switch.

  • And, um, uh, in the HVAC, for example, there are two of those switches and we make it possible for all these, all these GPUs to work and, and, um, uh, run the four or five B's really performant.

  • The reason why we do it is because the, the engineer's times are so valuable to us.

  • You know, we want to use the best possible model.

  • The fact that it's cost effective by a few pennies, who cares?

  • And so we, we, we just want to make sure that the best quality of result is presented to them.

  • Yeah.

  • Well, I mean, the four or five, I think is about half the cost to inference of the GPT 4.0 model.

  • So I mean, at that level, it's already, I mean, it's, it's pretty good, but yeah.

  • I mean, I think people are doing stuff on devices or want smaller models.

  • They're just going to distill it down.

  • So that's like a whole different set of services.

  • That AI is running and let's, let's pretend for a second that we're hiring that AI, that AI for chip design is probably $10 an hour you're using.

  • You know, and, and, uh, um, if you're using it constantly and you're sharing that AI across a whole bunch of engineers, so each engineer probably has an AI that's sitting, sitting with them, that doesn't cost very much.

  • And we pay the engineers a lot of money.

  • And so, so to us, a few dollars an hour, uh, amplifies the capabilities of somebody that's really valuable.

  • Yeah.

  • Yeah.

  • I mean, you don't need to convince me.

  • If you haven't, if you haven't, if you haven't hired an AI, do it right away.

  • That's all we're saying.

  • And so, so, um, I, I let's, let's talk about, let's talk about, um, the next, the next wave, um, you know, one of the things that I really love about the work that you guys do, computer vision.

  • Um, uh, one of the models that we use a lot internally, uh, is segment everything.

  • And, um, uh, you know, that, that we're now training AI models on video so that we can understand the world model.

  • Now our use case, our use cases for robotics and, and, uh, industrial, industrial, uh, digitalization and, um, uh, connecting these AI models into omniverse so that we can, we can, um, uh, model and represent the physical world.

  • Better.

  • I have robots that operate in the physical world.

  • Your, your application, uh, uh, the, the Ray-Ban metaglass, um, uh, your vision for, for bringing AI into the virtual world, uh, is really interesting.

  • Tell us about that.

  • Yeah.

  • Well, okay.

  • A lot to unpack in there.

  • Um, the segment, anything model that you're talking about, we're actually presenting, I think the next version of that here at SIGGRAPH segment, anything to, um, to, to, to, to, to, to, to, to, to, to, to, to, to, to, to, to, to, to, to copy and paste.

  • Um, and then we will actually release the full video, which this is coming in at a later date, so have a chance to watch them hit the link to the SIGGRAPHs, uh, links, um, I'll get to the links from there.

  • Um, so I'm about to post, um, a copy, which I was just showing you guys on my Twitter prevision.

  • Alright, let's go there.

  • Yeah, next time we do.

  • So Mark, Mark came up to my house and we made Philly cheesesteak together.

  • Next time you're bringing the towel, I was more of a sous chef, but it was really good.

  • It was really good.

  • That's sous chef comment.

  • Okay.

  • Listen, at the end of the night though, you were like, Hey, so you, you, you ate enough, right?

  • And I was like, I don't know.

  • I could eat another one.

  • You're like, really?

  • You know, usually when you say something like you're being like, yeah, it was definitely like, yeah, we're making more.

  • We're making more.

  • She's, did you get enough to eat?

  • Usually your guest says, oh yeah, I'm fine.

  • Make me another cheesesteak, Jensen.

  • So just to let you know how OCD he is.

  • So I turn around, I'm, I'm prepping the, the, the cheesesteak and I said, Mark, cut the tomatoes.

  • And so, so, uh, Mark, uh, I handed him a knife.

  • I'm a precision cutter.

  • And so he cuts, he cuts the, uh, the tomatoes.

  • Every single one of them are perfectly to the exact millimeter.

  • But the really interesting thing is I was expecting all the tomatoes to be sliced and kind of stacked up kind of like a deck of cards.

  • And, uh, but when I turned around, he said he needed another plate.

  • And the reason for that was because all of the tomatoes he cut, none of them touched each other.

  • Once he separates one slice of tomato from the other tomato, they shall not touch again.

  • Look, man, if you wanted them to touch, you needed to tell me that, right?

  • I'm just a sous chef.

  • Okay.

  • That's why he needs an AI that doesn't judge.

  • Yeah.

  • It's like, this is super cool.

  • Okay.

  • So it's recognizing the cows track.

  • It's recognizing tracking the cows.

  • Yeah.

  • Yeah.

  • So it's, um, a lot of fun effects will be able to be made with this and because it'll be open a lot of more serious applications across the industry too.

  • So, I mean, scientists use this stuff to, you know, study, um, like coral reefs and natural habitats and, um, and kind of evolution of landscapes and things like that, but I mean, it's, uh, being able to do this in video and having it be a zero shot and be able to kind of interact with it and tell it what you want to track is, um, it's, it's, uh, it's pretty cool research.

  • So for example, the reason why we use it, uh, for example, you have a warehouse and there's got a whole bunch of cameras and the warehouse, uh, AI, uh, is watching everything that's going on.

  • And let's say, uh, uh, you know, a stack of boxes fell, uh, or somebody spilled water on the ground.

  • Um, or, you know, whatever accident is about to happen, the AI recognizes it generates the text, send it to somebody and, you know, uh, you know, help will come along the way.

  • And so that's one way of using it.

  • Uh, instead of recording everything, if there's an accident, instead of recording every nanosecond of video and then going back and retrieve that moment, it just, it just records the important stuff because it knows what it's looking at.

  • And so, so having a video understanding model, a video language model is really, really powerful for all, all these, these interesting applications.

  • Now, what else, what else are you guys going to work on beyond, uh, Ray talk, talk to me about, yes, there's all the smart glasses, right?

  • So I think when we think about the next computing platform, you know, we kind of break it down into mixed reality, the headsets and the smart glasses and the smart glasses.

  • I think it's easier for people to wrap their head around that and wearing it.

  • Cause it's, you know, pretty much everyone who's wearing a pair of glasses today will end up, that'll get upgraded to smart glasses.

  • And that's like more than a billion people in the world.

  • So that's going to be a pretty big thing.

  • Um, the VRM are headsets.

  • I think some people find it interesting for gaming or different uses.

  • Some don't yet.

  • My view is that they're going to be both in the world.

  • I think the smart glasses are going to be sort of the mobile phone kind of always on version of the next computing platform.

  • And the mixed reality headsets are going to be more like your workstation or your game console, where when you're sitting down for a more immersive session and you want access to more compute, I mean, look, I mean, the glasses are just very small form factor.

  • Um, there are going to be a lot of constraints on that.

  • Just like you can't do the same level of computing on a phone.

  • It came at exactly the time when all of these breakthroughs in generative AI happened.

  • Yeah.

  • So we, we basically for smart glasses, we've been, we've been going at the problem from two different directions.

  • On the one hand, we've been building what we think is sort of the technology that you need for the kind of ideal holographic AR glasses.

  • And we're doing all the custom Silicon work, all the custom display stack work, like all the stuff that you would need to do to make that work in their glasses.

  • Right.

  • It's not a headset.

  • It's not like a VR or MR headset.

  • They look like glasses, but, um, there's still quite a bit far off from the glasses that you're wearing now.

  • I mean, those are very thin, but, um, but even, even the Ray bands that we, that we make, you couldn't quite fit all the tech that you need to into that yet for kind of full holographic AR, though we're getting close.

  • And over the next few years, I think we'll, we'll basically get closer.

  • It'll still be pretty expensive, but, but I think that'll start to be a product.

  • Um, the other angle that we've come at this is let's start with good looking glasses by partnering with the best glasses maker in the world, Essler Luxottica.

  • They basically make, they have all, all the big brands that you use.

  • Um, you know, it's Ray-Ban or Oakley or Oliver Peoples or just like a handful of others, it's kind of all Essler Luxottica.

  • The NVIDIA glasses.

  • Um, I think that, you know, it's, um, I think they would probably like that analogy, but, um, I mean, who wouldn't, who wouldn't at this point?

  • Um, but, uh, so we've been working with them on, on the Ray-Bans.

  • We're on the second generation and the goal there has been, okay, let's constrain the form factor to just something that looks great idea.

  • And within that, let's put in as much technology as we can, understanding that we're not going to get to the kind of ideal of what we want to fit into it technically, but it'll, it'll, but at the end it'll be like great looking glasses.

  • And we, at this point we have, we have camera sensors, so you can, you can take photos and videos.

  • You can actually live stream to Instagram.

  • You can take video calls on WhatsApp and stream to the other person, um, you know, what you're seeing.

  • Um, you can, I mean, it has, it has a microphone and speaker.

  • So, I mean, the speaker is actually really good.

  • It's like, it's open ear.

  • So a lot of people find it more comfortable than, than earbuds.

  • Um, you can listen to music and it's just like this private experience.

  • That's pretty neat.

  • People love that.

  • You take phone calls on it.

  • Um, but then it just turned out that that sensor package was exactly what you needed to be able to talk to AI too.

  • So that was sort of an accident.

  • If you'd asked me five years ago, were we going to get holographic AR before AI?

  • I would have said, yeah, probably.

  • Right.

  • I mean, it's, it just seems like kind of the graphics progression and the display progression on all the virtual and mixed reality stuff and building up the new display stack, we were just making continual progress towards that.

  • And then this breakthrough happened with LLMs.

  • And it turned out that we have sort of really high quality AI now and getting better at a really fast rate before you have holographic AR.

  • So it's sort of this inversion that, that I didn't really expect.

  • I mean, we're, we're fortunately well-positioned because we were working on all these different products, but I think what you're going to end up with is, um, just a whole series of different potential glasses products at different price points with different levels of technology in them.

  • So I kind of think, um, based on what we're seeing now with the Ray-Ban Metas,

  • I would guess that display-less AI glasses at like a $300 price point are going to be a really big product that like tens of millions of people or hundreds of millions of people eventually are going to have.

  • Um, and then you're going to have super interactive AI that you're talking to.

  • Yeah.

  • Visual.

  • You have visual language understanding that you just showed.

  • You have real-time translation.

  • You could talk to me in one language.

  • I hear it in another language.

  • And then the display is obviously going to be great too, but it's going to add a little bit of weight to the glasses and it's going to make them more expensive.

  • So I think for, there will be a lot of people who want the kind of full holographic display, but there are also going to be a lot of people for whom, um, you know, they, they want something that eventually is going to be like really thin glasses.

  • Well, for industrial applications and for some work applications, we need that.

  • We need that.

  • I think for consumer stuff too.

  • You think so?

  • Yeah.

  • I mean, I, I think, you know, it's, I was thinking about this a lot during the, you know, during COVID when, when everyone kind of went remote for a bit, it's like you're spending all this time on Zoom.

  • It's like, okay, this is like, it's great that we have this, but, um, but in the future, we're like not that many years away from being able to have a virtual meeting where like, you know, it's like, I'm not here physically.

  • It's just my hologram.

  • And like, it just feels like we're there and we're physically present.

  • We can work on something and collaborate on something together.

  • But I think this is going to be especially important with AI.

  • I could live with, with a device that, that I'm not wearing all the time.

  • Oh yeah.

  • But I think we're going to get to the point where it actually is.

  • It'll be, I mean, there's within glasses, there's like thinner frames and there's thicker frames and there's like all these styles.

  • But, um, so I don't, I think we're, we're a while away from having full holographic glasses in the form factor of your glasses, but I think having it in a pair of stylish kind of chunkier frame glasses is not that far off.

  • These sunglasses are the face size these days.

  • I could see that.

  • Yeah.

  • And, and you know what, that's, um, that's a very helpful style.

  • True.

  • Yeah.

  • Exactly.

  • That's a very helpful, you know, it's like, like I'm trying to make my way into becoming like a style influencer so I can like influence this before, um, you know, before the glasses come to the market, but, you know, I don't know.

  • Attempting it.

  • How's your style influencing working out for you?

  • You know, it's early.

  • Yeah.

  • It's early, it's early.

  • Um, but I don't know.

  • I feel like if, if, if a big part of the future of the business is going to be building, um, kind of stylish glasses that people wear, um, this is something

  • I should probably start paying a little more attention to, right?

  • So yeah, we're going to have to retire the version of me that wore the same thing every day, but I mean, that's the thing about glasses too.

  • I think it's, um, you know, it's unlike, you know, even the watch or, or phones.

  • Like people really do not want to all look the same.

  • Right.

  • And, and it's like, so I do think that it's, you know, it's, it's a, it's a platform that I think is going to lend itself going back to the theme that we talked about before towards being an open ecosystem, because I think the diversity of form factors that people in styles that people are going to demand is going to be immense.

  • Yeah.

  • Um, it's not like everyone is not going to want to put like the one kind of pair of glasses that, you know, whoever else designs, like, that's not, I don't think that's going to fly for this.

  • Yeah.

  • I think that's right.

  • Well, Mark, it's sort of incredible that we're living through a time where, where the entire computing stack is re being reinvented, how we think about software.

  • You know, what, what Andre calls software one and software two.

  • And now we're basically in software three.

  • Now, the way we compute, um, from general purpose computing to these generative neural network processing way of doing computing, um, the capabilities, the applications we could develop now are unthinkable in the past and, and this technology generative AI, uh, I don't remember another technology that, that in such a fast rate influenced consumers, enterprise industries, and science.

  • Yeah.

  • And to be able to, to cut across, cut across, um, all these different fields of science from, from climate tech to biotech, um, uh, to, uh, physical sciences, uh, in every single field that we're encountered, uh, generative AI is, is right in the middle of that, uh, fundamental transition.

  • And, and it's, and, and in addition to that, uh, the things that you're talking about, generative AI is going to make a profound impact in society, you know, the products that we're making.

  • And one of the things that I'm super excited about, and somebody asked me earlier, is there going to be a, you know, Jensen AI, um, uh, well, that's exactly the creative AI you were talking about, you know, where we just build our own AIs and I, I loaded up with all of the things that I've written and, and I,

  • I fine tune it with, with, uh, uh, with the way I answer questions and, and, uh, and hopefully, hopefully, uh, over time, the accumulation of use and, you know, it becomes a really, really great assistant and companion, uh, for, for, uh, uh, for a whole lot of people who just wants to, you know, ask questions or, um, bounce ideas off of.

  • And, and it'll be the version of Jensen that, uh, as, as you were saying earlier, that's, that's not judgmental.

  • You're not afraid of being judged.

  • And so you could come and interact with it all the time.

  • But, but I just think, I think that those, those are really incredible things.

  • And, and you know, we, we write, we write a lot of things all the time.

  • And, and how incredible is it just to give it, you know, three or four topics.

  • Now, these are the basic themes of what I want to write about and write it in my voice and just use that as a starting point.

  • And so there's, there's just so many things that we can do now.

  • Uh, it is really terrific working with you.

  • And, uh, uh, I know that, I know that, uh, uh, it's not easy building a company and you pivoted yours from desktop to mobile, to VR, to AI, all these devices.

  • Uh, it's really, really, really extraordinary to watch.

  • And NVIDIA has pivoted many times ourselves.

  • I know exactly how hard it is doing that.

  • And, uh, uh, you know, both of us have, have gotten kicked in our teeth a lot, plenty over the years.

  • But that's, that's what it takes to, to, uh, uh, to want to be a pioneer and, and innovate.

  • And so it's really great watching you.

  • Well, and likewise, I mean, it's like, it's not sure if it's a pivot, if you keep doing the thing you were doing before, but, but as well, but it's, but you add to it.

  • I mean, there's more chapters to all the, to, to all of this.

  • And I think the same thing for, it's been fun watching.

  • I mean, the journey that you guys have been on, I mean, just, and you went, we went through this period where everyone was like, nah, everything is going to kind of move to these devices and, you know, just going to get super kind of cheap compute and, and you guys just kept on plugging away at this and it's like, no, like actually you're going to want these big systems that can, that can paralyze went the other way.

  • Yeah, no.

  • And it's, I mean, yeah, we went and instead of building smaller and smaller devices, we made computers as fashionable for a while, a little unfashionable, super unfashionable, but now, now it's cool.

  • And, and instead of, and you know, we, we started building a graphics chip, a

  • GPU, and, and now when you, when, uh, when you're deploying a GPU, you still call it Hopper H100, but so you guys know when, when, when Zuck calls it H100, his data center of H100s, there's like, I think you're coming up on 600,000.

  • And, and there, we're good customers.

  • That's how you get the Jensen Q&A at SIGGRAPH.

  • Wow.

  • Hang on to that.

  • I was getting the Mark Zuckerberg Q&A.

  • You were my guest and I wanted to make sure that that's called one day and you're like, Hey, you know, in like a couple of weeks, we're doing this thing at SIGGRAPH.

  • I'm like, yeah, I don't think I'm doing anything that day out of Denver.

  • It sounds fun.

  • Exactly.

  • I'm not doing anything that afternoon.

  • You just showed up and, and, uh, but, but the thing, the thing is just incredible.

  • These, these systems that you guys build, uh, they're, they're giant systems, incredibly hard to orchestrate, incredibly hard to run.

  • And, you know, you said that, that, uh, you got into the GPU, uh, journey later than, than most, uh, but you're operating larger than just about anybody.

  • And it's, it's incredible to watch and congratulations on everything that you've done.

  • And, uh, you, you are quite the style icon now.

  • Check, check out this guy.

  • Early stage working on it.

  • It's, uh, ladies and gentlemen, Mark Zuckerberg.

  • Thank you.

  • Hang on, hang on.

  • Well, um, you, you know, you know, um, so it turns out the last time that we got together, uh, after dinner, Mark, Mark and I, uh, Jersey swap, Jersey swap.

  • And, and, uh, we took a picture and, and, and turned it, it turned into something viral and, um, and now I thought that he, he has no trouble wearing my jacket.

  • I don't know.

  • Is that my look?

  • I don't think I should be.

  • Should be.

  • Is that right?

  • Yeah.

  • I actually, I, um, I made one for you.

  • You did?

  • Yeah.

  • That one's Mark's.

  • I mean, Hey, let's see.

  • We got, we got a box back here.

  • It's black and leather and shearling.

  • Oh, I didn't make this.

  • I just ordered it online.

  • It's a little chilly in here.

  • I think this is my goodness.

  • I mean, it's a vibe.

  • You just need, is this me?

  • I mean, get this guy a chain next time I see him bringing a gold chain.

  • So fair is fair.

  • So I let you know, I was telling everybody that Lori bought me a new jacket to celebrate this year's SIGGRAPH.

  • SIGGRAPH is a big thing in our company.

  • As you could imagine, RTX was launched here.

  • Amazing things were launched here.

  • And this is a brand new jacket.

  • It's literally two hours old.

  • Wow.

  • And so I think we had a Jersey swap again.

  • All right.

  • Well, one's yours.

  • I mean, this is worth more because it's used.

  • Let's see.

  • I don't know.

  • I think, I think Mark is pretty buff.

  • He's a little, the guy's pretty jacked.

  • I mean, you too, man.

  • All right.

  • All right, everybody.

  • Thank you.

  • John and Mark Zuckerberg.

  • Have a great SIGGRAPH!

  • SIGGRAPH.

Ladies and gentlemen, I have a very special guest, but could I ask everybody to sit down?

Subtitles and vocabulary

Click the word to look it up Click the word to find further inforamtion about it