Introducing your copilot for the web: AI-powered Bing and Microsoft Edge

Subtitles section Play video

[MUSIC]
SATYA NADELLA: Hi, everyone.
Welcome. It's great
to see all of you in Seattle in person.
We have an unbelievable show.
I see Scott Guthrie even wore his red shirt.
We welcome you to
the Azure Kubernetes Container Service 2023 launch.
No, don't worry. We'll have some fun.
Scott is not coming up to show you
code onscreen anytime soon,
but look, it's an exciting time in tech.
The broad contours of
this next platform are
just getting clearer and clearer each day.
The advances, what's possible.
That's what obviously excites us in our industry.
But they're also grounded
in what's happening in the broader world.
There's no question. There are
enormous challenges out there.
In fact, it reminds me and I've spoken about
this before of the very founding of Microsoft in 1975.
In fact, when the popular electronics cover
came out with the Altair,
which of course our founders
picked up and ran with it and
created essentially what's the software industry
as we know of it today.
That same week,
Newsweek had a cover with President Carter
trying to fight off
the three-headed monster of inflation,
recession, and an energy crisis.
Today you'd have something similar.
You will have AI on one cover and
then we'll have those three challenges
plus for good measure,
we can add a few more.
We, as Microsoft,
we as the tech industry have to really
ground ourselves in how do we relate one to the other?
In other words, can we use technology to overcome
the challenges that people and
organizations and countries face?
That's really the pursuit here.
In that context, I would say,
I just want to share a couple of
anecdotes which gives me great hope.
Quite honestly, it gives me personally a lot of
satisfaction around working at Microsoft,
working in this industry to
push the state of the art of technology.
The first one, obviously when Sam and
his team late last year launched ChatGPT,
that's the only thing anybody your friends and
family wanted to talk about throughout the holidays.
It was just crazy house.
It was like the Mosaic moment.
The closest we've come.
It's been 30 years now since when Mosaic launch,
which I distinctly remember.
It was very exciting time.
I went on a holiday first and then I was in
the first week of January I was in India.
On Jan 1st,
I look at my news feed and I see
this tweet that Andrej Karpathy put out.
Who is our ex OpenAI and Tesla,
and is now an independent AI developer.
He had this thing about the product that really he
was most excited about the previous year was
GitHub Copilot and he was saying how 80 percent of
his code was being generated by this,
I would say first at
scale product built on NLM technology.
This doesn't mean he's
80 percent somehow not doing his work.
In fact, he's getting so much more leverage.
In fact, recently we crossed
100 million developers on GitHub.
Think about this. There are
100 million developers on GitHub.
If we can improve their productivity,
just like how Andrej was able to observe.
Then let's say in the next decade we double that number,
may be double it again.
We get close to a half a billion developers,
what economic opportunity it would create.
Because there is not a meeting that I
go to today with any CEO,
CXO of any organization who's not
looking for more software developers,
more digital skills.
That's the currency in every sector of the economy,
in every country of the world.
That's the opportunity we
have to be able to take
this technology and make a difference.
But then the next day I went
to Mumbai and then I saw this demo.
This was just for me,
the most profound thing that I've seen in a long time.
The demo was actually built by
the Ministry of Electronics in
India because they are building
out a digital public goods.
Their idea is look, India has got
multiple official languages and they wanted to
democratize essentially access to language translation.
They're building it out as a digital public good.
In fact, Microsoft and Azure and
Microsoft Research are all involved in that project.
This is basically speech-to-text,
text-to-speech across all of the languages in India.
They showed me this demo where
a farmer speaking in
Hindi expresses a pretty complex thought
about how he had heard
about some government program and wants to
apply for a subsidy that he thinks he's eligible for.
It's a pretty complex prompt query.
There's technology,
that's a good job.
It goes to the bot, recognizes the speech,
comes back and says, you know what?
You should go to this portal,
fill out these forms,
then you'll get your subsidy.
He says, look, I'm not going to go into any portal.
I'm not going to fill out any forms,
can you help me? He does it.
Then I was told that a developer said, you know what?
That is Daisy Ching a model that was
trained on all of the documents of
the government of India using
GPT with this speech recognition software.
Basically two models coming together to really
help a rural farmer in India
trying to get access to a government program.
Look, I grew up in India.
I dreamt every day that someday
the industrial revolution will get evenly
distributed across the world.
Here I was,
seeing something so profound,
something that is developed by the folks at
OpenAI in the West Coast of
the United States a few months earlier,
used by a developer locally
to have an impact on a rural farmer.
That to me is what gives
me meaning and I think gives
us all meaning in our industry.
It is just fantastic to see that.
Now of course we've got to scale it and scale
it with a real understanding that we can break things.
It's about being also clear-eyed about
the unintended consequences of any new technology.
In fact, that's why way back in
2016 is when we came out with the AI principles.
You have both we as Microsoft and our partners at OpenAI,
deeply care about this.
In fact, the entire genesis of
OpenAI is from that foundation.
We built these principles,
but we've not just put those principles as a document,
but we've been practicing it because that's the only way
technology gets better in this particular case.
When you're talking about AI, it's about alignment with
human preferences and societal norms
and you're not going to do that in a lab.
You have to do that out there in the world.
It starts by the way with design decision ones makes.
When you think about AI, you
can have the human in the loop,
you can hear how the human on
the loop or you can have the human out of the loop.
Those are decisions we as product makers build,
build into the products.
Whenever we have come up with
some new things and new models,
we in fact put a premium on human agency.
When you think about these generative AI models,
remember one thing, they are prompted to do things.
The prompting comes from a human being.
We get to in fact help them prompt.
We get them to take the draft that gets generated.
They should review the draft.
They get to review the draft,
they get to approve the draft using judgment.
We want to give them as many perspectives,
as many ways to regenerate.
We want to take even just the design of AI products as
a first-class construct and build that into our products.
But that's not sufficient, we realize that.
You have to write at
the pre-training stage when it comes to the data,
the pre-training itself, the model itself has to be safe.
The safety system around the model,
the application context, all will matter.
We absolutely, and you'll see that today,
we take all of that as
first-class things which we want to reduce,
not just to principles but to engineering practice.
Such that we can build AI
that's more aligned with human values,
more aligned with what our preferences are,
both individually and as a society.
What is it that we should do and what should we build?
I think that this technology is going to reshape
pretty much every software category.
We know, we've seen that and we've solved.
If you think about the web, we've had what?
Three at least very distinct platform shifts
that have shaped the web.
The web was born on the PC and
the server and then it evolved with mobile and Cloud.
Now the question is,
how is AI going to reshape the web?
Each time in each one of those phases,
some real foundational technology layers,
sometimes I describe them as
these organizational layers of technology, were born.
The browser was the first one.
Without the browser, the web would have not
been as popular as it is today.
Same thing with search.
Search organized the web.
Then when it came to the mobile
generation and mobile and Cloud,
in fact these super apps,
especially outside of the United States around messaging,
became the way people consume the web.
We also had app stores.
The question is, what are the constructs?
What are the organizing layers going forward?
You'll see some of that today.
We think there are two things that are emerging.
One is this conversational intelligent agents.
I think they're going to be things that
we're going to have everywhere we go.
All computer interaction is
going to be mediated with an agent helping you.
In fact, we're going to have this notion of a
co-pilot that's going to be there
across every application canvas inside of
an operating system shell in a browser.
We want to show you some of this innovation starting with
how it's going to reshape
the largest software category on planet Earth,
which I've been working on for a long time,
which we're very, very excited about, Search.
It's a new day in Search.
It's a new paradigm for search.
Rapid innovation is going to come.
In fact, a race starts
today in terms of
what you can expect and we're going to move.
We're going to move fast.
For us every day,
we want to bring out new things.
Most importantly, we want to have
a lot of fun innovating
again in Search because it's high time.
With that, let me turn it over to Yusuf.
YUSUF MEHDI: Thanks, Satya. It's great
to be here with all of you today.
We've been working on something we think is pretty
special and we're just eager to share it with you.
In my time at the company,
I've been lucky to witness a few important moments.
I think this might be another one of those.
We believe as Satya said that we can improve
the way billions of people can benefit from the Internet.
Created by really an amazing team across Microsoft,
from the core web engineering folks in Michael's group,
to the brilliant folks at Microsoft Research,
to the tireless people that work on
the Azure AI supercomputer,
we think of it humbly as
the next-generation of search and browsing.
Infused with AI and
assembled as an integrated experience,
we're going to re-imagine the search engine,
the web browser, and new chat experiences
into something we think of as your copilot for the web.
Now, copilot is a critical word because we
believe in the empowering nature of AI in which you,
the individual, are in charge.
Now what it is, is not even as
important as what it represents.
That is for us an aspiration
to unlock the joy of discovery,
the wonder of creation,
and that feeling of empowerment from
being able to harness the world's knowledge.
At the center of this new co-pilot experience is
an all new Bing search engine and Edge web browser,
and it's going to do four things for you.
First, it's a better search.
It's the search you know and love but
it's better because it's AI powered.
Second, not only does it give you the search results,
but it will actually answer your questions.
Third, we're going to make it incredibly easy to use.
We're going to let you chat, we're going to
let you just talk to it naturally.
Last, when you need that spark of creativity,
Bing can generate content for
you automatically to help you get started.
Let's talk a little bit about the opportunity
in search and why
we believe we're at the start of the next generation.
No. Sorry about that, we're on to the next generation.
All seriousness, with
over 10 billion queries a day,
we all know the search engine is an incredible tool.
We do. Yet as the web has grown,
we've run into challenges.
People are overwhelmed increasingly with
too many links when they're
trying to find simple answers.
But 40 percent of the time people click on search links,
they click back immediately.
That's a sign they're not finding what they want.
Most notably, we have to adapt to
search versus the other way around.
It turns out search works better
if you give it fewer keywords.
You might be surprised to know that 75 percent of
all searches are three key words or less.
But come on, we shouldn't be surprised.
Search has remained fundamentally
the same since the last major inflection.
When we moved from a directory approach to
search to an algorithmic approach to search.
The user experience and the approach
underlying are essentially the same as 20 years ago.
Few keywords and you get millions of links.
But as the world around us has changed,
the way people use search has changed, we all know this.
People are trying to use search to do
more than it was designed to do.
To illustrate that let's just look at a breakdown
of search queries by type to understand this point.
Search querries today you can bucket them
into serve three types of categories.
The first are what we call navigational.
This is people searching for a website.
When you need to renew your driver's license,
you need to go to the DMV.
When you want to tell your friend
about that great pizza place,
you send them the WebLink.
That's what search was designed to do,
and it does it super well.
The second, third of categories
are what we call informational.
These are things like what's the weather forecast?
What was last night sports scores?
What's today's stock price?
Search works great for that.
But for anything more complex, like hey,
I got to pick up this love seat at the store,
is it going to fit in the back of my car?
Search is going to fall short on that.
Then the final third,
we're back into what we call everything else.
These are typically more deep research in nature.
This is things like trip planning or complex shopping.
An example query here that people are putting in is, hey,
can you recommend a five-day itinerary to Mexico City?
For these, search falls far short of the desired goal.
What this means if you just do the simple math,
it means that roughly half of
all searches aren't delivering the job that people want.
If you go on that 10 million queries,
it means that every second,
50,000 people searches potentially go unanswered.
This is why we believe
it's time for a new approach in search.
For the billions of queries that are going unanswered,
we've seen new attempts to try address the problem.
As you all know, there are vertical search attempts.
Amazon has done a better job for shopping.
YouTube is great for video.
Reddit is a great place to come get advice.
The benefits of search are well-known.
It's fast, it's timely,
and there's a great business model.
Then more recently they've been another vector,
more disruptive ideas like leveraging AI to
answer questions directly and to generate content.
These are amazing as well,
and they show what's possible.
But what if you could get the two to come together?
Not only would you get two things in one,
but we think you could actually solve problems in each.
We think you could get to something that is
really one plus one equals three.
We've done that with the new Bing.
I want to share with you
four technical breakthroughs the Team
has achieved to make this come to life.
First, through our fantastic partnership
with SAM and the brilliant team at OpenAI,
I'm excited to announce that Bing is running on
a new next-generation large language model.
One that is much more powerful than ChatGPT,
and one that is customized specifically for search.
It's unlike anything you've had a chance to play
with and we can't wait for you to try it.
Second, we've developed a proprietary way of working
with OpenAI that allows us to best leverage the power.
We call this collection of capabilities and
techniques the Prometheus model.
The core idea here is that both
at training and at runtime,
we engage with the OpenAI model more
intelligently through our knowledge of the web,
via the Bing index and some special query techniques.
We're going to dig into this a little bit more later,
but the benefits are the following.
First, we can improve
the relevancy of answers by feeding in
and better tuning queries given
our understanding of the web search index.
Next, we can annotate
the answers with specific web links and citations.
We can get you more up-to-date
information because search crawls the web every day.
We can improve understanding of geolocation.
Finally, we can increase
the safety of the answers as well by catching
queries at initiation and then
checking that again at the delivery of an answer.
Next, we've been making
steady improvements on the Bing algorithms for years.
We test these with independent judges.
The shows that are search experience is on par or
better than any search experience
when you take away the brands.
But a few weeks ago, something special happened.
We applied the AI model to
our core search ranking engine and we
saw the largest jump in relevance in two decades.
We believe we can continue to drive
breakthroughs as we improve the models.
Then finally, we are re-imagining how you
interact with all of these capabilities across search,
browser, and chat by
pulling them into a unified experience.
I want you to think about search
coming together with answers,
search coming together with chat,
and search coming together with the browser.
As we all know and as the folks at OpenAI taught us,
the user experience is as important as
the underlying technical platform. Enough talk.
You guys ready to see it in action?
Let's show you how we're going to enable
the Copilot for the web.
Now before I do, I want to call
out two things for clarity.
First, because there's so much I'm going to show you
until you don't have to watch me type every search.
I recorded these searches live just yesterday.
Second, in case there are skeptics in the room,
you're going to get a chance to play with it directly,
put your hands on it and type the same queries as well
as your own favorites right after this presentation.
You're all familiar with search,
so I'm not going to show you a search.
I'm going to share what you can't do with today's search.
I'm going to focus on answers,
chat, and the ability to create.
But first let me introduce you to the new Bing homepage.
You're going to notice some subtle but important changes.
First, we have an expanded search box
capable of accepting up to 1,000 characters.
Because now Bing works with natural language.
You saw a little hint to chat,
which I'm going to get back to in a second.
Now want to set up the first search scenario.
My daughter and I, we both love art.
She studied art at school and I'd like to stay
connected with her on our mutual passion.
Last semester, she was learning about Mexican painters.
I'd like to get a quick summary of
the most influential Mexican painters
and the works to learn a bit more about the topic.
If I type the full query of what I'd like to
know in today's search, here's what I get.
I'll just type in, compare
the most influential Mexican artists
and their top paintings,
and you'll get what you expect. In some links.
It's fine, but we can do better.
Let's try this now in the new Bing.
What you'll see as we pull up, is first,
you see the web results here on the left,
but then on the right, you start to see
how we start to compile the answer.
What you get here now is we have
the ability to highlight these web links.
We can annotate the results.
That's because we're able to go in and
apply our index onto the answers there.
In other words, the answers and
the search on one page has
saved me a huge amount of time.
This gives you a little bit of a
sense of what you can do.
Now you've seen some of this before,
you might say, Hey, I've seen some of these.
Let's show you how we can do some additional things.
I'm going to show you another query.
Where we use the timeliness of search.
Let's go ahead and ask about events in
Scottsdale during the Super Bowl.
What you'll see is we get back and answer here,
where we have events and we're
able to do that because Bing cross web.
Notice how we can find not only
that the Super Bowl was played in Glendale on the 12th,
but then there are events like
Cardi B Super Bowl party that's
on the 10th also shows up.
We were able to pull these things together.
You start to get a sense of how we can build on what's
today with the Bing index.
Now I'm going to show you a few
more of these types of answers quickly,
so you can get a sense of the power,
and the time savings from Bing.
When I'm running an errand,
like the example I gave to you earlier,
I can ask Bing to determine if
that new loveseat from IKEA is
going to fit in the back of my Honda Odyssey.
What you'll see is they can
actually find the dimensions of the loveseat,
the interior space of the car,
and then make an estimation as to whether it will fit.
In this case, I'm choosing
an example of where Bing does not know the answer.
We know that we can't be definitive about it.
The reason I'm doing that is because we know we
won't be able to answer every question, every time.
But Bing can still provide
some helpful information as you can see on this answer.
We also know we'll make our share of mistakes,
and Bing rule this out.
We've added a quick feedback button
at the top of every search,
so that you can give us feedback and we can learn.
Another example, when I'm shopping,
I can ask Bing to search, find,
and compare the top three selling pet vacuums
listing the pros and cons.
Look how great this answer is.
It has all three of the products I'm looking for,
super-helpful pros and cons. Stop and think.
If you had to compile that,
how much time that would take you to do?
As you can see at the top of the page,
we still have the advertising in this example,
because we know when people are
shopping, those ads are helpful.
Finally on this one, if I'm cooking,
and I realize I've forgotten
a key ingredient, in this case, e.g.
eggs for my cake recipe,
Bing can not only find the egg substitutes,
it can get me the exact amount for each ingredient.
Take a look at this.
I love this. You can actually see e.g.
if you go with vinegar and baking soda,
the cake is lighter and more fluffy.
These are just little helpful tips that
everyday help make your life a little better.
These are just some examples.
Then you can start to get a sense of how, with answers.
We go far beyond what you can do with search day.
We can actually help you get what you want to get done.
Now let me tell you about how
Bing goes further to help you with
particularly complex questions for
which there is not a precise answer.
I wanted to introduce to you
the new chat experience in Bing.
I think of this as search with your own personal helper,
to help you refine your query until
you get exactly what you're looking for.
This comes in handy for activities
like trip planning and shopping research.
Let's start with shopping. I'm going
to look for 65 inch TV.
Again, you see ads at the top,
the result of the links on the left,
and the answer is here on the right.
You can pick whichever you'd like.
We'll give you a good set of answers.
But now I want to refine this query,
so I can do that by going into chat.
Now I can either swipe up with
my fingers or look up here at the top of the screen.
We have now a new chat scope.
With that, with one click,
you are now into chat.
Look how beautiful that is.
Searched to check, just so seamless.
Now we take away all of
the content that was in your place,
and you focus on your query.
The search box you can see,
now that can take up to 2.000 characters.
You can just talk to it. You can just ask for it.
In this case, let's say I'm going to
ask for gaming optimized TV.
All I have to say is which of these are best for gaming?
We remember all of the context.
We know that we're talking about flat screens,
we know we're talking about 65-inch TV's.
Look how Bing start to come back.
It does all the queries on
my behalf and comes back with a great answer.
I just want to highlight a few things for you.
Since we know you're asking about gaming TV's,
we pull out, this one has
a game optimizer, this has Game Mode.
We make that really helpful. I'm on a budget.
I'll ask it to adjust it for,
which one of these is the cheapest.
Again, Bing knows the context,
and it just goes in and refines the queries.
Easy, you just talk to it,
and you can refine your shopping experience.
Again, we find the prices here.
I didn't know you could get a flat-screen for under $500,
but that's a good deal there on
Bing if you're looking for TV.
We think that's going to make shopping
easier. Let's talk about travel.
Before I jump in, I want to
just have you remind yourself,
when you're going to plan a trip to a foreign country,
think about all the things you go through.
Travel times, what sites do I want to see,
regulations to observe, budget.
Our research shows on Bing,
people spend on average weeks
to even months to plan a trip,
and to use our organizational tools.
I'm going to show you how we make that so
much easier with the new Bing.
I'm traveling to Mexico for
my cousin's wedding, and with the new Bing,
I now don't have to start
with something that's dumbed down,
like Mexico City travel tips.
I can ask for what I want.
First, let me just compare that against
what you get in today's search engines.
I'll type in this long query of what I really want,
and you get what you expect.
Links to go try to find the answer for yourself.
But we can do much better.
Let's try it in the new Bing.
I'll put in the long query,
which is essentially create an itinerary for
a five-day trip to Mexico City for me and my family.
Just like that, Bing Goes to work.
Just take a look at how it starts to compile.
Starts with Day 1, and we put it in there.
Look, arrive in Mexico City,
check into your hotel,
go check out maybe de Bella de Artes, have some lunch.
Then there's Day 2.
You see, isn't this just
so much better as a starting point?
Look, if you want to learn more, if you like, hey,
I don't love these five days, no problem.
Down there, we have links where
you can go and learn more.
We put in some nice touches in there as well.
Again, now, let's say business travel changes.
Oh, I only have a three day.
I don't have to go back out there and figure out.
I just say, Hey, make this a three-day trip,
and Bing reflows
that recommendation into a three-day trip.
Now let's just have some fun.
Let's say, Okay, yeah, I'm still trip planning.
We'd like to shop, where can I shop?
You get some shopping recommendations.
Like to go out at night. Make the most of the trip.
Where's the nightlife?
You get a list of nightlife.
You see this is just so much better than
today's search to start for your travel planning.
Let me show you a final example of chat and how I think
the new language models are going to maybe
help bring the world a little bit closer together.
Understanding different cultures is often
done through the arts like music and literature.
I've been fascinated by Japanese traditions.
[inaudible] share with me one of
his favorite searches from a while back on poetry.
With that, I'm going to use that as inspiration.
Without a clear idea of what I want necessarily,
I'm just going to type in a simple prompt,
top Japanese poets and
Bing starts to respond with a nice list.
You see this, it does a great job of
mixing the Japanese language and
the English language and it
knows since I'm Korean and English to do that.
Right away, I learn about this poet, Matsuo Basho.
It turns out he is one
of the greatest haiku masters of the world.
I love how it now enlists his name,
but we go ahead and we give you one of his famous haikus
in Japanese and we auto-translate it in English.
Great. I can say, I want to learn
a little bit more about Matsuo.
I can say, tell me a little bit more.
It'll give you another jumping-off point, that's great.
Then I can say, hey, tell me about another haiku.
Now, what I want you to reflect on is look how
easy this is to discover something new.
Normally, I might not have done this.
I might not have gone to learn about something
new in the world through search
because it's cumbersome to
click on links and have to deal with foreign languages.
But this is what we mean
by unlocking the joy of discovery.
Finally, when what you are searching for
doesn't exist and you need that spark of creativity,
bank and generate the content to help you get started.
We just finished planning that trip to Mexico and now
what I would like to do is I'd like to
share that information with my family.
They're all over the world but I can simply ask Bing,
hey, write an email,
sharing this itinerary that I've
researched and put it
into a thing for me to be able to send to my family.
Notice here how the emails starts, it's a great email.
Personalized touches, has the trip highlights,
it'll close here with a nice heartfelt message
, it's just a grip.
It just saves a bunch of time on your everyday work.
Now, for my family,
English isn't necessarily always
the first language so I can
ask Bing to just translate that in Spanish.
With a simple request, just basically say, hey,
translate that to Spanish,
Bing knows to take that entire email and
itinerary and convert that into Spanish.
In fact, Bing can translate
automatically over 100 languages.
My sister, I'm going to admit to you,
she's a better Spanish writer than I am,
but I might just impress her with this one here.
Even if I don't, I at least I saved
myself a lot of time from having to type the mail.
Our focus with Bing is to help generate
content and inspiration that
helps you with your daily life.
I'm going to give you a couple of more examples.
Let's show you one here.
I want to create a weekly meal-plan for my family of
four that has vegetarian options
and caters to those who don't like nuts.
Since I'm not the healthiest of eater,
I'm going to admit to you here on occasion,
this has given me a great list day-by-day,
breakfast, lunch, and dinner with all of the ingredients.
So much easier to help you
start to create a meal plan that says,
hey, let me get to a healthier meal plan.
Now, there's some things
in there I learned when I did this query,
which is chia seeds, which I don't normally eat.
That sparked another idea to show you.
Let's say I wanted to get this grocery list
by grocery section.
All I can do is say give me
that grocery list by groceries section.
Bing now takes that exact menu for the week and puts
all of the ingredients I
need by groceries section so when I go shopping,
now can be super efficient.
You see all of the ingredients there.
It's a great help.
Then finally, if you're looking for
family activities and you're struggling for ideas,
Bing can help you with fun things
like spontaneous trivia games.
My family and I were into music so I
asked Bing to create a '90s music trivia game.
Just look how great some of these questions
are and the answer so just creates that game for you.
We're having a little debate backstage
about one of the questions that comes up here,
which is, who wrote the hips song, Jump Around?
Is it Kris Kross or House of Pain?
I call it out because it shows you how clever it is.
Because for those of you who know your '90s hip-hop,
Kris Kross wrote a song called Jump,
but House of Pain was the one that wrote Jump Around.
That's how clever Bing is.
Bing is going to help you look great with
your family and have a bunch of fun.
Let's summarize where we are so far.
You've seen better search.
You've seen complete answers.
You've seen an incredible new chat experience and
the ability to spark your creativity all with Bing,
your AI powered cop-out for the web.
But what if that's still too much work?
What if you could have that co-pilot right alongside you?
Is there at the ready at anytime you want it,
aware of the context in which you're in.
What if you could get that co-pilot
on the 1.4 billion Windows PCs,
on the most used application on the PC, the browser?
Where you don't have to wonder, Team has created it.
I want to introduce you to your AI
powered co-pilot in Edge.
We've just updated Edge with
a new look and feel and new AI capabilities.
As you can see here, it's sleeker,
it's lighter and you're going to notice now that we've
integrated Bing in a really cool new way.
Let me show you.
Here I am on the Gap website.
I'm browsing around with my new Edge browser.
I want to read Gap's quarterly report.
I can navigate down to their earnings,
Click on "Q3",
and up comes the 15 page Gap PDF. It's pretty long.
I won't have time to read all that.
What I'd love is a summary of the key points.
I want to show you how now with the power of
Bing's AI capabilities within Edge, we can help.
With one click, I can open up the sidebar.
Now as you can see at the top of the Window,
we have two features, we have chat and compose.
Let me show how chat works.
I can use chat in Edge to simply
ask it to give me the key takeaways of the page I'm on.
I'll just say, key takeaways from the page,
and Bing and AI can now read that PDF and look how
great it comes up
with the summary of the key points here,
their earnings, the fact,
it's going to reaffirm full year guidance.
Very cool, a massive time savings.
But now I want to compare this with say,
Lululemon who also has their third quarter earnings.
Bing can now call out to the web,
pull information from outside of this page,
bring it into Edge,
compare it with the information that's on
this page all within Edge,
and I asked it to it in a table and
look how amazing this is.
Just like that in one table,
I can get an answer this question.
Think about how much time that
would have taken otherwise.
Let me try if we can take it one step further.
A top use case we've learned from our friends at
OpenAI is that developers are
really being more productive with ChatGPT.
Here we are on a Stack Overflow website discussion board
to learn a little bit about programming.
In this case, I'm researching
tips on how to parse the JSON file.
As we read through,
we find this great little code snippet and I'm like, oh,
it's fantastic except it's in
Python and we need it in REST.
All we need to do is highlight that text,
have it automatically copied over into the Edge sidebar,
and now, Bing
inside of Edge says, what do you want to do?
We'll say, hey, rewrite this code in REST.
With that simple command,
bin can go and take that code and rewrite
that automatically in the new programming language.
This is amazing. GitHub Copilot
has been a huge boost in developer productivity.
Imagine what the co-pilot can do for
people everywhere on any page.
One final thing to show you.
Not only can you better consume information,
but you can better create.
After our big announcement,
I'm going to want to write a LinkedIn post, let's say.
I'll just click "Create" on
the post and up
comes with a creation dialogue and LinkedIn.
But now I can open up the Bing
sidebar and you can see here,
I now will go to compose and I'll just give it a prompt.
I'll say, hey, introducing
the new AI powered Bing in Edge,
let's make that enthusiastic and generate
a draft and just lightly to help with enthusiasm.
Just like that, you get
a little draft and I can edit it and then with one-click,
it copies right over into my post dialogue,
I can add some hashtags to get it
some juice and just like that,
I've created a post.
All of these amazing new capabilities and
what we think is a revolutionary new experience.
World-class search, the ability to actually get
answers to your questions made easy with integrated chat,
and the ability to generate content when you need it to
spark your imagination brought
to you not only when you're searching,
but everywhere on the web,
courtesy of the new Edge browser.
With your copilot for the web,
we aspire to unlock that joy of discovery,
that wonder of creation,
and that feeling of empowerment and being
able to harness the world's knowledge.
Thank you on that on the presentation.
In a moment,
two of our engineering leaders
are going to come up and unpack a little bit about
the technical details of how we built
the new Bing and our approach to responsible AI.
But before I do, I'd like to invite Sam Altman to come up
and share his thoughts on this moment and
our joint work. Welcome, Sam.
SAM ALTMAN: Thank you Yusuf.
It's great to be here.
The new Bing experience really
looks fantastic and the depth of
it doesn't come through until you
get to use it. I hope you-all will enjoy.
OpenAI and Microsoft have been working
together for more than three years now.
We're so grateful to have
a partner that shares our vision,
our values of building advanced AI that's
safe and will have a very positive impact on society.
Thanks to our partnership and Azure's AI infrastructure,
we've been able to make pretty
significant strides in our research,
which has led to the creation of systems like
ChatGPT, Dolly, and Codex.
But we want to make the benefits of AI
available to as many people as possible.
That's why we've worked with Microsoft to get
this AI technology into the hands of
millions of people through Azure Open AI service,
GitHub Copilot and starting today, Bing.
The new Bing experience is powered by one of
our next-generation models that
Microsoft has customized specifically for search.
It takes key learnings from ChatGPT,
GPT-3.5, and the new model is faster,
more accurate and more capable.
We all search for things many times a day
and Microsoft has created and shipped,
a much better, more useful,
and more enjoyable search experience.
I feel like we've been waiting for this for 20 years.
I'm very happy it's here. I believe
that using AI to transform
critical tasks like these is going to greatly
improve our productivity and day-to-day quality of life.
I think this is the beginning of a very new era.
Going forward, we are excited to
continue to collaborate with
Microsoft very far into the future.
The two companies share a deep sense of
responsibility in ensuring that AI gets deployed safely.
It's very important to us we're
eager to continue learning from
real-world use so they
will create better and better AI systems,
you've got to do that in the real-world, not in the lab.
Our teams will continue working together to set standards
for the use of these systems across the entire industry.
Thank you very much, and now I'll hand it over to Dena.
DENA SAUNDERS: Thank you, Sam, Yusuf.
Hello. I'm Dina Saunders.
I'm the product leader of
the team who brought together the magic
behind being combined with
OpenAI's most powerful model to date.
Yusuf introduced you to Prometheus.
I'm here to tell you about the technical innovation
behind the experience.
There are five aspects contributing to the new Bing.
The first is a substantial under
the hood change to every layer
of things technological stack.
The second is a proprietary system
called Bing chat orchestration.
The third is state of the art prompt generation.
The fourth is around inference and
a brand new interactive user experience.
Our last development is the infrastructure
to scale for the opportunity ahead.
Today we'll touch on three of these.
Let me start with Bing chat orchestration.
This is what we use to gather
information needed to answer your question.
The chat orchestrator ingest
your long semantic query and fans out multiple searches.
In fact, you might remember as Yusuf was sharing earlier,
you can actually see in
the user experience the searches that
Bing plus Prometheus is issuing on your behalf.
In addition to searching,
we also inject additional sources of
information to help inform the model.
Let me pause here and say that is incredibly important.
We're actually pulling in fresh data, news, answers,
contextual signals such as your location and
the context of the conversation to feed into the model,
to help ground the information
that the model is then using to reason over.
With this fresh information,
some of the experiences that US have showed you earlier,
such as a sports example or inferring on recent events,
those would simply not be possible.
There's more though. This orchestrator in
parallel actually parses the documents
that it finds and identifies
important and relevant pieces of information.
This orchestrator is by nature,
curious and if it comes
across something particularly interesting
or insightful and actually starts that same cycle again.
This virtuous loop results in a package of
information that informs and
fuels the responses you are seeing.
This magic combination of reasoning capabilities with
fresh contextual data sparks
the innovative and groundbreaking combination
that powers the new Bing.
Next, I'll talk about inference and
the new interactive user experience.
The new interactive experience
seamlessly blends search and chat.
We feed a prompt, ie,
a proprietary set of instructions
into the model which synthesizes
the information reasons over all of the contexts
that we've gathered to form an answer to your question.
This overall process is called modal completion.
Think of it like when someone knows you
well enough to complete your sentence.
We run model inferences to generate
tokens ie words which we
stream to you in the user experience
real-time as you saw in Yusuf's examples.
These words combine with sentences
to form the response to your question.
When appropriate, Prometheus
enhances the text answer with
citations to sources and
rich structured answers from Bing.
In users presentation, you saw that
these inferences display in a
brand new, innovative, dual,
interactive user experience that
blends new conversational elements,
a classic search experience,
and a new interactive, immersive chat mode.
The last aspect of our differentiation and
innovation is around the infrastructure we have to scale.
Our differentiation comes really
from taking this innovation that we just shared with
you today and being able to ship
this at scale to millions of users worldwide.
Earlier, I talked about how we touched
every aspect of the tech stack to deliver this.
All of those steps that we just talked about,
those are done in the order of
milliseconds served at the speed of search latency.
When Yusuf was showing you the search page,
you can see near instant search results
juxtaposed on the conversational
insights, all streaming in.
This is possible because we're built on Azure,
the world supercomputer,
the best and most trusted Cloud platform available.
This was a huge effort, incredibly hard.
Behind the scenes, teams were working on powering and
building out machines in data centers worldwide.
We were carefully orchestrating and configuring
a complex set of distributed resources.
We built new platform pieces
designed to help load balance,
optimize performance and scale like never before.
There is no other company that can
do this like Microsoft and Bing.
As we roll out the product globally,
we're starting out small, focused on
learning and evolving with the help of your feedback.
We understand that with this major technological shift,
there will be new challenges along the way.
We're committed to learning with you.
On behalf of the team, we're delighted to
share this moment with you and with that,
let me introduce you to Sarah,
who's one of Microsoft's renowned responsible AI experts.
SARAH BIRD: Hi, I'm Sarah Bird.
I lead our Responsible AI engineering team for
new foundational AI technologies
like the Prometheus Model.
I was one of the first people to touch
the new OpenAI model as part of
an advanced red team that we pulled together jointly
with OpenAI to understand the technology.
My first reaction was just, wow,
it's the most exciting and powerful technology
I have ever touched.
But with the technology this powerful,
I also know that we have
an even greater responsibility
to ensure that it is developed,
deployed, and used properly.
Which means there's a lot that we have
to do to make it ready for users.
Fortunately, at Microsoft,
we're not starting from scratch.
We have been preparing for this moment for many years.
Since 2017, we have been investing in
a cross company program to ensure that
our AI systems are responsible by design,
which has pushed us to innovate across
our entire portfolio of
products to solve Responsible AI challenges.
Through that we have created a foundation of
Responsible AI product implementation patterns
that we can build on top of.
We're also not new to
working with generative AI technologies.
We've been using early versions in Office and Bing
for spell checking and grammar rewriting for years.
Obviously, lately, it's only getting more exciting.
In the past year, we launched the GitHub Copilot GA,
based on the codecs co-generation model,
the Bing creator powered by
the DALL-E image generation model,
and our own Florence multi-modal model.
That's a lot of generative AI.
With each one, we have learned more about
the risks that generative AI technologies can bring,
including well-known ones like
the perpetuation of stereotypes and bias,
as well as novel risks such as
jail breaks and we
have developed mitigations to address them.
In developing the new Bing,
we are building on our years of operating
large-scale consumer and enterprise services
and our deep partnership with OpenAI.
However, for this product,
we went further than we've ever gone before.
We have marshaled the full strength
of our Responsible AI ecosystem.
Scientists, researchers, ethicist, engineers,
and legal and policy experts to work together as
a single team to develop
approaches to measurement and risk mitigation strategies.
We've added Responsible AI to every layer,
from the core AI model to the user experience.
First, starting with the base technology,
we are partnering with OpenAI to
improve the model behavior through fine-tuning.
Second, we've built a state-of-the-art safety system.
Bing has been maturing its defensive technologies
over many years of operating a web scale search engine.
These include AI systems that understand user queries and
classified documents to ensure
safe and quality results for users.
We have combined these with newer AI technologies
that we've created in Azure
to moderate generative models.
We are continuously retraining all of
these safety models by mining anonymous Bing query logs,
and using the new OpenAI model to
generate thousands of example conversations.
We can update our defenses in minutes to
respond to gaps we find or changes in the world.
The result is a safety system that enables us
to act and understand data at
every level of the application so we can defend against
intentional misuse and mistakes from the AI.
Finally, at the application layer,
we are iterating on instructions as part of
the meta-prompt to guide
the model to produce great responses.
We have designed the user experience to ensure
users understand and are in control.
How do we know all of this works?
Measuring responsible harms is
a challenging new area of research.
This is where we really needed to innovate.
We had a key idea that we could actually use
the new OpenAI model as a Responsible AI tool to
help us test for
potential risk and we
developed a new testing system based on this idea.
Let's look at a tech planning
as an example of how this works.
Early red teaming showed that the model can generate
much more sophisticated instructions than
earlier versions of the technology
to help someone plan an attack,
for example on a school.
Obviously, we don't want to
aid illegal activities in the new Bing.
However, the fact that
the model understands these activities
means we can use it to identify and defend against them.
First, we took advantage
of the model's ability to conduct
realistic conversations
to develop a conversation simulator.
The model pretends to be
an adversarial user to conduct thousands of
different potentially harmful conversations
with Bing to see how it reacts.
As a result, we're able to
continuously test our system on
a wide range of conversations
before any real user ever touches it.
Once we have the conversations,
the next step is to analyze them,
to see where Bing is doing
the right thing versus where we have defects.
Conversations are difficult for most AI to
classify because they're multi turn
and often more varied.
But with the new model,
we were able to push the boundary of what is possible.
We took guidelines that are typically
used by expert linguists to label
data and modified them so
the model could understand them as labeling instructions.
We iterated it with it and
the human experts until there was
significant agreement in their labels.
We then used it to classify
conversations automatically so we could
understand the gaps in our system
and experiment with options to improve them.
This system enables us to create a tight loop of testing,
analyzing, and improving,
which has led to significant new innovations
and improvements in
our Responsible AI mitigations from
our initial implementation to where we are today.
The same system enables us to test
many different Responsible AI risk,
for example, how accurate and fresh the information is.
Of course, there's still more to do here today
and we do see places where the model is making mistakes.
We wanted to empower users to understand
the sources of any information
and detect errors themselves,
which is why we have provided
references in the interface.
We've also added feedback features
so that users can point
out issues they find so that we can get better over time.
Maturing and new technology takes time and collaboration,
which is why we're very excited for
this next phase where we can
share the technology with the real-world.
We look forward to seeing users unlock
its full potential and getting feedback from
stakeholders across society to help
develop this into an essential tool for the future.
Our goal is to continuously advance the state of the art.
Yusuf, back to you.
YUSUF MEHDI: Thank you, Sarah. To close,
we have one final piece of news to share.
I'm pleased to announce that the new Bing is live
today for desktop limited preview.
What this means is that we're going to
make Bing available today for
everyone to try on a limited number of queries
and then to sign up on the wait list to
get access to the full experience.
We're also starting with a select group of folks
who will have access to the full experience right away.
All of you here in the room,
you're all going to be on that list so you're all
going to get to try it right away.
In addition then we plan to expand
to millions of people in the coming weeks.
We are also going to be launching our mobile version.
Today is an important part of our journey.
But it's just the beginning. As Sarah said,
we intend to innovate quickly,
get your feedback, and continue to bring
new innovations and capabilities to everyone.
Welcome to the new Bing and Edge.
Thanks very much, thanks for being here.
[MUSIC]