EITAN MARDER-EPPSTEIN: How's everyone doing today?

Yeah?

Good.

All right.

Well, welcome to Google I/O. My name is Eitan Marder-Eppstein.

And I am an engineering manager here at Google.

And I work on augmented reality.

And I'm going to take a few polls throughout this talk.

And the first one is how many of you

are familiar with augmented reality in general?

OK.

Every time I give a talk like this, more hands go up,

which is a really, really great thing.

And today, what I'm going to do is

give a quick refresher about augmented reality for those

of you who maybe aren't quite as familiar with it,

and especially how augmented reality relates to smartphones,

which is something that we're really, really

excited about here at Google.

And then I'm going to talk about some of the things

that we're doing at Google to improve

our platform for augmented reality and the capabilities

that we give to some of these devices.

All right.

So I need my clicker.

So I'm actually going to go over here to get

the presentation started.

But off we go.

So smartphone AR stems from this observation

that over the last decade, our phones

have gotten immensely more powerful,

CPUs and GPUs have improved a lot.

But the ability of phones to see and understand

their environments, and really make sense of the world

around them, until very recently was

largely unchanged and limited.

So if you pointed your phone at this table,

it would allow you to take a picture of the table

or even a video of your friend climbing over the table.

But your phone wouldn't really have an understanding

of the geometry of the table, of its position

relative to the table as it moves through space.

And so what augmented reality seeks to do on smartphones

is to take all of this amazing advancement in computing power

and leverage it to bring new capabilities to your phone,

and to take your phone from beyond just the screen,

beyond its own little box, to expand it to understanding

the world around it.

So now, when my phone looks at this table,

it can see that there's a surface there,

that there are chairs next to it.

And as I move through the environment,

my phone can actually track its position as it moves.

And we think at Google that augmented reality

is really exciting.

And we've been excited to see some of the stuff

that you've built. And we've categorized it

into two main buckets where we think

augmented reality can be really, really great for applications.

So the first bucket is we think that augmented reality can

be useful on smartphones.

So recently, I was remodeling my kitchen.

All right, another poll-- how many of you

have remodeled anything in a house?

All right.

So if you've done that, you know that measurements

is a real pain.

And what I needed to do was measure for a backsplash.

We were buying some subway tile for our kitchen.

And I, instead of taking a tape measure out,

actually pulled out my phone, went to my counter,

and measured from point A to B to C. And I did all of that

without moving any of my appliances

where I would have normally had to move in order

to get an accurate measurement with my tape measure.

So AR can be useful in that way, just

from providing a better geometric understanding

about your environment.

AR can also be useful for shopping applications.

So recently, we had some very old chairs at my house.

And my partner and I were looking to replace them,

kind of like these chairs here.

And we were getting into a debate over which

chairs we liked more.

And so with augmented reality, we

were able to take a 3D model of a chair,

place it in the environment, see the exact size and scale

and color.

And we could have our arguments about inevitably

what kind of chair we would have at home

rather than exposing everyone to it at the store,

and be more targeted about how we made our purchase

and even buy this furniture online and feel much more

comfortable with it.

So that's how AR can just provide more utility

in your daily life.

But AR can also be fun.

So imagine a character running across the floor,

jumping onto this chair, and jumping onto this table,

or me sitting in one of these chairs

and having the floor drop out from under me

to create an ice fishing game.

Ice fishing sounds a little bit boring,

but I can tell you that in this game,

it's actually a lot of fun.

And AR can also be used for creative expression.

So here, now in your pocket, you have a lot of ability

to go out and create new things that

were previously only capable to be created by professionals.

So you can generate computer-generated content

on the go, on the fly.

You can take your favorite character

and put them into your scene, and have

your friend pose next to them.

Or you can take pizza or hot dogs or your favorite food

items, as we showed here, and put them

on the table in front of you.

But now, you have this amazing video editing capability

in your pocket.

And for those of you who have seen our AR Stickers

application on the Google Pixel phone,

you know what I'm talking about.

And for those who haven't, please check it out.

It's really, really cool to have this creation

power in your pocket.

All right.

So that's great.

AR can be useful.

AR can be fun.

But how do you actually build applications for AR?

How do you get involved as developers?

This is a developer conference.

So how many of you are familiar with ARCore, when I say ARCore?

All right, about half of you.

So ARCore is Google's development platform

for augmented reality.

We want to make it easy for you to build applications that

take advantage of these new capabilities

that phones provide, of the ability of phones

to see and understand their environments,

and to build applications that actually

react to this understanding.

And ARCore was launched a few months ago.

And it provides three main capabilities

to allow you to do this.

The first is something we call motion tracking.

So here, consider the example of taking the Scarecrow

from "The Wizard of Oz" and wanting

to place the Scarecrow at a taco stand

and make it seem like he's waiting in line for tacos

because everyone loves tacos.

So here, if I look at the Scarecrow with my phone,

ARCore actually understands its position

relative to a virtual object that I've placed in space.

So as I move a meter forward, the phone

knows that I've moved a meter in this direction.

And as I turn left, the phone also knows that.

It's able to track its motion as I move through space.

And now, if I combine that with my desire

to place the Scarecrow a meter in front of me,

I can put the Scarecrow right here.

And as I move my phone around, I can

change where I'm rendering the Scarecrow in the virtual scene

to match my physical environment.

So that allows you to register virtual objects

to your physical scene in a very natural and intuitive way.

The second capability that ARCore provides

is something called lighting estimation.

So here, continuing our "Wizard of Oz" theme,

we've got the Cowardly Lion.

And when you turn off the lights,

say we want to make the lion afraid because it's cowardly.

So here, ARCore is looking at the camera feed

and it is estimating the real world

lighting of your environment.

And with that estimate, ARCore can now

light characters in a realistic fashion,

helping you to build a more immersive experience that

looks natural because the virtual objects that you're

putting in your scene look correct.

So you can see the tone on the lion change

when it goes from light to dark.

And you can even script interactions

for your characters.

In this case, making the lion afraid when the lights go off.

And the third capability that ARCore provides

is environment understanding.

So here, as ARCore is moving around the world

and it's tracking its motion and it's also

estimating the lighting of the environment,

ARCore is also trying to recognize surfaces.

So ARCore might recognize this plane

below me which is the ground, or this surface here

which is the table, or even maybe

this vertical surface behind me.

And it allows you to place objects

that are grounded to reality.

So if we want to place the Android character

on this table, I can detect the surface

and actually place my virtual character on a physical object

in the world.

So those are three capabilities-- motion tracking,

lighting estimation, and environment understanding.

And when you combine them together,

it allows you to build these experiences that

were previously impossible, that bring

the virtual and physical worlds together

and meld them into a new reality that

enables people to see and experience your application

in a new and different light.

And we're really excited about this and the opportunity

to bring apps to our ecosystem for it.

And so we have worked really, really

hard to expose support for ARCare

on as many devices as possible.

And with help from our partners in our Android OEM ecosystem,

today ARCore is supported on over 100 million devices.

And we're working to increase that number every single day.

We believe that augmented reality

is a next shift in computing, and that soon everyone

will take for granted that this power is in their devices.

So that's our scale.

But we're also interested in scaling

the capabilities of ARCore.

We want to teach ARCore to do new and interesting things.

And that's what the rest of the talk is going to be about.

So today, we're announcing some new things in ARCore.

And they fall broadly into two categories.

The first is we're announcing some new capabilities

for ARCore, improving what these devices can do.

And those are Augmented Images and Cloud Anchors.

And we'll talk about them in the talk today.

And then we're also announcing some new tools for ARCore.

One new tool is how you can use augmented reality

on the web, which we think is really exciting.

And you can check a talk to that later today at 12:30 PM.

And another is how you can more easily

write 3D applications for Android and AR specifically.

We've introduced our scene form library,

which is a helper library for 3D rendering on Android.

And we encourage you to check out that talk at 5:30 today.

So enough about the preamble.

We're now going to get into the meat of it

and talk about what's new in ARCore.

And I'm going to kick it off with our first feature, which

are augmented images.