So, welcome to this episode of TensorFlow Meets,

where I'm really delighted to welcome Razial

straight from his talk at the TensorFlow Dev Summit.

We spoke all about TensorFlow Lite.

Welcome, Raziel.

Can you tell us a little bit about you and what you do?

Thank you.

I'm an Engineering Lead at TensorFlow Lite.

I'm particularly focused on model optimization,

which the goal is change the models in certain ways

that can be smaller and faster to execute in edge devices.

(Laurence) I see, okay.

Now, I have to say, I geek out about TensorFlow Lite.

If I have a favorite thing in TensorFlow, it's TF Lite.

I just love doing it, the demos are always great.

When I do talks, it's usually about that.

So, it's really fun to have you here to learn a little bit more.

Can you tell us what's going on with TensorFlow Lite right now?

Yeah, we are doing a lot of things.

Like I said during the talk,

we're really trying to make as easy as possible

to deploy machine learning on devices.

And this is really important

because machine learning is moving from the server

to these edge devices.

And because of all of these characteristics

that you have on the Edge, it is challenging,

so we're trying to make that very, very easy.

Now, on the flip side, it also means that you have access to a lot of data,

it has a strong privacy-preserving component.

So, really, you can build products that otherwise wouldn't be possible

if you just rely on server-side execution.

So, the way I see our work is we're enabling the next generation

of AI-based applications.

Now, to the people who may be familiar with TensorFlow Lite,

they would think about it as running on Android or iOS,

but you're also extending now into IoT devices.

Yeah, we're extending to a lot of places.

The Google Assistant is one example of that

where if you think the assistant is running on Android,

is running on Google Home devices,

but it's also running in many other types of wearables.

They're really trying to push the boundaries

of where machine learning can execute.

So, we're trying to engage with them

and learn about those use cases, and power those.

We also have this new project about micro-controllers.

So, that's also very exciting

because we're pushing now more to IoT devices.

Again, we're really trying to push machine learning everywhere.

So, one of my favorite scenarios of using it on a mobile device

was the Cassava-- if you've seen that.

Where it's like these plants in Tanzania, and you can diagnose them for disease

by just waving your phone over them like this.

So, it's just a really exciting scenario.

Do you see any other good, fun scenarios around TensorFlow Lite

or maybe even using it in embedded systems?

Yeah, like I said, one of the great things of machine learning on the device

is that you also do away with the requirements of connectivity.

This is very important for many different countries--

the connectivity, perhaps, comes at a challenge or is costly,

so we're trying to enable all those things.

And, at the same time,

we're trying to support more and more hardware

so you can execute better and larger and more complex models on these devices

and do even more exciting things with them.

Like, one example is the Edge TPU,

and that's something we're really excited that we support it right now,

and we're working with them to make it even better and even faster.

If you took a look at my presentation--

we have this very nice slide of MobileNet.

MobileNet is a very common image recognition neural network,

and you can see the progress from executing in floating point in CPU

which is the baseline to be able to execute in CPU [quantized],

then with the GPU support that we have.

The GPU was already seven times faster than what you can get from a regular CPU.

And if you look at the numbers for the Edge TPU,

they're like over 60 times faster.

And this is really important because the quality of the model

is very much directly related to the size and the computation.

So, by enabling larger models, then we can also enable better quality

- for the users. - (Laurence) Excellent.

One of the things you were mentioning there about particularly complex models,

like quantization and pruning.

Now, it used to be that was part of the conversion process

when you're converting to TF Lite, but now you can do it beforehand?

Yes, we can do it before and we can do it after.

Each one has its pros and cons.

The nice thing of beforehand is that you can get even better quality.

So, typically, with all of these techniques,

you have a model, and you apply one of these techniques,

and it might affect your accuracy a little bit.

So, by doing it at training time, we're really trying to minimize that loss,

and in many cases, there is none.

Now, the nice thing is also that being part of the TensorFlow,

we have also Keras

as this really nice abstraction for neural network layers.

So, we're building on top of Keras,

and if you look at other APIs that are coming very soon,

it's extremely easy for anybody to just--

if they have already a Keras model,

it's very easy to apply quantization and pruning.

At the same time,

we're making a huge effort also on the post-training side

to make it very easy for everybody to also quantize a model, for example.

And that's because some people are not very familiar with training

or the first thing they want to try

is to just have a model that they got somewhere

and they want to quantize it.

So, that's why we have this other path of doing post-training quantization.

So, we're very invested in doing both,

like power users and somebody that just wants to try it

and see how things go.

That makes perfect sense.

So, if somebody is really inspired by this and wants to give it a try,

how would you recommend they get started?

Well, we have a lot of sites,

a lot of examples in the tensorflow.org site.

We have right now four completed applications

that people can try.

And we're trying to really build up our model repository.

And the goal is that if there is somebody out there

that perhaps doesn't have the experience or doesn't want to invest right away

but has a cool idea for a very interesting product,

they can go to our repository, see what is there,

and maybe there is something already that they can take advantage of.

And maybe they can build on that and expand

and others can learn from their experience.

Yeah, exactly.

I think it's all about making friction-less adoption.

Like, once you feel comfortable doing something,

then you can try more complex things, and more complex things.

So, thanks so much, Raziel.

That was, as always, informative and inspiring.

- Thank you. - So, thanks, everybody

for watching this episode of TensorFlow Meets.

If you have any questions for me or any questions for Raziel,

please leave them in the comments below.

And I'll also put a link to his talk from the TensorFlow Developer Summit,

so you can learn a lot more about TensorFlow Lite.

So, thank you so much.

♪ (ending music) ♪