Introduction to TensorFlow 2.0: Easier for beginners, and more powerful for experts (TF World '19)

Subtitles section Play video

ASHLEY: My name is Ashley.
I'll be your host for today.
We'll get this show on the road.
I'd like to introduce our first speaker, Josh Gordon, who's
on the TensorFlow team.
Josh is going to talk about the ease of TensorFlow 2.0
and will walk us through the three
styles of model building APIs complete with coding examples.
So please help me in welcoming Josh.
[APPLAUSE]
JOSHUA GORDON: Thanks so much.
How's it going everybody?
AUDIENCE: Good.
JOSHUA GORDON: So let me just unlock this laptop.
And we will get started.
So I have only good news about TensorFlow 2.0,
which is about a month old.
It's wonderful.
It is massively easier to use.
And it's great both for beginners and for experts.
And also from a teaching perspective,
it has a lot of things to recommend it
that I'll talk about too.
And no big deal.
But maybe we can close the doors in the back.
It's a little bit loud.
Thanks.
All right.
So I'm going to get into an outline in a sec.
But one thing I wanted to mention right off the bat
is TensorFlow 2.0 has all the power of graphs that we
had in TensorFlow 1.0 except they're massively, massively,
massively easier to use.
In TensorFlow 1.0, the name "TensorFlow"-- a tensor
is basically a fancy word for an array.
So a scalar's a tensor.
A list is a tensor.
A cube is a tensor.
And flow refers to a data flow graph.
And in TensorFlow 1.0, you would manually define a graph.
And you would execute it with a session.
And this felt a little bit like metaprogramming.
And this is exactly the system you
would have wanted several years back
if you were an engineer and the challenge you faced was
massively distributed training.
That's great for that.
However, as a developer or a student or as a researcher,
you want something that feels a lot more like Python.
And on the right, you're seeing how TensorFlow 2.0 looks.
Basically, you can think of a tensor in a very similar way
to a NumPy ndarray.
And you can work with them imperatively,
exactly as you would expect in Python.
So you no longer need to use things like sessions
or anything like that.
And it works as you would expect, which is great.
But that's low-level details.
And here's some of the things I'd like to talk about.
So this is a rough schematic of how TensorFlow 2.0 looks.
And TensorFlow 2.0 is a very, very large system
with many moving pieces.
It's a whole framework for doing machine learning.
And what I'd like to do here is show you a couple of the pieces
and what some of your options are to use it.
So we will start with designing models using my favorite API
of all time, which is Keras.
And what's awesome in TensorFlow 2.0--
and this is a really important point--
there is a spectrum of use cases.
And all of these are built into the same framework.
And you can mix and match as you go.
And what this means is if you're a total novice
to deep learning, you can start with something
called the Sequential API, which is
by far the easiest and clearest way
to develop deep learning models today.
And it's wonderful.
You can build a stack of layers.
You can call things like compile and fit.
And that is 100% valid TensorFlow 2.0 code.
It is just as fast as any other way of writing code.
There is no downsides to it at all
if your use case falls into that bucket.
And what's really important in TensorFlow 2.0 is, as helpful
to you, you can optionally scale up in complexity
using things like the Functional API
or going all the way to subclassing
all in the same framework.
And you can mix and match as you go.
And what this means is when I'm teaching this, for example,
I can start in these really simple, clear, easy ways.
And then when I want to write gradient descent from scratch,
I can do that.
Or write custom layers from scratch, I can do that.
It's very easy.
So let's take a look at what some of these pieces look like.
And I only have one slide on sequential.
And the reason is we have so many tutorials on it.
There's an entire book on it, which
I'll recommend at the end.
So I've just one slide on this.
But in case you're new to this, the Sequential API
lets you define a stack of layers.
And this is by far the most common way
to build your models.
And something like, you know, I'd
say 80% to 90% a machine learning models
will fit into this framework.
So you define a stack of models.
And that's great.
What's interesting is when you're using the Sequential
API and the Functional, although a lot of developers
don't realize this, what you're actually doing
is defining a data structure.
And this means you can do things like model.summary
and see a printout of all the layers and all the weights.
It also means that we can do compile time checks.
So when you call model.compile, we
can make sure all your layers are compatible.
It also means that when you share your model
with other people, for example, imagine
that you want to do fine tuning for transfer learning.
If you have a model that's defined with a Sequential
API or the Functional API, because it
has a data structure of a stack of layers or, with a Functional
API, a graph of layers, you can inspect that data structure
and you can pull layers out of it and get the activations.
And you can do fine tuning things like that really easily.
Anyway, defining a stack of layers is very common.
In TensorFlow 2.0, this works exactly
like it does in Keras.io with the multipack in Keras.
So that's great.
One thing that's also extremely powerful that a lot of people
are new to is the Functional API.
And the Sequential API is for building stacks.
The Functional API is for building DAGS
or Directed Graphs.
And I just want to show you how powerful this is.
So to be honest, most of what I've been doing myself
is using either the Sequential API
or going all the way to subclassing and just
write everything from scratch.
But I heard a really awesome talk on the Functional API
a couple weeks ago in Montreal.
And I've been using it a lot since.
And I love it.
So I just want to show you what it can do.
And so I just want to show you what a quick model would
look like for something like visual question answering.
And a lot of the time, when you start with machine learning,
you spend a lot of your time-- or deep learning,
rather-- you spend a lot of your time building image classifiers
and doing things like cats and dogs.
But we can take a look at a slightly more sophisticated
model.
And this is VQA.
And in VQA, you're given two inputs.
One, you're given an image.
So here we have a pair of dogs.
And you're given a question in natural language.
And here, the question is asking, what color
is the dog on the right?
And so to answer a question like this,
you need a much, much, much more sophisticated model
than just an image classifier.
You can still phrase this as a classification problem.
And if you Google for VQA, there's
two really excellent papers that will go into detail.
But you can imagine you have some model--
well, let's talk about how we would do this.
Here, we have a model with two inputs.
We have an image and a question.
And if you take a machine learning course or deep
learning course, rather, you'll learn
about processing images with convolutional layers
and max pooling layers.
And you'll learn about processing text with things
like LSTMs and embeddings.
And one thing that's really powerful about deep learning
is all of these layers, regardless of what they are,
they take vectors as input.
And if you're a dense layer, you don't
care if your input happens to be the output
of some convolutional layer or if your input happens
to be the output of some LSTM.
It's just numbers that you're taking as input.
So in deep learning, there's no reason
that we can't process the image with the CNN,
process the text with an LSTM, and then concatenate the result
and feed that into a dense layer and phrase
this as a classification problem.
So you can imagine the output of our dense layer
might have 1,000 different classes.
And each class corresponds to one possible answer.
So here, if the answer is golden,
we want to classify both of these inputs jointly as golden.
And I want to show you how quickly we can design something
like this with the Functional API, which is really amazing.
So, cool, I probably should've flipped to this slide.
But this is what I was just talking through.
So this is the architecture that we want.
This is one model.
And it's going to have two heads.
And the first head is going to be a standard stack of CNNs
and max pooling layers.
And this is exactly the same model
you would use to classify cats and dogs.
And you can do all the same tricks
that you will learn about there.
You can import, like here, I want to show you
how to write it from scratch.
But there's no reason that you couldn't import like MobileNet
v whatever, and use that to get activations for the images.
But basically we're going to go from an image to a vector.
And in the other head, we're going to process the question.
And we're going to go from a question to a vector.
And to do that, we use an embedding in an LSTM.
At the end, we can concatenate the results and classify it.
And this is nearly the complete code for this entire VQA model.
Actually it is the complete code for the model-- a Hello World
version of it, which is nuts when you look at it.
So here, this is our image classifier.
And you would want something much deeper.
But this would be your Hello World image classifier.
So a vector is going to go in.
And just some random tips.
I can see my slides.
That's cool.
So you'll notice in the first layer,
this is a convolutional layer.
It has 64 filters, each of which is 3 by 3.
And it has relu activation.
And you see in the input shape there,
I'm specifying how large my image is.
Although you'll find that Keras can often
infer the input shape, whenever you can you should specify it.
And it's just one less thing that can go wrong.
So fully specify it, catch bugs early.
After that we're doing max pooling.
And the important part is we're flattening it.
So that's actually a sequential model
that we're using inside the Functional API.
After that, we're creating an input layer.
And this is for the Functional API.
And we're beginning to chain layers
together to build up a graph.
So here what we're doing is we're changing the vision model
to the input layer.
So that's the first half of our model.
And here's the second half.
And we're almost done.
So this is the model that's going to process the question.
Here we're creating another input.
And I don't have the preprocessing here.
But you can imagine that we've tokenized the text.
And we vectorized it.
We've padded it.
And then what we're doing is we're
feeding that into an embedding and then into an LSTM.
And this is exactly what you might
do if you were training a text classifier.
The important thing is that a vector is coming out,
and we're chaining these together to build a graph.
And at the very end--
and this is the magic bit about deep learning-- we can simply
concatenate the results.
Nothing simple, but it's one line of code, which is nice.
Nothing simple conceptually.
But what we can do is we can concatenate the results.
And now we just have a vector.
And just like any other problem, now that we have this vector,
we can feed it into dense layers, and we can classify it.
And so here's the tail of our model.
And now we have a TensorFlow 2.0 model that will do VQA.
And this will work exactly like any other Keras model.
So if you want, you can call model.fit on this thing.
You can call model.train_on_batch.
You can use callbacks.
If you want, you can write a custom training
loop using GradientTape.
And so I think this is really powerful.
And what's nice about these Functional APIs,
just like sequential models, because there's
a data structure behind the scenes, there's a graph.
TensorFlow 2.0 can run compatibility checks
and make sure your layers work with each other.
So it's really, really nice.
So basically, if you haven't used the Functional API,
either you just learned about Keras Sequential
from a lot of books or you're coming from PyTorch
and you've only used things like subclassing,
I really encourage you to try it out.
I've had only positive results in the last few weeks.
So I love it.
Other things, of course, so that graph
I just made in Google Slides.
But because you have a data structure,
instead of calling something like model.summary,
you can call a model plot model.
And you'll actually get a nice rendering
of a graph that looks exactly like what I just showed you.
So for complicated models--
so this is cool.
The only time I found it really useful
is when you have complicated models, things like ResNets,
you can actually plot out the whole graph
and just make sure that it looks as you
expect as you're assembling it.
So it's really nice.
Anyway, and then there's another style in TensorFlow 2.0.
So the last two things I showed you
were built into what you'll find at Keras.io
and that's multipack in Keras.
Keras in TensorFlow 2.0 is a superset
of what you find in Keras.io.
And this is something that's new.
So this is subclassing.
And this is a Chainer/PyTorch style
of developing models, which is also really, really nice.
And basically you'll see this little spectrum here.
You're getting increasing control as you move up.
And so in this style, you can be a researcher
or a student learning this for the first time.
But what you're saying is, I just
want to write everything from scratch to learn how it works
or because I have some special use case that doesn't
fit into the other ones.
So here what we're doing is we're
defining a subclass model.
And I also really love this.
This feels a lot like object-oriented NumPy
development.
So what we're doing-- and the idea is basically--
it's very, very similar in all these frameworks.
The framework gives you a class.
And here this class happens to be model.
And there's two chunks to writing a subclass model.
You have the constructor and the call method
or the forward method or the predict method.
And in the constructor, you define your layers.
So here I'm creating a pair of dense layers.
And it's the exact same layers that you'd
find in the Sequential and Functional model APIs,
which is great.
So basically, you learn these layers once,
you can use them all over the place.
And in the call method or the forward method,
you describe how these layers are chained together.
So here I have some inputs.
And I'm feeding the inputs through my dense layer.
And then I'm feeding that result through my second dense layer
and returning it.
And what's nice is this is not symbolic.
So if you're curious what x, is you can just
do print x, of course, like you would in Python,
and that will give you the activations
of that first dense layer.
If you want you can modify it.
So for example, here I've highlighted relu.
Let's say for some reason I'm not
interested in using the built-in relu activation.
I want to write my own.
What you can do simply is just remove that.
And you can write your own with just regular Python flow right
there.
So there, I've written relu using the built-in method.
But I could also just write that using regular Python.
So this is great for hacking on things.
It's great.
If you want to really know the details of what exactly
is flowing in and out of these layers,
it's a perfect way to do it.
Also for example, there's nothing here
that saying that you have to use these built-in dense layers.
So if you look at the code for the dense layer, that's
doing something like wx plus b, you
can absolutely just write that from scratch in Python
and use that here instead.
And then these are just a whole bunch of references.
I'm going to tweet out these slides when we finish so you
don't have to write it down.
But what these are, these are guides
from TensorFlow 2.0 that will go into detail on how you write--
how do you use each of these three styles of the Keras API?
How do you write custom layers and stuff like that?
They're great.
And then we have a couple of tutorials
that I recommend that are really, really nice
that show different ways of building stuff.
If you haven't seen these, by the way,
the segmentation one is super-nice.
We wrote it this summer.
It runs really, really fast.
And I think you'll like it.
Also, by the way, because this is an Intro to TensorFlow 2.0
talk, let me just show you what the tutorials are
in case you're new to these.
And there's just one thing I want to point out.
So this is the tutorial on our website.
Big surprise.
What I wanted to mention, and I think
this is a really nice feature of TensorFlow.org,
obviously this is a web page HTML.
But this is just a direct rendering
of this Jupyter Notebook.
So the web page is just the Jupyter Notebook.
And the reason we've done that is all the tutorials
are runnable end to end.
So you can install TensorFlow 2.0 locally
and run this tutorial.
Or if you click run in Colab, this is exactly the same page
as you have on TensorFlow.org.
For all of these, you can do Runtime, Run all.
And this has the complete code to reproduce the results
that you see here.
And what this means is that our tutorials are testable.
And if you know the expression, trust but verify?
So for a long time, I've seen nifty tutorials
that are like, yeah, like let me show you
how to write this like neural machine translation model.
And then the code doesn't work or it's
missing a key piece that's left as an exercise to the reader.
So at least all the tutorials on the website, all of them
run end to end, which is really, really nice.
Some we still have plenty of work to do cleaning them up.
But at least they guarantee you they have the complete code
to do the thing.
So I really like that a lot.
All right.
I wanted to talk a little bit about training models.
And so basically, there are several ways to train models.
And again, you can use-- the nice thing
about TensorFlow 2.0 is you can use the one that's
most helpful for your use case.
So you don't always need to write a custom training
loop from scratch.
So you have other options.
And the first is that you might be familiar with from Keras
is just simply calling model.fit.
And what's really nice about model.fit,
it doesn't care if you have a sequential model, a functional
model, or a subclass model.
It works for all them.
And model.fit-- it's fast.
It's performant.
It's simple.
One thing that's a little bit less obvious,
when you do model.fit, this is not just
the baby way of training models.
So if you're working in a team and you call model.fit,
you've reduced your code footprint by a lot.
This is one less thing that your friends
need to worry about when they're playing with your models
down the road.
So if you can use the simple things,
you should unless there's a reason for more
complexity, of course, just like in regular software
engineering.
The nice thing about fit is you can pass in different metrics.
In a lot of examples, you'll see things like accuracy.
By the way, TensorFlow 2.0 has really nice metrics
for things like precision and recall built in.
You can also write custom metrics.
And something that's really helpful too is callbacks.
So for instance, these are things
that I don't see a lot of new developers using.
And they're super helpful.
So callbacks, and one of my favorites is EarlyStopping.
And so typically, when we're training models,
we need to prevent overfitting.
And a really wonderful way to do that
is to make plots of your loss over time
and so on and so forth.
These callbacks can do things like that for you
automatically, which can be helpful as well.
You can also write custom callbacks.
So a cool thing would be like let's say
you're training a model that takes
a very long time to train.
You could write a callback to send you a Slack notification
after every epoch of training completes.
And so that can be really nifty too.
So callbacks are great.
And then I don't have slides for train_on_batch here.
But I did want to show you custom training
with a GradientTape because this is also very powerful,
especially for students who are learning this
for the first time and don't want a black box
or for researchers.
And so here is a custom training loop.
And I have an example of this for you in a minute
just with linear regression so you
can see exactly how this works.
But this is a custom training loop.
And what we're doing here is we have some function.
And for now you can pretend that atf.function in the orange box
doesn't exist.
So that's optional.
So just pretend that doesn't exist.
We have some function that's taking features and labels
as input.
And whenever we're doing training in deep learning,
we're doing gradient descent.
The first step in doing gradient descent
is getting the gradients.
And the way all frameworks do this is by backdrop,
which is reverse-mode autodiff.
And the implementation in TensorFlow
is we start recording operations on a tape.
So here we're creating a tape.
And we're recording what's happening beneath that tape.
So we have just regular Python code
that's calling the forward method or the call
method, rather, on your model.
So we forward the features through our model.
And we're computing some loss.
And maybe we're doing regression.
And that's squared error.
And then what we're doing is we're
getting the gradients of the loss with respect
to all the variables in the model.
And if you print those out, you'll
see exactly what the gradients are.
And then here, we're doing gradient descent manually.
We're applying them on an optimizer.
We can also write our own optimizer.
And I'll show you that in a second.
Anyway, this is a custom training loop from scratch.
And what this means is that if you--
so in model.fit, you can use optimizers
like RMSprop and Adam and all this stuff.
But if you'd like to write like the [? Sarah ?] optimizer,
you can go ahead and write it in Python.
And it will fit right in with your model.
So this is great for research.
tf.function, by the way, first of all,
you never need to write it.
Your code will work the same.
But if you do want a graph in TensorFlow 2.0 or if you--
basically, if you want to compile your code
and have it run faster, you can write that tf.function
annotation.
And what this means is that TensorFlow 2.0 will trace
your computation, compile it, and the second and on time that
you run this function it will be much, much,
much faster because it's running entirely in C++.
So all there is to graphs in TensorFlow 2.0 is basically
at tf.function.
But it's optional.
You don't even need to use it.
But it's easy performance if you need it.
And then I just want to make this super concrete
because this is a Getting Started with TensorFlow 2.0
talk.
There's a lot of awesome tutorials
that will quickly show you on the website
how to train image classifiers and whatnot.
But I think a good place to start too
is just looking at linear regression.
And the reason is it's gradient descent.
All deep neural networks are trained by gradient descent.
And a nice place to start is seeing exactly what that is.
And because I have a--
I know it's tiny.
But because I have a graphic here on the left,
I'm just briefly going to explain
how linear regression works.
And it's the same pattern for deep neural networks
too, which is really surprising.
So in linear regression or deep neural networks,
you need three things.
The first thing you need is a model,
which is a function that makes a prediction.
And so a model for linear regression,
you might have learned in high school, could be y
equals mx plus b.
We're trying to find the best fit line.
And we can define a line y equals mx plus b.
That means we have two parameters or variables
that we need to set.
We have m, which is the slope, right?
And we have b, which is the intercept.
And by wiggling those variables, we
can fit the line to our data.
So on the right, you'll see a plot.
And we have a scatter plot with a bunch of points.
And we have the best fit line.
And the idea is now that we have a model or a line
that we can wiggle, we need a way of saying or quantifying,
how well does this line fit the data?
One way to quantify how well the line fits the data
is squared error.
What that means is you drop a line on the page.
And then you measure the distance
from your line to all the points.
And you take the sum of the squares of that.
The higher the sum of the squares is,
the worse your line fits the data.
The better your line fits the data,
the lower the sum of the squares.
So you can have a single number, which
is called loss, that describes how badly your line fits
the data.
And then you want to reduce that loss.
And you know that if the loss gets to a minimum,
you're line will fit the data well.
And you found the best fit line.
The way we reduce the loss is gradient descent.
And on the left, you'll see a gradient descent plot.
And we're looking at loss or a squared error
as a function of two variables-- m and b.
And you can see that if we set m and b with a random guess
to start, our loss is pretty high.
And then as we wiggle them, we can reduce the loss.
The trick is, how do we figure out
which way to wiggle m and b?
And briefly-- I don't want to go on too much of a tangent--
there's two ways to do that.
If you forget calculus, you can find the gradient numerically.
And it's not rocket science.
You take m, and you wiggle it up a little bit,
recompute your loss.
Then you take m, and you wiggle it down a little bit.
And you recompute your loss.
You figure out which way makes the loss go down.
That's the direction you're going to be wiggling m.
Do the same thing or b.
That's very, very slow.
And there's faster ways to do it too.
But I want to show you what this code looks
like in TensorFlow 2.0.
So basically, you don't have to use Keras at all.
You can also use TensorFlow 2.0 a lot like you would use NumPy.
And basically, whenever you see something like tensor,
just replace that in your head with NumPy ndarray.
So we have constants.
And you see as you print out a constant,
it has shape and a data type.
One really nice thing about TensorFlow tensors
is they have a NumPy method.
So you can go straight from tensors to NumPy,
which is great.
So you're free of the clutches of TensorFlow 2.0.
Tensors have a shape and a data type.
And then just like you would expect in NumPy,
we have things like distributions.
So if you want to create some random normal,
here's how you would do it.
I just want to fly through this really quick.
And you can do math in TensorFlow 2.0 a lot
like you would do math in NumPy.
So basically, just like you have things like numpy.square
and numpy.matrixmultiply, TensorFlow
has all these same things too.
And the idea is the same.
The names might be slightly different.
You might have to poke around a little bit.
But the names are all there.
And here's an example of like very, very simple.
Here's how we get the gradients using the GradientTape.
But this is more concrete.
So here we have a constant.
That's 3.
And we have a function, which is x squared.
And so if you think about your rules of calculus,
if we have 3x squared, you take the 2.
And you multiply it by the 3.
And you get 6.
And if you walk through this code,
you'll see that it returns you 6 too.
So basically, this is how we get the gradients using
GradientTape.
And you can also do that with all the variables and layers
at once.
So here we have a pair of dense layers.
And we're calling the dense layers on some data.
And we're getting the gradients also under the tape.
And let me just show you what this looks
like in linear regression just to make this concrete.
So this is code for y equals mx plus b.
And the first thing I wanted to mention
is how you install TensorFlow 2.0.
If you're running in Colab--
right now in Colab, TensorFlow 1.0 is installed by default.
But there's a magic flag you can run.
So that magic command will give you the latest
version of TensorFlow 2.0.
If you're running this locally, you can visit
Tensorflow.org/install.
And you can do pip install tensorflow.
So anything you can do in Colab, you can do locally.
But I have this here.
This is convenient.
TensorFlow is just a Python library.
You can import as you always would.
And the first thing we do in this notebook--
I know I flew through some of those code examples--
but what we're going to do is just create a scatterplot,
so just some random data.
And then we're going to find the best fit line.
So here what we're doing is we're creating some data.
Let me see if I've run this before.
Yeah.
And then we're plotting the data.
And here's what we get.
The slides had TensorFlow constants.
And these are TensorFlow variables.
And basically, constants are constant,
and variables can be adjusted over time.
You almost never need to write code this low level.
This is pretending that we don't have Keras.
We don't have any built-in fit methods.
We just want to do this from scratch.
But here's how you would do it from scratch.
So I'm creating some variables.
Here I've initialized them to 0.
I probably should have initialized them
to a random number.
But this will work too.
And then here, this doesn't look scary at all.
This is the predict function for linear regression.
So this is our equation for a line y equals mx plus b.
And our goal is going to be to find good values for m and b.
Here's our loss function.
And what we're doing is we're taking the results
that we predicted minus the results that we wanted.
We're squaring it.
And we're taking the average.
So that's squared error.
And then as we go through this notebook,
we can see our squared error when we start.
And here's gradient descent from scratch
pretending that we didn't have anything like model.fit.
So for some number of steps, what we're doing is
we're taking our x's, and we're forwarding through the model
to predict our y's.
We're getting the squared error, which is a single number.
And then we're getting the gradients of m
and b with respect to the loss.
And this will literally tell us-- if you print those out,
those aer just numbers.
And the gradients point in the direction of steepest ascent.
So if we move in the direction of gradients,
our loss will increase.
We want the loss to decrease.
So we move in the reverse direction of the gradient,
which is gradient descent.
And here, again, this is like the lowest possible level
way to write this code.
We're doing gradient descent from scratch.
So we don't have any optimizer.
What we're doing is we're taking a step
in the negative gradient multiplied by our learning
rate.
And that adjusts m and b as we go.
And if you run this code, you'll see the losses decreasing.
And you'll see the final values for m and b
and plot the best fit line.
And then what I did for you is I wrote--
this is a little bit uglier.
But I wrote some code to produce this diagram
just so you can see exactly what the gradient descent is doing.
So that's how you would write things
from scratch in TensorFlow 2.0.
And what's really, really awesome,
when you move to things like neural networks,
this code is basically copy and pasted.
So if you can compare this custom training
loop for linear regression to the custom training
loop for DeepDream or any of the fancy models on the website,
it looks almost identical.
You always have this tape, and the steps are the same.
You make a prediction.
You get your loss.
You get the gradients.
And you go from there.
So that's really nice.
All right.
Other things I wanted to mention.
Oh, I got to move a lot faster here.
So in terms of data sets, basically
you have two options in TensorFlow 2.0.
The first option is at the top.
And these are all the existing Keras data
sets that you find at Keras.io.
And these are great.
They're good to start with.
They in NumPy format.
And they're usually really tiny.
They fit into memory no problem.
Then we have this enormous collection of research data
sets, which is awesome.
And that's called TensorFlow Datasets.
And here, I'm showing you how you can download something
like cycle_gan in TensorFlow data sets.
What's important to be aware of, I just
have a couple of quick tips.
If you're downloading a data set in TensorFlow data format,
by the way, it's going to give you
something called-- the data sets not going to be NumPy.
It's going to be in tf.data.
And tf.data, it's a high-performance format
for data.
It's slightly trickier to use than what you might be used to.
And so if you're using TensorFlow data sets,
you have to be very, very careful to benchmark your input
pipeline.
If you just import a data set and try and call
model.fit on the data set, it might be slow.
So it's important to take your time
and make sure that your data pipeline can read images
off disk and things like that efficiently.
And I have just a couple tips that might be helpful.
TensorFlow data sets recently added an in_memory flag.
So if you don't want to write fast input pipelines,
you can pass in_memory.
And you can insert the whole thing into RAM.
So that will make it really easy.
And it also added this caching function,
which is really, really nice.
So here's some code for tf.data.
And maybe we have an image data set,
and we have some code to preprocess the images.
And let's pretend that that preprocessing code is expensive
and we don't want to run it every time, on every epoch.
What you can do is you can add this cache line at the end.
Cache will keep the results of the preprocessing in memory.
And it will make subsequent runs of your pipeline much faster.
So cache is a really handy thing to be aware of.
The goal here is not to give you all the details for this.
It's just to point you to some things that
are useful to know about.
You can also cache to files.
So cache without any parameters will cache it into RAM.
If you pass a file name, you can actually
cache to a file on disk too.
One thing that's awesome in TensorFlow 2.0 that you--
if you're an expert, you'll care about,
and if not, you'll care about down
the road-- is distributed training.
And I'm going to skip some slides to move faster.
What I wanted to say briefly is distributed training
in TensorFlow 2.0 is awesome.
And it's awesome because if you're doing single machine,
multiple GPU synchronous data parallel training
or you're doing multimachine, multi-GPU synchronous data
parallel training, you don't need
to change the code of your model, which
is exactly what I care about.
And so it's awesome.
And basically, here is some Keras model.
This happens to be a built-in application for ResNet.
But it doesn't matter.
And I just want to show you how we run this code on one
machine with multiple GPUs.
We just wrap it in a block.
That's it.
And model.fit is distribute aware and will work.
So you don't need to change your model to run on multiple GPUs.
And this particular strategy is called the MirroredStrategy.
There's different strategies you can
use to distribute your models.
There's another strategy-- it's like Mirrored MultiWorker,
which you can change that one line.
And then if you have a network with multiple machines on it,
again, your code doesn't change.
So that's awesome.
I really, really like the design of this.
All right.
Other things that are awesome about TensorFlow 2.0
that I'd really encourage you to check out, especially
if you're learning or you have students,
is going beyond Python.
So we've talked about training models in Keras.
And I just want to show you some of the cool things you
can do to deploy them.
And roughly, there's a bunch of different ways
that you can deploy your models.
The way that I was used to a few years ago
as a Python developer was I'd throw up REST API.
And I can do that using like TensorFlow Serving or Flask
or whatever you want.
And I'd serve the thing behind an API
because that's what I know how to do.
There's a couple of things that I've been learning since then,
which have been great, and that's basically
deploying models in the browser with TensorFlow.js,
deploying them on Android and iOS
using TensorFlow Lite, and very recently,
running them on Arduino using TensorFlow Lite Micro.
And I have a couple of suggested projects for you
that I just wanted to point you to.
So the first is tinyML.
And this was a blog post--
this was a guest article on our blog
a few weeks ago by the Arduino team.
And this article is basically a tutorial.
And what we're looking at is an Arduino.
It's a microcontroller.
And if you're new to microcontrollers,
it's a system on a chip.
But it also has pins that it can run voltage to
or read voltage from.
And so for instance, you can plug an LED light bulb
into one of those pins, and you can
have C code which runs voltage to the pin
and turns on the light.
Likewise, you could have an accelerometer attached
to one of the pins, and you could
have C code that reads from the accelerometer
and gives you some time series data.
So that's what a microcontroller is.
It's a computer plus these, basically, pins
that you can read and write to.
TensorFlow Lite is the code we used to deploy
TensorFlow models onto phones.
But recently, TensorFlow Lite Micro
now lets you deploy them onto Arduino.
And these things are smaller than a stick of gum.
This is a really nice one.
It has-- this is the Nano that has built-in sensors.
I have one at home.
But it's still about $30.
And it has a built-in accelerometer, a temperature
sensor, stuff like that.
But anyway, what we're looking at here,
this is a demo using the accelerometer someone trained
to model to recognize two gestures.
One is like a punch, and one is an uppercut.
And you can see they're holding the Arduino in their hand.
And as they're moving it, the laptop
is recognizing the gestures.
But the workflow-- and this is in the blog post--
is not bad at all considering how much power you're
getting out of it.
So what you're doing--
and let me see if I can show you the steps.
Basically, the first thing you need to do is capture the data.
And I wanted to bring an Arduino with me.
I thought it would be better just
to quickly show you these GIFs.
But you can plug the Arduino into your laptop
with a USB cable.
You need to collect training data for your model.
So what you do is you hold the Arduino in your hand.
And you collect a bunch of data for your punching gesture.
And you save that to disk as a time series.
And the diagram on the right there--
I was just capturing the IDE this morning--
as you move the accelerator-- as you move the Arduino around,
you're reading from the accelerometer.
And what you get out is just a CSV file with time series data,
exactly like you would have if you were looking
through the time series forecasting tutorial
on TensorFlow.org.
And you can do the same thing for your other gestures.
So what you do is you gather data.
You save CSV files.
You upload them to Colab.
In Colab, you write a model using TensorFlow 2.0 in Keras
to classify the data.
And that's just a regular Python model.
You don't need to know anything special about TensorFlow Lite
to do it.
And then what I wanted to show you--
and you can find the complete code in the blog post--
there's a very small amount of code
that you need to convert your model from Python down
into TensorFlow Lite format to run on device.
And this is a tiny amount of code.
Once you have this model--
and this is a little unusual--
we're going to convert it to a C array.
And that's because this Arduino, it
has like 1 megabyte of disk space
and I think like 256 KB of RAM.
So we're converting it into the smallest,
simplest possible format.
But you convert it to a C array.
And then what you can do-- we have an example.
And you can paste the C array into the example.
And now you're running your TensorFlow model on device.
And it's amazing.
I've had so much fun doing this.
The reason I wanted to point it to you, if you've kids or you
have students that want to play with this stuff,
if you teach them just to train a time series forecasting
model, which is a really valuable skill.
We have a pretty good tutorial for it.
I mean, it's interesting.
But it's vastly more interesting if they can then
deploy it on device.
It gives them something tangible that they can play with
and show their friends.
It's super cool.
Also this is a brand new area.
So tinyML referring to doing machine
learning on small devices I think has a lot of promise.
Another way to deploy your models,
which is super-powerful, is in JavaScript.
And this is using TensorFlow.js.
Here's another project suggestion.
One of the first tutorials you might run through
in deep learning is sentiment analysis.
So given a sentence, predict if it's positive or negative.
Also really valuable skill can be somewhat dry, right?
But the first time I've ever been super-excited about
sentiment analysis, other than many years ago when I found
that I can make money for it--
that was not the link that I wanted.
If you go into this GitHub repo link from the slides,
you will find that you can run sentiment analysis live
in JavaScript.
So for example, this is just a web page.
And down at the bottom here, the movie was awesome.
So I wrote a sentence.
And you can see that's predicted positive.
And here, you can see that's predicted negative.
And what's nice about doing JavaScript in the browser
from a user perspective is there's nothing to install,
which means if you're a Python developer
and your goal is to have a cool demo,
instead of throwing up a REST API, you can create a web page
and share it with your friends.
And what's nice, this model was written
in TensorFlow 2.0 using Keras.
And we have a converter script that
will convert it into TensorFlow.js format
to run in the browser.
So following examples, I am not a JavaScript developer at all.
I can get through the examples.
And it's not too bad.
So it's possible to do.
But it's a really good opportunity
if you have friends that are good JavaScript developers.
This is a huge collaboration opportunity
where you can develop models, and your friends
can help you deploy them in the browser.
Also if you haven't seen it, you can also write models
from scratch in JavaScript.
And I just wanted to show you a couple of demos here.
I'm almost certainly going to accidentally unplug
this laptop.
But another super-convincing thing
about why you might want to do JavaScript in the browser,
this is a model called posenet.
This could be the end of the presentation.
This is a model called posenet.
It's running entirely client-side in the browser.
So nothing's being sent to a server.
And it's not meant for this many people.
But you can see that it's starting to recognize where
people are in the audience.
And what's cool--
I know this is all obvious for web developers.
But I'm not.
So this is all new to me.
For me to do this in Python would have been a nightmare.
Like I would have to be streaming data
from the video camera, sending it to a server,
classifying it server-side, sending the results back.
There's no way I'd be able to do that in real time like that.
Also for privacy reasons, that would not be cool.
But because this is running client-side in JavaScript,
we can do that.
And so immediately, like you can see all the things
that we can do on top of this.
There's other models like that too that are just
really compelling.
And then for people looking for applications,
so there is sentiment analysis.
But a model built right on top of that in the browser
is just a text classification.
But this is multiclass text classification.
And this is a toxicity detector.
It's basically a sentiment analysis model.
But the idea is you're given a sentence,
and you want to figure out if this is like a comment
that you might want to post on YouTube or something like this.
But you can build tools like this
that analyze text privately, quickly, client-side.
So you can imagine, for example, if you
had a job as like a Wikimedia moderator
and you wanted to take a look at article edits
to see if they were something you wanted to publish or not,
you might spend a lot of your time looking
for toxic comments.
With something like this, you could very quickly
preprocess-- you could immediately have code
that right in the web page highlights
the bad parts of the article.
And I need to move a little bit quicker.
So let me just point you to one demo and then I'll stop.
If you're new TensorFlow.js, by far the best demo--
there's two, and they're right here.
One is posenet, which you can get at that link.
There's also Pac-Man.
If you haven't seen Pac-Man, you can
control Pac-Man with your face.
You can train a model live in the browser
to do that, which is awesome.
And then flying through this, last comment,
then learning more.
If you're working in Colab and you're
used to using Keras, what you don't want to do
is import keras.
You want to say from tensorflow import keras.
And that will give you the version
of Keras in TensorFlow 2.0 that's
a superset of regular Keras.
This is only a problem in Colab because Keras is installed
there by default. If you ever see a message that
says using TensorFlow backend, you've
imported the wrong version of Keras.
And then last slide, here are four books that I'd recommend.
The very first book is about TensorFlow 2.0.
And this will give you low-level details.
It's great.
Only buy the second edition.
The first edition teaches TensorFlow 1.0,
which you do not need to learn how
to use to use TensorFlow 2.0.
The second book doesn't mention the word "TensorFlow" at all.
That's the Keras book by Francois Chollet.
But it's outstanding if you're new to deep learning.
It's a perfectly good place to start.
All the code from the second book
will also work inside TensorFlow 2.0
just by saying from tensorflow import keras.
Nothing else will change.
It's all completely good.
The next book is Keras in JavaScript,
so deep learning with JavaScript, which is great.
The fourth book is brand new.
It's by Pete Warden, TinyML.
So thanks very much.
And I'll stop there.
And I'll be around after for questions.
[APPLAUSE]
ASHLEY: Thank you so much, Josh.
Where can we find you?
JOSHUA GORDON: I'll be right outside.
ASHLEY: OK.
Awesome.
All right.
We're going to take a 10-minute break.
For those of you who are standing in the back,
there's some more seats over on this side of the room.
We'll be back in about 10 minutes.
See you soon.
Thanks again.
JOSHUA GORDON: Thanks a lot.