Introducing TensorFlow 2.0 and its high-level APIs (TF Dev Summit '19)

Subtitles section Play video

[MUSIC PLAYING]
MARTIN WICKE: I'm Martin Wicke.
I'm the engineer lead for a TensorFlow 2,
and I am going to, unsurprisingly, talk
about TensorFlow 2.
TensorFlow has been extremely successful.
And inside of Google, outside of Google,
it has grown into a vibrant community
that really when we started it we were not able to imagine.
And our users are building these ingenious, clever, elegant
things everyday from art and music,
from what you just heard about medieval manuscripts, science,
and medicine.
And the things people created were unexpected and beautiful.
And we learned a lot from that.
And we learned a lot from how people did it.
And TensorFlow flow has enabled all this creativity
and really jumpstarted this whole AI democratization,
which is great.
But it has been a little bit hard to use sometimes
And we know.
At sometimes it's been a little painful.
So using sessions wasn't the most natural thing
to do when coming from regular Python.
And the TensorFlow API has grown over time
and got a little bit cluttered, a little bit confusing.
In the end, you can do everything with TensorFlow,
but it wasn't always clear what's the best
way to do it with TensorFlow.
And so we've learned a lot since we started this,
and we realized that you need rapid prototyping,
you need easier debugging, there's a lot of clutter.
So we're fixing this.
We're addressing these issues with a new version
of TensorFlow--
today, releasing an alpha of TensorFlow 2.
Many of you have participated in the design reviews
that went into this for all the features
that we have implemented.
Those of you who are really living
on the bleeding edge of development
have probably played with a nightly.
You have installed it.
You have tested it.
You have found issues.
You have fixed issues.
So thank you very much for your help.
You can now install the alpha release of TensorFlow 2
using just pip install --pre tenserflow.
And you can go and play with it.
So what has changed?
When we create TensorFlow 2, really the biggest focus
is on usability.
We've adopted Keras as the high level API for TensorFlow.
We've integrated it very deeply with TensorFlow.
Keras gives you a clear path of how to develop models,
how to deployment models.
It has a [INAUDIBLE] API that makes people productive,
and we know that it gets you started quickly.
TensorFlow also includes a complete implementation
of Keras, of course.
But we went further.
We have extended Keras so that you
can use all of the advanced features in TensorFlow
directly from tf.keras.
The other big change in TensorFlow 2
is that it is Eager execution by default. Traditional 1.x
TensorFlow used a declarative style,
and it was often a little bit dissonant with the surrounding
Python.
And in TensorFlow 2, TensorFlow behaves just
like the surrounding code.
So if you add up two numbers, you
get a result back immediately, which is amazing, really.
[LAUGHTER]
But you still get all the benefits
of working with A graph--
robust program serializating, easy distribution
of programs, distributed computation,
optimizations on the graph.
All of that stuff is not going away, it's just getting easier.
So TensorFlow 2 is a major release,
which means that we have the ability to clean it up.
And we did-- like, a lot.
So we had a lot of duplicate functionality
that we have removed.
We've consolidated the API.
We've organized the API.
We've made sure that the API looks and feels consistent.
And this is not only about TensorFlow itself,
this is about the whole ecosystem of tools that
has grown around TensorFlow.
We've spent a lot of effort to try and align
all of these interfaces.
And we've defined exchange formats
that'll allow you to move all throughout this ecosystem
without hitting any barriers.
We have removed a lot of stuff from TensorFlow,
but that doesn't mean that it's not--
it has gotten more flexible, not less.
It retains and expands on the flexibility
we had that enabled all of this development.
And we've created a more complete low level
API that now exposes all of the internally use ops
to the users via the robs module.
We provide inheritable interfaces
for all of the crucial concepts in TensorFlow,
like variables and checkpoints.
And this allows framework authors
to build on top of TensorFlow while maintaining
interoperability with the rest of the ecosystem.
And you'll hear a lot of these examples later.
There'll be talk about TF agents later today.
There'll be a talk about Sonnet by DeepMind tomorrow.
So you'll see examples of how this works in practice.
Now, the real question for you is, of course, what
do I have to do to be a part of this great new era
of TensorFlow?
And we know it's always hard to upgrade to anything.
And that's especially true for major version upgrades.
At Google, we will now start the process
of converting one of the largest codebases in the world
to TensorFlow 2.
And while we're doing that, we'll
be writing a lot of migration guides.
we have started that already, and some of them
are online and will provide a lot of best practices for you.
And if we can do it at Google, you can probably do it too.
So we're giving you and us a lot of tools
to make this transition easier.
And first of all, as part of TensorFlow 2,
we're shipping a compatibility module--
we call it tf.compat.v1--
which contains all of the 1.x API.
So if you rely on a specific deprecated function,
you can still use it, and you can still find it there,
except for t tf.contrib, which is not
going to be included at all.
But we've created a community-supported alternative
that support your use case if you rely on something in there.
We're also publishing a script which will automatically
update your code so it runs on TensorFlow 2.
Let me show you how that works.
Let's say I want to convert this program, which
is a simple example that we have online of a simple language
model trained on Shakespeare.
So what I do is I simply run tf_upgrade_v2,
which is a utility that's included with any installation
of TensorFlow 2.
It tells me what it did.
And in this case, there's really just a handful of things
that it changed.
And you can see, if you look at what changed,
there's some functions that it renamed.
The reorganization of the API leads to renamed functions.
So multinomial was actually the wrong name for this function.
So it renamed it to random.categorical.
That's something the script will do for you,
so you don't have to worry too much about it.
A lot of functions had arguments renamed, their order changed,
or some arguments added or deleted.
As far as possible, the script will make those changes
for you.
And then, if all else fails, sometimes there
isn't really a perfect equivalent in TensorFlow 2
to a symbol that existed in TensorFlow 1
And then, we'll use the compatibility module
that I talked about earlier.
If there is no perfect replacement,
we'll use that in order to make sure
that your code still works as expected after the conversion.
So it's pretty conservative.
So for instance, the AdamOptimizer
is a very subtle behavior change that mostly probably
won't affect you.
But just in case it might, we will convert it
to compat.v1 AdamOptimizer.
Once the conversion is complete and there were no errors,
then you can run the new program in TensorFlow 2,
and it'll work.
That's the idea.
So it should be pretty easy for you.
Hopefully, you won't get trouble.
Note that this automatic conversion,
it will fix it so it works, but it won't fix your style
to the new TensorFlow 2 style.
That cannot be done automatically.
And there's a lot to learn about this process,
so if you want to know more, check out the talk by Anna
and Tomer tomorrow at 10:30 about 2.0
and porting your models.
That'll be worthwhile.
Of course, as we go through this process at Google,
we'll also publish a lot more documentation.
But this talk is the best start.
All right.
So let me give you an idea about the timeline of all of this.
We have cut 2.0 alpha today or yesterday--
it took some time building it.
We're now working on implementing some missing
features that we already know about.
We're converting libraries.
We're converting Google.
There's lots of testing, lots of optimization
that's going to happen over the next couple of months.
We're targeting to have a release candidate in spring.
Spring is a sort of flexible concept--
[LAUGHTER]
But in spring.
After we have this release candidate,
it still has to go through release testing and integration
testing, so I expect that to take a little bit longer than a
for our regular release chains.
But that's when you can see an RC, and then,
eventually, a final.
So if you want to follow along with the process, please do so.
All of this is tracked online.
It's all on the GitHub TensorFlow 2 project tracker.
So I would just go and look at that
to stay up to date if you were looking
for a particular feature or just want to know what's going on.
If you file any bugs, those end up there as well so everybody
knows what's going on.
So TensorFlow 2 really wouldn't be possible without you.
That has been the case already.
But in the future, even more so, we'll need you to test this.
We'll need you to tell us what works and what doesn't.
So please install the alpha today.
We're extremely excited to see what you can create with us.
So please go, install the alpha, check out the docs.
They're at tensorflow.org/r2.0.
And with that, I think you should probably
hear about what it's like to be working with TensorFlow 2.
We'll have plenty of content over the next couple of days
to tell you exactly about all the different aspects of it.
But we'll start with Karmel, who will speak about high level
APIs in TensorFlow 2.0.
Thank you
[APPLAUSE]
KARMEL ALLISON: Hi.
My name is Karmel Allison, and I'm an engineering manager
for TensorFlow.
Martin just told you about TensorFlow 2.0
and how to get there. and I'm going
to tell you a little bit about what we're bringing
in our high level API and what you can
expect to find when you arrive.
But first, what do I mean when I say high level APIs?
There are many common pieces and routines
in building machine learning models, just
like building houses.
And with our high level APIs, we bring your tools
to help you more easily and reproducibly build your models
and scale them.
As Martin mentioned, a couple of years
ago, TensorFlow adopted Keras as one of the high level
APIs we offered.
Keras is, at its heart, a specification
for model building.
It works with multiple machine learning frameworks,
and it provides a shared language
for defining layers, models, losses, optimizers, and so on.
We implemented a version of Keras
that has been optimized for TensorFlow
inside of core TensorFlow, and we
called it TF Keras, one of our high level APIs.
Raise your hand if you used TF Keras already?
OK, a lot of you.
That's good, because this next slide will seem familiar.
In its simplest form, Keras is just that, simple.
It was built from the ground up to be
Pythonic and easy to learn.
And as such, it has been instrumental in inviting people
in to the machine learning world.
The code here represents an entire model definition
and training loop, meaning it's really easy for you
to design and modify your model architectures without needing
to write tons of boilerplate.
At the same time, by relying on inheritance and interfaces,
Keras is extremely flexible and customizable,
which is critical for machine learning applications.
Here we have a subclass model.
I can define arbitrary model architectures,
modify what happens at each training step,
and even change the whole training loop if I want to.
Which is to say, Keras is simple and effective.
Anyone can figure out how to use it.
But we had a problem, TF Keras was built for smaller models.
And many machine learning problems,
including the ones we see internally at Google,
need to operate at a much larger scale.
And so we have estimators.
Estimators were built from the ground up to distribution
and scale with fault tolerance across hundreds of machines,
no questions asked.
This here is the much-loved wide and deep model,
a workhorse of the machine learning
world that took many years of research to define
but is available as a built-in for training and deployment.
Which is to say, estimators are powerful machines.
But you've told us that there's a steep learning curve,
and it's not always easy to figure out
which parts to connect where.
And we learned a lot in the past two years
building out estimator in TF Keras
And we've reached the point that we
don't think you should have to choose
between a simple API and a scalable API.
We want a higher level API that takes
you all the way from [INAUDIBLE] to planet scale,
no questions asked.
So in TensorFlow 2.0, we are standardizing on the Keras API
for building layers and models.
but we are bringing all the power of estimators
into TF Keras, so that you can move from prototype
to distributed training to production serving in one go.
OK, so brass tacks--
how are we doing this?
Well, this is a TF Keras model definition in TensorFlow 1.13.
And this is the same model definition in 2.0.
[LAUGHTER]
Some of you may notice the code is exactly the same.
OK, so what actually is different?
Well, we've done a lot of work to integrate Keras
with all of the other features that TensorFlow 2.0 brings
to the table.
For example, in 1.13 this built a graph-based model that
ran a session under the hood.
In 2.0, the same model definition
will run in Eager mode without any modification.
This lets you take advantage of everything Eager can do for us.
Here we see that our data set pipeline behaves just
like a numpy array, easy to debug
and flows natively into our Keras model.
But at the same time, data sets have
been optimized for performance so that you
can iterate over and train with a data set
with minimal performance overhead.
We're able to achieve this performance
with data sets in Keras by taking advantage of graphs,
even in an Eager context.
Eager makes debugging and prototyping easy,
and all the while Keras builds an Eager-friendly function,
under the hood, that tracks your model,
and the TensorFlow runtime then takes
care of optimizing for performance and scaling.
That said, you can also explicitly run
your model step-by-step in Eager mode with the flag run_eagerly.
Even though for this example, it might be overkill,
you can see that run_eagerly allows
you to use Python control flow and Eager mode for debugging
while you're prototyping your model.
Another big change coming in 2.0 is
that we have consolidated many APIs across TensorFlow
under the Keras heading, reducing duplicative classes
and making it easier to know what you should use and when.
We now have one set of optimizers.
These optimizers work across TensorFlow
in and out of eager mode, on one machine or distributed.
You can retrieve and set hyperparamters
like normal Python attributes, and these optimizers
fully serializable.
The weights and configuration can
be saved both in the TensorFlow checkpoint
format and the backend diagnostic Keras format.
We have one set of metrics that encompasses all the former TF
metrics and Keras metrics and allow for easy subclassing
if you want even more.
Losses, similarly, have been consolidated into a single set
with a number of frequently used built-ins and an easily
customizable interface if you so choose.
And we have one set of layers, built
in the style of Keras layers.
Which means that they are class-based
and fully configurable.
There are a great number of built in layers, including
all of the ones that come with the Keras API specification.
One set of layers that we took particular care with where
the RNN layers in TensorFlow.
Raise your hand if you've used RNN layers in TensorFlow?
OK, these slides are for you.
[LAUGHTER]
In TensorFlow V1 we had several different versions
of LSTm and GRUs, and you had to know ahead
of time what device you were optimizing for in order
to get peak performance with cuDNN kernels.
In 2.0, there is one version of the LSTM
and one version of the GRU layer.
And they select the write operation
for the available device at runtime
so that you don't have to.
This code here runs the cuDNN kernel
if you have GPUs available, but also
falls back to CPU ops if you don't have GPUs available.
In addition to all of those, if you
need layers that are not included in the built-in set,
TF Keras provides an API that is easy to subclass and customize
so that you can innovate on top of the existing layers.
This, in fact, is how the community repository
TensorFlow add-ons operates.
They provide specialized and particularly complex layers--
metrics, optimizers, and so on--
to the TensorFlow community by building
on top of the Keras base classes.
All right.
So we streamlined API--
that's a start.
The next thing we did was integrate Keras
across all of the tools and capabilities
that TensorFlow has.
One of the tools we found critical to the development
of estimators was easily configurable structured data
passing with feature columns.
In TensorFlow 2.0 you can use feature columns
to parse your data and feed it directly
into downstream Keras layers.
These feature columns work both with Keras and estimators,
so you can mix and match to create reusable data input
pipelines.
OK.
So you have data in your model, and you're ready to train.
One of the most loved tools we have for running your training
jobs is TensorBoard.
And I'm pleased to say in TensorFlow 2.0,
TensorBoard integration with Keras is as simple as one line.
Here we add a TensorBoard callback to our model
when training.
And this gets us both our training progress--
here we see accuracy and loss--
and a conceptual graph representing the model we
built layer-by-layer.
As an added bonus, this same callback even
includes full profiling of your model,
so you can better understand your model's performance
and device placement, and you can more readily find
ways to minimize bottlenecks.
Speaking of performance, one of the exciting pieces
we're bringing to TensorFlow 2.0 is
the tf.distribute.Strategy API.
In TensorFlow 2.0 we will have a set
of built in strategies for distributing your training
workflows that work natively with Keras.
These APIs have been designed to be easy to use,
to have great out-of-the-box performance,
and to be versatile enough to handle many different
distribution architectures and devices.
So how do they work?
Well, with Keras you can add and switch distribution strategies
with only a few lines of code.
Here we are distributing the same model you just
saw across multiple GPUs by defining the model network,
compiling, and fitting within the distribution strategy
scope.
Because we've integrated distribution strategies
throughout Keras in TensorFlow, you get a lot
with these few lines of code.
Data will be prefetched to GPUs batch-by-batch,
variables will be mirrored in sync
across all available devices using allreduce,
and we see greater than 90% over multiple GPUs.
This means you can scale your model up without changing code
and without losing any of the conveniences of Keras.
Once you've trained your model, you're
likely going to want to package it for use with production
systems, mobile phones, or other programming languages,
Keras models can now export directly
to saved_model, the serialization
format that works across the TensorFlow ecosystem.
This functionality is experimental
while we iron out a few API details, but today,
and going into 2.0, you can easily
export your models for use with TF Serving, TF Lite, and more.
You can also easily reload these models back
into Python to continue training and using
in your normal workflow.
So that's where we are today in the alpha.
But we're obviously not done yet,
so let me tell you a little bit about what's coming up
in the next few months.
We talked about distributing over multi-GPU,
but that's just the start.
Using the same model code, we can swap out our strategy
and scale up to multiple nodes.
Here we take the same Keras model
we just saw and distribute it across multiple machines,
all working in sync using collective ops
to train your model faster than ever before.
Multi-worker mirrored strategy currently supports ring
and nickel for allreduce, which allows us to achieve great
out-of-the-box performance.
This API is still being developed,
but you can try it out today in the nightlies
if you are interested in a TensorFlow-native solution
to synchronous multi worker training.
And we are very excited to say that using Keras
and distribute strategies you will also
be able to use the same code to distribute your model on TPUs.
You'll have to wait for the next TensorFlow release for this
to actually work on TPUs.
But when it's released, this strategy
will still work on Google Cloud TPUs
and with Colab, which can now access TPUs directly.
And as we approach the final 2.0 release,
we will continue bringing scalability into Keras.
We will be enabling distribution with parameter service
strategy, which is the multi-node asynchronous
training that estimator does today.
We will also be exposing canned estimator like wide and deep
directly from the Keras API for those of you
who want a higher level higher level API.
And we will be adding support for partitioned variables
across many machines for some of the largest scale models,
like those we run at Google.
And that's just a sneak peak.
So please do check out the 2.0 preview
and try out TF Keras if you haven't already.
If you have more questions, you'll
be hearing more throughout the day about Keras and high level
API and how they integrate across TensorFlow.
And we also have a workshop tomorrow that Martin mentioned.
[MUSIC PLAYING]