MALCOLM REYNOLDS: My name is Malcolm

and I'm a research engineer at DeepMind.

TAMARA NORMAN: Hi, my name is Tamara

and I'm a software engineer at DeepMind.

MALCOLM REYNOLDS: So we're here today to talk about Sonnet

and hopefully answer some questions for you,

so what is Sonnet, why DeepMind has found it useful,

and why it might be useful to you,

and then talk a bit about our plans

for the future of the library.

So Sonnet is DeepMind's library for constructing

neural networks in TensorFlow.

We've been working on it since 2016

and it's been in open source for the last couple of years.

And you can try the TensorFlow 1.0 version with this command

line here.

But you might be thinking, sounds interesting.

Why does this exist?

And that is a very legitimate question.

So to answer that, we have to take a little trip

down memory lane.

So 2014, TensorFlow is in development at Brain,

but there's no open-source release yet.

And unrelatedly I guess, DeepMind is acquired by Google.

So a little bit of code archeology here.

So does anybody recognize what system this is written in?

Hands up if it looks vaguely familiar.

OK, not too many hands.

So I censored some of this just to make it slightly harder.

So now it may be clearer that this is actually

Torch7 code, or Lua Torch.

And specifically, we're defining a module here.

That's what the red arrow is pointing to.

So DeepMind was an entirely Torch7 operation,

at least in the research side when I joined.

And it's a really great piece of software.

If you're interested in the sort of history of deep learning

abstractions, you know, the abstractions in Torch

really go back to stuff from the '90s, like Lush.

So fast forward a few years, we have

dozens of research projects going on, hundreds of people

involved, and being able to kind of quickly share code

between the projects was really critical.

So somebody comes up with a new RNN,

everyone else can try that out with minimal fuss

and no changing of the code.

And we thought we had a pretty good workflow

and we were pretty happy with it,

and then we decided to just get rid of it

all and start from scratch.

Seems questionable.

But the reason is we were transitioning to TensorFlow.

And obviously, there's many great reasons

to use TensorFlow.

We've been hearing about them for the last few days.

Top of the list for us was better alignment

and collaboration with our colleagues at Google.

It was designed for distributed training from day one, which

Torch really wasn't, and that was becoming a problem

in some of our research.

And we wanted to really take the best advantage of TPUs,

and obviously, TensorFlow is the best way to use them.

And we knew from initial assessments

this was going to be flexible enough to build

whatever we need on top of.

But we weren't actually starting from scratch.

We had some stuff we really wanted to preserve.

And it's not that we want the same APIs as Lua Torch.

That would not have been the right decision.

But we wanted the same engineering philosophy.

So what we mean by this is like hopefully we're

going to state through this talk.

So first of all, kind of composable

modules using object-oriented programming.

Now it's definitely totally valid

to have a kind of purely functional library.

But we found that when you start having lots of variable reuse

RNNs, those kind of features, objects really

made a lot more sense.

So we definitely want to keep that.

We wanted to decouple the model definition

from how it's actually trained.

And this is really key, and I will comment on this

again shortly.

Hackability is really crucial.

We don't think we can anticipate everything our research

scientists might want, like years or even months ahead

of time.

But what we can do is have code that they

are able to dig down, hopefully not too far,

make the changes they need to, maybe by forking the module,

and we're kind of comfortable with that,

and then continue with their research.

And it's important to emphasize we're really

optimizing for research.

You've heard a lot about a lot of great solutions

for kind of moving things into production.

And that's really important, but for DeepMind research, that

really happens very rarely.

So we prefer to optimize the libraries

based on research progress, next paper, that kind of thing.

And overall, we're looking for a standard interface

with really minimal assumptions that doesn't prevent you

from doing anything.

So composability.

What do I mean by this?

Hopefully, anyone who's used an LSTM would be like,

yes, you can implement this out of some Linear modules,

and they contain the variables and LSTM

is a thing around all of that.

But you can go many levels up from there.

So the differentiable neural computer,

which was the first project I worked on at DeepMind,

you know, that's a module that contains

an LSTM plus some other stuff.

And we worked on that and got the paper out,

and then the code is sitting there,

and then another team could come along and be like, hey,

we have some system which requires an RNN.

It's currently an LSTM, but maybe we'll

just try this instead.

And you know, it's been used in many places,

but I think the most significant maybe was the Capture the Flag

work, which had some really cool results, specifically

with the DNC controller.

And you know, if they could reuse the code,

like they didn't need to ask us about how it worked.

They didn't even need to read the code.

They're just like, this is an RNN that conforms to some API,

we'll just drop it in, and it works.

Orthogonality to the training setup dimension.

So I did a quick survey.

We have roughly two setups for unsupervised learning,

four for reinforcement learning, and many more setups

for single projects.

And to be clear, what I mean here,

this is kind of everything that goes on above the model.

Like how do you feed it the data,

how do you launch experiments, monitor them, like

what configurability is there for the researchers.

All this kind of stuff is what I'm

referring to as training setup.

And this might seem like a ton of, you know,

repeated effort and duplicated code,

but we don't actually view this as a problem.

These different setups exist for different research goals.

And trying to push everything into a single training setup,

even just for RL, it's been tried.

We don't think it works.

So we're kind of happy with lots of different setups

which kind of coexist.

And the reason it's not a problem,

apart from the serving different research goals,

is that we can reuse Sonnet modules between all of them.

And that means we're not like redoing the DNC from scratch.

So I've talked a bunch about what we've already done

with Sonnet and TensorFlow 1.

Tamara is going to talk a bit about the future.

TAMARA NORMAN: So yeah.

Cheers, Malcolm.

So what will Sonnet look like in TF 2?

DeepMind are really excited for TF 2,

and are planning once again to invest

and changing our workflow in adopting TF 2.

Eager execution has already been widely tried by researchers,

and they really enjoy the improved flexibility,

debugging, and simplicity that come from this.

And they're looking forward to the benefits

of this being part of TF 2.

But one thing we've noticed is that Sonnet 1 was

based on features, like variable scripts quite heavily,

and these are going away.

This isn't something that we're worried about.

We're really excited and follow TF

into this much more natural and Pythonic world.

So how do we plan on making Sonnet in this new world?

So we've built tf.Module as the base of Sonnet 2.

This is a stateful container, which provides both variable

and model tracking.

It's been designed by engineers at DeepMind

in collaboration between many individuals at DeepMind

and Brain, learning lessons from both Sonnet 1 and Lua Torch.

It's now been upstreamed into TensorFlow.

And you can try it out in the alpha

that's been released over the course of Dev Summit.

It will soon form the basis of many higher level

components in TF, including things like TF Keras.

So what about modules?

Modules can have multiple forward methods,

addressing a limitation that we found within Sonnet 1.

They can be nested arbitrarily, allowing

for the creation of complex models,

like the [INAUDIBLE] that was talked about earlier.

And they have automatic name scoping, which

allows for easy debugging.

So what are we trying to do with Sonnet 2?

We're aiming to create a library which

makes very few assumptions about what users want to do,

both with their networks and their training loops.

It's been designed in close collaboration with researchers.

And we think it's a lightweight, simple abstraction that's

as close to the maths as it can be.

So what the features of Sonnet?

Sonnet has the ability to have multiple forward methods,

something enabled by tf.Module.

But what does that really mean?

There will be an easy way to create [? excessive ?] methods

on an object that can have access

to the same state and variables.

An example of where this is useful

will be in a few slides time.

One thing that we've found that's really

crucial with Sonnet 1 is that it's really easy to compose it

with other systems.

Sonnet 1 works out of the box with Replicator,

which is our internal solution to distributed training,

even though these libraries were never designed

with each other in mind.

They don't even have a dependency on each other.

They both just use TensorFlow under the hood.

It's up to users to define how they

want to compose their system together and decide where they

want to put things like that [INAUDIBLE]

to ensure that the gradients are the same across all

the devices.

We plan for the same philosophy to be implemented in Sonnet 2.

Replicator has been merged with Distribution Strategy,

so you'll all be able to try this out.

And you can find more details on Replicator on the DeepMind blog

post.

So what do we want to do within Sonnet?

We also want to continue to provide the flexibility

and composability that we've already

heard about that they have in v1, such as things that models

can be composed in arbitrary ways,

and that no training loop is provided.

We still don't want to aim to predict

where the research will go.

So what does the code look like?

This is the linear layer.

It's pretty much the implementation, just

without some of the comments.

And one thing we've been really excited by

is how simple an implementation we've