Subtitles section Play video Print subtitles SPEAKER: It's a pleasure and an honor to welcome our next speakers, Paige Bailey and Brennan Saeta, who are going to talk to us about Swift for TensorFlow, a next generation machine learning platform that leverages innovation. So please help me in welcoming Paige and Brennan. [APPLAUSE] PAIGE BAILEY: Good morning, everyone. So thank you for coming to join us during the lunchtime. We very much appreciate you-- you taking the time to come to TensorFlow world and also to learn a little bit more about Swift. I'm Paige. I'm the product manager for Swift for TF and-- BRENNAN SAETA: I'm Brennan. I'm tech lead manager for Swift for TensorFlow. PAIGE BAILEY: Excellent. And so, first, we want to get a feel for the room. How many of you have used Swift before? Excellent. BRENNAN SAETA: Awesome. PAIGE BAILEY: And how many of you attended Paris and Tim and their session, their workshop on-- earlier this week? Few hands, cool. So this is a great place to get started if you've never used Swift before. And it's also a wonderful place to learn how Swift is getting more and more functionality added specifically for machine learning and for deep learning. So it's an exciting time. Do you want to get started, Brennan? BRENNAN SAETA: Yeah, let's do it. PAIGE BAILEY: Perfect. BRENNAN SAETA: So today we want to bring you a little bit different kind of a presentation since a lot of you actually know Swift, at least to some degree. We wanted to really dig one level deeper and really explore Swift for TensorFlow from its foundations. So before we get into the meat of the talk, I wanted to take a moment to acknowledge a few very important folks. So number one is-- this work would not be possible without the rest of the Swift for TensorFlow team that really does a huge amount of work to make this platform awesome, so huge thank you to them. But also not just the core team itself, But there are a number of other teams that we collaborate with at Google-- from the JAX team to the rest of TensorFlow and beyond. And so we really are collaborating together and building on each other's work to make something awesome. So thank you to everyone, both at Google and then, of course, our awesome open source contributors beyond. We have a lot of contributors that are really making a huge difference to the project. And, finally, for those of you who are familiar with Fast.ai, some of the slides may be familiar. So Jeremy Howard and Chris Lattner co-presented a little bit Swift for TensorFlow earlier this year. And I encourage you all to check that out. So, to begin with, I like to start with a question-- why Swift for TensorFlow? PAIGE BAILEY: And I could give my answer. BRENNAN SAETA: Let's hear it. PAIGE BAILEY: Awesome. So I've been using Python for a long time. I really love the flexibility and often the user experience that you get using the language. It feels very understandable. You read the code on the page and it reads almost like a plain sentence and it also has a vibrant ecosystem of data science tooling. But something that I was always really frustrated about whenever I was trying to deploy Python models into production is that often you have to do a lot of rewrites. Part of the-- part of Python's benefits don't necessarily translate so well to deployment scenarios. So Swift is, to me, at least, a happy way to get a lot of the great benefits of Python, but also the benefits of a typed language. BRENNAN SAETA: That's right. And so what we are going for with Swift for TensorFlow is an infinitely hackable platform that gets you all the way from research, into production, and, most importantly, that everything is customizable. Because although there's a lot of flexibility that you need to do research, you need just the same amount of flexibility and capability, often sometimes slightly different, to really make a high quality production application. PAIGE BAILEY: Yeah. BRENNAN SAETA: And that's what we're really excited about with Swift for TensorFlow, and especially with the Swift platform, how we can do interesting things to optimize our productivity. So let's take a moment to ask the question what is Swift? PAIGE BAILEY: And Swift is a programming language that was created by Chris Lattner pretty recently. BRENNAN SAETA: Yeah. PAIGE BAILEY: One, he was spending some time over at Apple. Since then, Chris has come to join us and worked at Google. You'll be hearing a lot about one of Chris's other projects later this week. MLIR, I believe, is in the keynote tomorrow. But Swift is a programming language that's focused on that extensibility component that Brennan just mentioned as well as readability, usability, and production readiness. BRENNAN SAETA: That's right. So this is Swift for TensorFlow from foundation. So let's really dig in and let's really go from the foundations. So I think an important question to understand what is Swift at a deeper level is to understand what is a compiler. And so as you program computers, there's two fundamental things that are immovable and that like to think about the world in their own particular ways that are sometimes incompatible. You've got humans, on one hand, that want to think about the world in a particular way and you've got hardware which also thinks about the world in its own particular way. And a programming language is basically an intermediate point in between these two immovable, fundamental truths of technology. And so programming languages are designed for humans to express ideas, but they're also designed to be executed on hardware. And they're different points in that space that you can pick and Swift is just one of such points. But we think it's a really interesting one and we'll talk about why in a couple of minutes. PAIGE BAILEY: Right, so one of the benefits of having a typed language is that a lot of the responsibility that might be pushed on you if you're using a language like Python is suddenly abstracted away. So I don't have to care so much about baking performance optimizations into my code immediately, the compiler can just do it for me. Another great thing is that if you look at the screen right now, you can see an example of a few commands in Swift. It looks very, very similar to Python. There's some lets sprinkled in, there are some-- a few indications of types, but other than that, pretty straightforward to understand. If you're only experienced using typed languages as something like C++, this probably looks a great deal friendlier. BRENNAN SAETA: That's right. And Swift is a very modern and powerful programming language. So even though Swift is statically typed, you don't actually see very many types in advanced Swift code. Here we're defining a dictionary numbers from strings to lists of integers, the compiler can just figure this out for you. One of the principles of Swift is that you have a very helpful compiler. And you can define higher order operations, reduces a common pattern from functional programming across a sequence of elements. And you can then call this x.sum, mapped over all the values in the dictionary. And voila, you've now used a number of higher order functions together without having to write types in a very seamless and nice manner. So what we're trying to get across is that Swift is almost as high level as Python. So this is actually a slide that I am borrowing from Chris and Jeremy's Fast AI lectures. And at the top, you have for Swift for TensorFlow model. And at the bottom, you actually have a Python model. And if you look at the code, if you squint a little bit, they're almost exactly the same thing, right? You've got a few more self dots on the bottom. And you've got some more curly braces on top. But they're basically the same thing. So Swift is just as high level, almost, as Python. PAIGE BAILEY: We also have fewer lines of code in that example, is that right? BRENNAN SAETA: Well, it depends on how you count the curly braces. So little bit of here, little bit of there. But what's really important about Swift is that it's designed to be fast, right? It's practically in the name. So who here has actually looked at Assembly in the last six months? Ah, fewer hands this time. Well, bet you didn't expect to be looking at Assembly today. So for those of you who don't know, the Godbolt Compiler Explorer is really neat. You can plug-in some source code on the left-hand side. And it will give you the assembly on the right-hand side. And here we have Swift. We're calling a higher order function reduce on a generic type-- so it's actually an array, in this case, an array of integers-- passing in the plus function, reducing over everything. And if you look at the loop that is going on here, so you've got your function prelude. You do a couple checks to make sure that everything is reasonable. You actually just have these five instructions that form the main body of your loop. You do the add. You check to make sure that you don't have any overflows. You then increment one to get to the next value in the array. You compare to make sure you've not reached the end of the array. And if you haven't, then you do your loop again. This is like the most efficient set of instructions you could possibly do to sum across an array. And so even though you have all these higher order things, you're not even writing types, the Swift compiler can helpfully optimize it all the way down for you. Now actually, for those of you who really know about Performance and Assembly, a really helpful compiler will actually vectorize this operation for you. And so Swift, by default, is safe. So when you use Plus, you're getting safe integer add that checks for overflows. But if you do Ampersand plus-- the Ampersand plus function, this says just give me-- 2 is compliment, don't bother checking for overflows-- and the Swift compiler will hopefully vectorize your operation for you to make it run extremely quickly. So Swift is designed to be safe, to be high level, but of course, also fast. So programming languages are all about atoms and composition. And so the atoms in Python are C code that implements Python objects. And the composition is an interpreter that lets you combine these calls in very interesting ways, right, like Python array, Python dictionary. They're very, very carefully implemented in Python to be as quick as possible in C. And then you get to build your whole universe on top of it. C++, however, takes a different tack. There is no C underneath C++, right? It's just C++. So the atoms in C++ are int, float, C arrays, and pointers. You got a few other atoms as well. And you could develop your own structures in C++, like structs classes, std::complex, st::string, std::vector. So that's a C++, and C++'s view of the world. But as you think about C++, what's the real difference between string and array? They both are sequences of things, right? And so programming language has actually evolved since when C++ was initially created. And we actually can do more interesting things today. And so Swift builds on top of a lot of this programming language evolution and design. So Swift is really syntactic sugar for LLVM. Remember getting back to that picture? You've got humans. And you've got hardware. Programming languages are in the middle. You've got text editors to have humans interface with a programming language. You've got compilers to interact with-- from program [INAUDIBLE] to the hardware. And so why don't you define your language in terms of the compiler. You can just get rid of a lot of extra intermediate things. And by doing so you can make a programming language that is more powerful. And so the primitives in Swift are LLVM instructions. And the composition are structs and classes just like C++. But in Swift, instead of float and int being defined by the language, they're defined in the standard library. So that you can define your own int and float. Now this may seem like an esoteric question that that isn't terribly important, but, well, as it turns out, the definition of float is something that's in question these days. For those of you who are not familiar with TPUs, but TPUs define their own version of floating point format. They define bfloat16, which is short for the brain floating point format that was developed as part of the Google Brain research efforts. And we found that the bfloat16 format is much better for both efficiency in the hardware and for training neural networks. And so Swift as a language is especially interesting for us because it lets us define a bfloat16 type that works exactly like int and float in the rest of the language. So what does that actually look like? It can't be, is it super weird? What's this black magic? Well, again, this is Swift from the foundation. So let's understand, what is the implementation of float? You can totally check it out on GitHub. But here's sort of what it looks like, right. You've got a struct float because your composition is structs. And you define the value type, which is defined in terms of the LLVM primitives, this builtin.FPIEEE32. Plus equals is just a function, just a normal function that's defined in Swift. And it's implemented using the LLVM primitives. PAIGE BAILEY: And bfloat16, as Brennan mentioned, is incredibly important to us specifically for TPUs. But we're seeing even more kind of exotic data formats and sort of requirements for floats for other specialized hardware. So we anticipate this will become even more important in the future. And it's a place that we're really excited to explore more with Swift. BRENNAN SAETA: That's right. And so even though int and float, our elements in the standard library, they're just as fast straight as we looked at the assembly instructions before. But that also means that Swift as a language is super flexible, right. So you can define extensions on int. And you can define the is/odd property of int just under Jupyter Notebook. You can define an extension on Boolean symbol, which returns a Unicode character string of thumbs up or thumbs down. And you can then compose all of these exactly as you'd expect to print out these silly true/falses, and emojis, and whatnot. But Swift, again, even though you're using all these high level operations, is just as fast as C. And so these notebooks, which I encourage you to check out, from Jeremy Howard and Chris Lattner's [INAUDIBLE] course, actually show you how you can build a matmul in Swift that is just as fast as C, perfect performance parody. So that's a little bit about Swift. Let's talk about deep learning. PAIGE BAILEY: Yes. BRENNAN SAETA: OK, no, let's actually have a brief detour before we get there. PAIGE BAILEY: Well, it's also very important to understand the mathematics involved in implementing a deep learning, a deep learning project. You have to understand a little bit about linear algebra, and a little bit about math in order to effectively architecture experiments. So for this next slide, we have a quiz. And I promise it isn't too scary. But somebody in the room, shout out what you think B will print based on this slide. AUDIENCE: [INAUDIBLE]. PAIGE BAILEY: I think I heard a 3. So that is correct. BRENNAN SAETA: 3, very good. PAIGE BAILEY: Excellent. BRENNAN SAETA: This is value semantics. This is how integer is work in Python. It is exactly what you'd expect. If a is 3, b is equal to a. That means b is 3. No matter what you do with a, 3 is still 3. B is still 3. We're all good. PAIGE BAILEY: Yeah, so let's take a look at this next slide. We have a Python list. And this, crazily enough, whenever you add 4 would print 3 and 4 for the values of b. So this is not value semantics. And this offers a great many frustrations for developers, as they're attempting to architect their experiments. BRENNAN SAETA: That's right. Math operates on value semantics. And Swift is designed to be a very, very fast programming language, but at the same time really push forward value semantics. And so in Swift, arrays actually do have value semantics, and behave exactly as you'd expect. So here is a copy/paste from a terminal, where b stays the same. And just to show you that I don't have anything up my sleeve, right, a is actually 3 and 4. But see in Swift, not just arrays, but so do dictionaries, and so do all other higher level types, many high level types that you build can have value semantics as well. And this has important ramifications for how to write neural networks, and do machine learning, and automatic differentiation, which leads us to our next topic. PAIGE BAILEY: Excellent, automatic differentiation is one of the sort of product-defining features of Swift. And it's one of the things that we're most excited about upstreaming to the Swift programming language. So we showed in one of our slides previously, I believe, a function that was prefaced with at differentiable. And this is sort of functionality that allows you to differentiate not just sort of aspects of machine learning models, but literally any function in Swift you can take the gradient of. So you want to show an example, Brennan? BRENNAN SAETA: You know what, it's better to show, not tell. So let's see. Here we are in a Google Colab. So this is a hosted Jupyter Notebook. And here we define my function. That just takes two doubles, does some math operations, and returns on the double. And so to make this function differentiable, you just annotate it at differentiable. And once you've done that and run the cell, the compiler has now constructed the forward pass, but also the backwards pass for this function automatically. And so to then figure out what the derivative of that function is, you just call the gradient function. So you do the gradient at 0.5 of my function. And this gets you the partial derivative with respect to a. Of course, you may want to take the partial derivative with respect to both a and b. And so that is exactly as you'd expect. So you have now both points. When you're training neural networks, you sometimes also want like the value of the function in addition to the partial derivatives at a particular point. And so to do that, instead of calling gradient, just call value with gradient. And this will return a result that has, it's just a tuple. So in Swift, you've got named tuples, optionally. So you've got the value in the gradient. And you can manipulate them exactly as you'd expect. PAIGE BAILEY: Excellent, and if you're coming from Python land, or you stuck around for the presentation a little bit earlier, this would be a sort of taking the equivalent place of gradient tape and TensorFlow. So instead of having to create a gradient tape and collecting your variables on that, you can just go ahead and do the annotations for your functions. BRENNAN SAETA: That's right. And the thing that we're really excited about is, again, this is language integrated. So these aren't just on Tensor of double that are 1D or 0D Tensors. These are just regular, built-in int and flow and double and beyond. So that's how you write a function, a custom function and take the derivative of it. But we really want to build this next generation platform that allows for maximum flexibility. So you need to be able write your own types that can also be differentiable. So let's see how to do that. So here we define a point in 2D space. You've got x and y. And we just mark it as differentiable. We can then use, we defined some properties on it, right. So the function dot and the [INAUDIBLE] helper function. And you can then compute the gradients of this dot product, this point dotted with itself. And so there you go. You then get the tangent of this point at that space. Now often it's a good idea to mark your functions as a differential because this helps the compiler catch errors for you. And it's also a great form of documentation. And here we see an example of the compiler catching an error for us. So Paige, this says, error, can only differentiate functions with results that conform to differentiable. But int does not conform to differentiable. What does that mean? PAIGE BAILEY: Because all of the values are discrete. Like, you can't add a little incremental step change. BRENNAN SAETA: That's exactly right. So integers, right, the derivatives are about infinitesimally small steps. And integers don't allow you to take infinitesimally small steps, right? You've got 0 and 1. There's nothing in between. And so this function is not differentiable. And the compiler is very helpful in telling you, hey, you can't do this. And this is why, instead of silently giving you a wrong answer. So if we remove the annotation from that function, compiler caught an error for us. Great, we can keep going on. But let's say we want to take things to the next level. What if we want to define an additional property, say like the magnitude property, which is defined by a vector from the origin to that particular point? PAIGE BAILEY: And it looks like you're importing Glibc here, which means, do you get all of the mathematics operations that are already within the C-- OK cool. BRENNAN SAETA: That's right. So this is a great excuse for us to show the Swift C interop. So in Swift you can just import any arbitrary C header and use the symbols to find in it, no boilerplate, no wrappers, no nothing, just go straight for it. So here we're going to import Glibc. And this is the square root function defined by C standard library. So we can then use that to define point, the magnitude, sorry, extension on point, which is the square root of x-squared plus y-squared. So we run this. And we get magnitude. And dang it. We've got another compiler. What's this saying? PAIGE BAILEY: So read through, cannot differentiate functions that have not been marked differentiable and that are defined in other files. So it looks like you have to go and change this in another file. BRENNAN SAETA: That's right. So square root is defined by the C compiler. And the C compiler hasn't been taught how to take derivatives. So this is a great excuse to show you how you can write your own custom derivatives for arbitrary functions. So if you recall that the derivative of the square root of x is 1 over 2 times the square root of x, we can define my square root-- which underneath the hood, just call C. And we can define the derivative right here with this function closure, in line closure here. So we run that. And that compiles successfully. Go here, change the square root to my square root. That runs. And voila, we're off to the races. We've now defined a differentiable magnitude function on point. We can then use this function inside other differentiable functions exactly way you'd expect, everything fully composes. So here we define this silly function, taking points and doubles, right. You can mix arbitrary differentiable types. And we're going to define our point, take the gradient of it. We can get our tangent vector, which has the partial derivatives with respect to both x and y at that particular point. PAIGE BAILEY: Excellent. And I so I love that you're able to use all of the great code that's been written to support mathematics. But I also really love that with Swift you get C and C++ interop. And as Brennan mentioned, you can import any header, use it as is or extend it to meet your use case. So if you're working in an enterprise and you have a massive existing C++ code base, like well, Google. And then it's really helpful to be able to not have to rewrite the entire wheel. Like you can just import things that people have created over time, use it. And then everything as part of a Swift program would compile down to a .so files. So you can deploy it on any platform, Windows, Linux, Mac, iOS, Android, embedded devices, even. BRENNAN SAETA: That's right. PAIGE BAILEY: It really is as sort of the most multi-platform approach that I've seen for machine learning. BRENNAN SAETA: That's right. So I think this is enough. There's actually more power in this Swift for TensorFlow automatic differentiation system. And I encourage you to check out more details later. But let's move on to actually talking about neural networks. PAIGE BAILEY: Cool. BRENNAN SAETA: And here we go. So, tell us about writing neural networks in Swift. PAIGE BAILEY: Writing neural networks in Swift, we just saw an example of it. It's pretty straightforward. You only have about five lines of code. It looks very similar to Keras. And if you open up a Colab notebook-- BRENNAN SAETA: There you go. PAIGE BAILEY: --you can see an example of it here. BRENNAN SAETA: Great, so why don't you walk us through this right here. PAIGE BAILEY: Right, so you create a model by adding some layers. Here we see a convolutional layer, Conv2D, a pooling layer. We flatten it. And then we call a function with active, or we preface the callAsFunction without differentiable. And that gets us our gradients. BRENNAN SAETA: Yep. So here we define a sequential model just in Swift. So now let's actually use this as we would to actually train this model on some data. So here we define some random training data, just because for convenience purposes. We instantiate the model and an optimizer for it. And then we're going to run this for 10 training steps. So we do this in just a few lines of code. We're actually handwriting out the training loop up here. But here we get the value with gradient, which gets us our loss and our gradient. And we then print out the loss and use the optimizer to update our model along the gradient. And so just like that, we're training a model. And you can see the loss value is decreasing. PAIGE BAILEY: Yep, and if you're not a fan of writing out as many custom steps as we have expressed here, Jeremy Howard did a great collaboration with Chris and with the Swift for TensorFlow team, where he re-implemented Fast AI on top of Swift. So if you're looking for higher level constructs, you just want things that are out of the box, I highly recommend checking out those notebooks as well. BRENNAN SAETA: That's right. But one of the common workflows that anyone who's doing machine learning is they have their first version of the model. They train it. And they figure how to make it better. Right, that's one of the great promises of machine learning, is you can keep incrementing and optimizing. So let's see how we can optimize this model to make it a little bit better. So you may be following a lot of research. And you may hear that skip connections, or residual networks are a really good idea. And so in the next 30 seconds, we're going to convert this network to be a residual network with a skip connection. You ready? All right, here we go. So what we need to do is we need to define our second dense layer that we're going to use as part of the skip connection. So we're going to dense float. And I can't type. And here you can see the autocomplete, right. We care a lot about developer productivity. Autocomplete, that's helping to fill in the initialize or parameters for us. Fix up a little bit of the shapes, great, now we're all good. So we've now added the extra layer. Now we need to add in that skip connection. So let tmp is equal to input. And we need to capture the value after the flatten step. Let temp 2 is equal to dense of tmp. So this is going to be the first part. And now we do our skip connection, dense of tmp plus tmp 2. Voila, we have now added an additional layer and added a skip connection to our model right here in a Jupyter notebook. So we now recompile that to make sure that that's working well. Looks like it is. And so we're going to reinstantiate our model, because it's now a different model. And we're going to retrain it for 10 steps. And looks like we actually got slightly lower, or similar loss values. So this is good. But we can, of course, do better. But this is sort of an example workflow how easy it is to customize models using Swift for TensorFlow. But for those of you who are, for those of you who are just applying models to standard data sets, customizing the [INAUDIBLE] architecture is the bread and butter workflow. But we're finding more and more that both research and production, you sometimes need to be a little bit more advanced. And so Swift for TensorFlow, the platform is infinite hackability, infinite customizability. Let's actually see how you can [INAUDIBLE] your own custom layer also in a Jupyter Notebook. All right, so let's say if we're a researcher. And we wanted to define a custom layer that we add two bias terms to our dense layer instead of just one. Well, in about 10 seconds we're going to write this. All right, ready? Here we go. Boom, all right. Great, it worked this time. All right, so here we define our weights, our weight tensor, and our first bias term, and our second bias term. We have our constructor. And then finally, the actual forward function, or callAsFunction, where you take the matrix multiplication of the input in the weights and you add in the two bias terms. So there we go. This should compile. Ha ha ha, and once it does-- there we go-- we can then use it in our new model. So instead of a dense layer, we use a double bias dense layer. We can then compile this, instantiate the model, and whoops, this doesn't quite work. Again, here you see the Swift compiler being extremely helpful. You have a quick typo here. Instead of label. it should be labels. And it just tells you this is what you should be doing. And you're off to the races. So we fix that up. And we can run the training steps. And-- PAIGE BAILEY: The loss is worse. BRENNAN SAETA: Dang it. We have a little bit of bug in our implementation. And so exercise for the audience to figure out where that bug was, but we'll let you think about that on your own time. Anyway, that's a brief tour about how to use Swift for TensorFlow as an ML practitioner, applying and fitting against data sets and models. Excellent, so we'll head back to the slides. Oh actually, I forgot. So one other thing, right, so we showed how to write a custom model. We showed how to write a custom layer. But if you're doing research into different optimizers, you also going to be able to customize that. And so optimizer in Swift are just pure Swift code. And so here we have the salient bits of the stochastic gradient descent optimizer that we were using just now. And so it's just 10 lines of code to implement momentum, nesterov and other advanced features for a stochastic gradient descent optimizer. So we've been spending a lot of time really thinking about what are the right APIs, and what are the right ways to factor machine learning to make it easy to use, powerful, and flexible. And we're pretty excited by where we are today. But we're always looking, of course, to improve. So if you're interested in pushing the limits of the state-of-the-art, here are a bunch of other things that we'd love to have you try and give us feedback, and really to help us improve our APIs. PAIGE BAILEY: So we mentioned before that one of the biggest sort of wins for Python is that an entire ecosystem of data science products have been built around it. And the good news is with Swift, you don't have to give any of that up. So I think Brennan has a demo coming up showing that you can use tooling like Matplotlib, or NumPy, or SciPy, or any of your favorite Python packages directly from Swift, almost identical to the same way that you would run it in a Python Jupyter notebook. Is that correct? BRENNAN SAETA: That's right. PAIGE BAILEY: Yeah. BRENNAN SAETA: So let's take a look. Ah, my secret notes. All right, there we go. So here is a Jupyter Notebook, or actually Google Colab, where we import TensorFlow. And we also import Python. And once we've imported Python, we can use this Python object to import arbitrary Python libraries with no wrappers at all. So here we're going to import Matplotlib, the pyplot actually type from there, and NumPy. And we're gonna assign these to plot and NP. And after that, you can write Swift code-- now again, this is Swift code that looks almost exactly like Python. So here we're gonna call np.linspace, assign that to x. And we're going to plot x, and sine of x, and x and cosine of x. And just to show you that this really does work, we'll run it. And voila, you're using pyplot from that Matplotlib and NumPy right from Swift. In short, you don't have to lose all the great Python ecosystem while taking advantage of all the capabilities that Swift offers. Now sure, what about a more wackier library? I mean, these are kind of wacky. But really one of the important parts of the Python data science ecosystem are all the machine learning libraries. And one of my favorites is actually OpenAI Gym. So this is a Python library that has a bunch of reinforcement learning environments that you can use to train and evaluate neural networks as you're doing research in reinforcement learning. And so it's a Python library. But as it turns out, with Swift for TensorFlow, you can just use it. So here we can use Python to import Gym. We're going to define some hyper-parameters. Here are going to define our neural network using Swift for TensorFlow. so just like you saw before, we've got two dense layers. We have some code to interact with the Python environment, and just including defining the next batch. But here we use that Gym library to make the cartpole environment. We instantiate our neural network and our optimizer for it. And then we're going to train it for as many steps as we need. Again, we see that the Swift compiler is actually really helpful. It's telling us that on line 24, instead of one hot labels, we probably mean probabilities. And if I type that right, we should now see this training. So here we are using a neural network, defined in Swift for TensorFlow, training in the Python OpenAI Gym library, totally seamlessly, right in a Jupyter notebook. In fact, it solves it. And of course, you can take the results that you've just, the intermediate values that you've computed while you've been training this model, and plot it using that Matplotlib right in the notebook. So it all totally works, all back and forth. PAIGE BAILEY: And as a machine learning engineer, I think an important thing to note with this is that your workflow doesn't change, right? The only sort of incremental step that you need to add is importing Python and assigning [INAUDIBLE].. And that's about it. BRENNAN SAETA: Yeah, excellent. So that's a number of demos and notebooks that you can see, sort of the state of the world today. But let's talk a little bit more about what's been going on in the future, the directions that we're heading. So we have C interop, and we've been working on C++ interop as well. So here's sort of an example of what we have. So on the left-hand, the left column, you define a C++ type, header file, et cetera. And on the right-hand side, if you define this in example.h, you can then import it in Swift as import example. And you can then call these methods directly on it. We have a number of features already supported, including instantiated templates. Because as it turns out, C++ is not quite the simplest language. But we're seeing that it's available to be useful for a number of things already today. One of the mantras is infinite flexibility and an infinite hackability. And so we've talked about writing custom networks, custom layers, custom optimizers, but what about custom kernels as well? As it turns out, we've been working on that. And so this is some of the work that is preliminary work on the team. But here you can define 1D average pooling in pure Swift. And this actually runs on both CPUs and GPUs in the prototype that we put together. Now we're still iterating and getting the fundamentals of the design right. But this is sort of an example or the direction that we're trying to go with Swift for TensorFlow with infinite hackability, so that you're not limited to what's in the box for you. The whole world's your oyster. You get to program at any level of the stack doing whatever you need to do. You're never limited when you're building on the Swift for TensorFlow platform. We've had a lot of amazing things happen as part of the community that's been growing around Swift for TensorFlow. So we have in development a model garden with a variety of supervised learning models, and some unsupervised learning models. But one of the collaborations that I've been super excited about has been with the DeepMind team. That's part of Alphabet. They recently released OpenSpiel, which is a collection of reinforcement environments and algorithms. And a subset of those have been imported to Swift. And so that's a great place if you want to check it out, if you're really interested in reinforcement learning. So if you've been interested, how do you get started, Paige? PAIGE BAILEY: Well, you can get started writing Google Colab, so the examples that Brennan showed. We also have a variety of tutorials available at tensorflow.org/swift. And I also encourage you to look at Jeremy Howard's Swift AI implementation of Fast AI that's available on GitHub. BRENNAN SAETA: I did want to take a moment to really highlight some of the community projects that's been going on. So we talked about OpenSpiel. But Anthony Pantanios has done amazing work with Swift RL, another reinforcement learning library, and Swift Atari Learning Environment, which I think are really great. There's SwiftPlot, swiftML that have happened as a result of the Google Summer of Code, projects that have happened this summer that Paige has organized that I think are really fantastic. SwiftAI came out of the Fast AI collaboration. We have Tensors Fitting Perfectly, where Adam [INAUDIBLE] join the Swift for TensorFlow team this past summer and did some really great research. Everything is open source. We'd love to have you join our community. We have open design meetings every Friday at 9 AM Pacific, or 1600 UTC. And we'd love to have you all join in and really help shape this platform together with us. We recently released 0.5 with a number of improvements. And stay tuned for our next release is coming up. And with that, I'd like to thank you all for listening to us today. PAIGE BAILEY: Excellent. [APPLAUSE] BRENNAN SAETA: We will be around here and in the hallways for the rest of the day. We have a-- for those of you here in person at the conference, we will be hanging out at the Ask a TensorFlower Cafe in the expo hall. Come with your questions, feedback. We'd love to hear what you're thinking and see you at our open design meetings. Thanks.
B1 swift brennan paige bailey paige bailey compiler Swift for TensorFlow (TF World '19) 2 0 林宜悉 posted on 2020/03/31 More Share Save Report Video vocabulary