Placeholder Image

Subtitles section Play video

  • (bell rings)

  • - Hello, welcome to a new video series.

  • I'm very excited about this video series

  • 'cause I'm going to show you how to write some code.

  • I always show you how to write code, well, not always,

  • but typically I show you how to write code,

  • and then I run the code on this computer here.

  • And what I'm going to show you is how to write code here

  • and have it run somewhere else.

  • And this can be very useful for a lot of reasons.

  • One of the main reasons these days you might want to do that

  • is to run your code on a more powerful computer

  • that can do some kind of machine learning task

  • that might take all night,

  • and you can just be playing your

  • whatever other video game

  • and checking your email

  • while that's going

  • and get the results when it's done.

  • And I'm going to show you how to do this

  • with a particular platform called Spell.

  • And this video is sponsored by Spell.

  • So I'm going to do a couple things.

  • I'm going to talk about what is Spell.

  • I'm going to show you how to use it

  • and I'm going to run a simple example

  • with just one little Python script

  • that I make on this computer,

  • run it over there and see the results.

  • Then look in this video's description,

  • there will be links to two upcoming

  • live streams with two guests.

  • First, Nabil Hassein is going to come

  • and show you how to train an LSTM,

  • Long Short-Term Memory network.

  • It's a special kind of neural network model

  • that's useful for working with sequences,

  • and in particular you could generate text

  • as a sequence of characters with it.

  • So we're going to look at

  • how to train an LSTM model using spell.run,

  • and then have that model that you've trained

  • work with, say, the ML5 TensorFlow.js library,

  • an ML5 or a TensorFlow.js library.

  • Then Yining Shi,

  • who has previously appeared

  • on The Coding Train

  • in the Brick Breaker Tutorial,

  • she's going to come and show you

  • some stuff about working with images,

  • maybe we'll do style transfer,

  • maybe we'll do Pix 2 Pix, maybe both.

  • And so she'll come and show you how to train a model

  • for those concepts and execute that on Spell,

  • and then also work with that with the ML5.js library.

  • So you could check the video's description

  • for the times of both those live streams.

  • You can tune in live and ask your questions,

  • or simply watch if the live streams have already happened

  • and you're watching this video, just click the link,

  • and you'll be able to watch it right there.

  • Okay wonderful.

  • Alright so first,

  • let's talk about what is Spell.

  • So I've talked about the idea of

  • client side programming

  • and server side programming,

  • and this is sort of a similar thing

  • in the sense that we do have a client computer,

  • my laptop which is like The Coding Train,

  • then there is this service which is called Spell,

  • if I could spell Spell, that would be good,

  • which has lots and lots of computers

  • sitting there somewhere in the cloud.

  • They have computers with different CPUs, GPUs,

  • and I'll kind of get to that

  • as we go through all this material.

  • So typically if I'm talking about client server,

  • I might be saying you program your website,

  • then you upload your website to some server

  • and then other people can access your website.

  • This is different.

  • What we're talking about is

  • allowing these computers to act as

  • a cloud computing resource essentially

  • that you can write a script,

  • so maybe you have this Python script,

  • you set up your entire Python environment on your computer,

  • the Python script works with it and you can just say,

  • with typing in just the instruction,

  • spell run my script.py.

  • This will get sent to a particular server,

  • one that you've requested,

  • the kind of server you've requested,

  • you're not requesting a specific computer,

  • you're requesting the kind of computer, I want this GPU.

  • GPU by the way means Graphics Processing Unit,

  • and that typically is a powerful processor

  • that's really useful with machine learning models.

  • So you're going to request a particular kind of computer.

  • It's going to run it, it might finish,

  • it might generate some files,

  • like maybe it's going to say,

  • hey here's your poem,

  • or here's your image.

  • And then you could request

  • to ask for the results back.

  • And if that task takes hours or even days

  • which is not crazy at all in the world

  • of running machine learning models,

  • you can send it, close your computer,

  • check, go to the library,

  • get on the computer there,

  • go to the Spell website,

  • see what the progress is,

  • go to sleep, wake up, check the progress,

  • did I get a notification?

  • All sorts of things are possible when it's done.

  • So this is exactly what I want to show you.

  • And in case you're following along,

  • this is my kind of mental list of all the pieces,

  • a little outline of what I'm showing.

  • So really all I've done so far

  • is talk about what is Spell.

  • The next thing I want to do is install Spell.

  • Okay so the first thing that I want to do

  • is just go right to the spell.run website.

  • I'll link to it in this video's description,

  • but it's pretty easy to type it in right to your browser.

  • The next thing you're going to want to do is click Sign Up

  • and sign up for an account with your email address,

  • name and email.

  • I already have done that so I'm just going to log in.

  • You pause this video and sign up for an account

  • if you want to follow along.

  • And then I'm now logged in,

  • and you'll notice I'm getting this message,

  • Spell is offering a $100 GPU credit,

  • meaning you can use their GPU machines for free

  • up to what would normally cost $100.

  • You'll have to enter a credit card

  • on the billing page

  • to unlock that feature,

  • and afterwards there's a charge per hour

  • using the resources.

  • I will note that they have

  • a good notification system set up

  • so you can set it up to warn you

  • whenever you're down to this much,

  • so you should never see

  • any unexpected charges on your card.

  • Okay so once you're signed up,

  • the first thing you want to do is type this,

  • pip install spell.

  • That is now where do you type that?

  • So one thing I should mention

  • is there are couple prerequisites

  • for this following along this tutorial.

  • One is you're going to need to have a Python environment.

  • Most computers kind of come with Python installed

  • or you can install it,

  • and I'll link to a few resources

  • for installing Python.

  • I'm using a particular tool

  • in this video called virtualenv,

  • virtual environment,

  • that lets me create a Python environment

  • that I can kind of like turn on and turn off.

  • Useful.

  • You're also going to need to know something about Git,

  • how Git works in order to work with Spell.

  • And I will also link,

  • I mean I have a video tutorial series

  • about Git and GitHub as well

  • which I will link to you,

  • but those are prerequisites.

  • Once you have a Python environment set up,

  • you should be able to go to any terminal, console,

  • application that you're using and type in pip,

  • Python package something rather install spell.

  • So I'm going to go to, I'm using iTerm,

  • and you can see that I'm on the Desktop,

  • and this spell-demo, this is a message

  • that's showing to me that's part of virtual environment,

  • so I have my spell.demo virtual environment set up,

  • and so I can just write pip install spell,

  • and then I can wait, dot dot, dot dot, oh, that was so fast,

  • and it is installed.

  • And I know it's installed because now if I type spell,

  • we can see that it's, something is happening.

  • Normally if I would just type spell into the console,

  • into the terminal, it would say I don't know what that is.

  • Okay the next thing I want to do is actually log in.

  • So what I've done, actually let me say something,

  • what I've done is I've installed,

  • I just have to come over here just to write this down,

  • a command line interface.

  • So how do you control Spell?

  • There are two ways to control Spell.

  • One is through their web browser interface

  • which I will show you at some point.

  • Another is through a command line interface,

  • meaning you can type in commands

  • on the command line on your computer

  • to interface with Spell and control it.

  • And every command is going to start with spell.

  • And now I'm going to say spell login.

  • And I'm going to put in my username,

  • and I'm going to put in my password, sillyunicorn.

  • And there it is, it says, Hello, danshiff!

  • So I know that I am logged in now.

  • And I can also do things like, say, spell whoami,

  • just to check like who did I log in as?

  • And you can see, okay, there's my username,

  • you can all email me now at daniel@thecodingtrain.com,

  • and you can see when my account was created, June 1st 2001,

  • and my last log in was 10 seconds ago

  • 'cause I log in a lot apparently.

  • Alright so we have Spell working.

  • This is actually pretty easy.

  • Now I can do stuff.

  • So I could type spell,

  • anything that I can run on the command line,

  • for example I could type echo Choo! Choo!

  • Echo is a command

  • for the computer to echo back to me

  • whatever I type after echo.

  • So now if I want this command echo Choo! Choo!

  • to not run on my computer but to run on Spell,

  • all I have to do is say spell run echo Choo! Choo!

  • And here we go.

  • Now it's going to take longer.

  • Oh whoa, oh so first of all, ah,

  • this is very important,

  • I'm very glad that this happened,

  • because I meant to mention this.

  • Spell, as I was saying,

  • you need to know something about Git

  • which is version control software.

  • Spell is designed to work best when you run commands

  • from within a directory that is a Git repository.

  • At the moment I don't need to worry about that.

  • I'm just trying to say Choo! Choo!

  • So I'm just going to say continue anyway.

  • And then of course in a moment,

  • I'm going to run the commands from a Git repository.

  • So you could see it's casting spell number nine.

  • This is the ninth time

  • I've ever tried to run something on Spell.

  • A machine request is done, it's building,

  • it ran, it's that Choo! Choo!

  • It's saving, pushing,

  • total run time five seconds, complete.

  • So here's the thing.

  • This took almost six seconds to run,

  • whereas if I just did this,

  • it happens in milliseconds.

  • There is overhead in communicating with the server,

  • requesting a machine, sending the command,

  • running the command, getting the message back,

  • but all that overhead is totally worth it

  • when you have to run something

  • that you need a powerful GPU

  • that doesn't exist on your laptop

  • or that just is going to take,

  • it doesn't matter if you're adding five seconds

  • if it's going to take five hours.

  • So next step, let's actually have it run some code.

  • So I have already,

  • I'm in a directory called Coding-Train-Spell-Demo.

  • And this is code band.py.

  • It's a Python script.

  • I actually wrote a Python script sort of.

  • This is borrowed from a tutorial from Allison Parrish,

  • and I will link to that in the video's description.

  • But this, what this does is it loads a JSON file,

  • this is important because I want to show you

  • something about loading data,

  • it loads a JSON file with some data in it,

  • it picks some random stuff out,

  • and then it spits that stuff back out to a text file.

  • So for example, if I just were to say python band.p,

  • this is me running the Python script on this laptop,

  • what it does is it picks a set of random instruments

  • and shows them to you.

  • We run them through again,

  • it picks 10 random instruments and there you go.

  • It's my random band, the randoms.

  • And it's also in a file called theband.txt.

  • It wrote it out to a file.

  • Well, I'm not suggesting

  • that this particular code

  • is anything interesting.

  • This shows you the machine learning process

  • if you're going to use this for machine learning.

  • You don't have, you can run any code on Spell,

  • and there might be other things,

  • but the thing that I'm going to show you with,

  • that we're going to show you

  • in future live streams with guests,

  • LSTM, style transfer,

  • the idea is that

  • you're going to load in some data,

  • crunch the numbers for a very long time,

  • and then spit out a result,

  • and that's exactly what this is doing,

  • loading a JSON file,

  • running over sort of an algorithm,

  • and writing out a text file.

  • So maybe that output is going to be

  • a model file or an image or something else,

  • but all the same stuff will apply.

  • It's time to run span.py now on Spell, not locally.

  • So I'm going to ask the Spell server to run that Python script.

  • So all I have to do is type spell run,

  • and then the same as that command

  • that I would have typed anyway,

  • python band.py.

  • Now here's the thing,

  • this is going to work very fluidly

  • because I am in a directory

  • that happens to be a Git repository.

  • Spell knows how to transfer files back and forth using Git,

  • how to look at your last commit method

  • and have that be as something that's noted,

  • at what time did you run this.

  • So all of that,

  • if you have your script

  • inside of a Git repository,

  • things are going to work really nicely.

  • And you could see that I have my terminal

  • configured in such a way to show me information

  • about what branch I'm on and into my Git repository.

  • And I have a separate video

  • that kind of shows how I'm doing that

  • which I'll link to in the video's description.

  • Okay I'm going to press Enter and run this.

  • Okay so here we are.

  • It's casting spell number 11.

  • It was spell number nine before

  • because I did something

  • that I made a mistake with number 10.

  • It got edited out.

  • Okay, so you could see it's building it,

  • it's running it, it's done,

  • it's printed out the output,

  • saving, done, and run number 11 is complete.

  • Now that it's finished,

  • I want to be able to look at the results.

  • I want to see the results of whatever algorithm I ran.

  • I can see it here 'cause I had it kind of console logged it,

  • but I want that file, that file is not on my computer.

  • Remember I ran the code on Spell's computers,

  • so I need to get that file.

  • And there's two different ways you can do it.

  • So one way you can do it

  • is actually just by going to the browser.

  • So here I am logged in to Spell.

  • I'm going to go here and click Runs.

  • There's also this idea of Workspaces,

  • but Runs is where I can see

  • all the different things

  • that I have done recently,

  • and I want to find one number 11.

  • So I'm going to click on Runs,

  • and I'm going to click on run number 11 here.

  • And here we could see, look, this is actually what ran,

  • spell run, now I didn't type this in,

  • I didn't request a CPU machine,

  • the default is the basic CPU machine,

  • meaning no special fancy graphics processing units.

  • And this is actually free.

  • So the basic computer,

  • it's not going to do anything much better

  • than I could do already on my laptop but that is free.

  • And then you can see python band.py.

  • There's information that's sort of

  • log of what happened is here.

  • And then right here, look at this,

  • theband.txt, that was the output,

  • and I can just click Download,

  • and there it is.

  • I've already done this in practice.

  • And you can see there it is,

  • that's the results.

  • So in a way (bell rings) we're done.

  • Like I'm going to show you more things,

  • but this is the basic idea of Spell.

  • I wrote some code,

  • I have an environment configured on my computer,

  • I don't want to have to do,

  • I want to run it somewhere else

  • but I don't want to have to do

  • a lot of work setting up another environment,

  • Spell is going to be able to duplicate your environment

  • and run your code very easily by using Git

  • and other things that it knows how to integrate with.

  • So this is really wonderful

  • that I have this like web interface

  • and I can see what happened,

  • I can click through my runs,

  • I can see band.txt,

  • I can just download it really easily.

  • But most of the time

  • you're going to want to just

  • continue all of your workflow

  • through that command line interface,

  • and in fact you can download the file

  • and do a lot of other things

  • from the command line interface.

  • I'm going to show you a couple more commands,

  • and then I'm also going to show you

  • where you find all the documentation for all those commands.

  • I actually don't have these things memorized,

  • you shouldn't have them either memorized,

  • it's all about looking up

  • and reading through the documentation.

  • Okay so let me go back to the terminal.

  • So one thing I'm just in the Desktop now.

  • I don't have to be in my project folder

  • to interface with Spell.

  • If I'm running the code,

  • I want to be in the project folder,

  • but if I just want a file,

  • I could say something like

  • spell, cp for copy, runs,

  • and then I need to specify which run.

  • Now the number of the run is the most important thing, /11.

  • It was run number 11 I remember that.

  • And so it's going to say copied one file.

  • I can actually go to the Desktop now,

  • and we should see that there it is,

  • there's this file, the band.txt,

  • and there once again is the band

  • that came out of that particular run.

  • Let's just really quickly do this again

  • to see how everything changes.

  • I'm going to run it one more time.

  • (soft upbeat music)

  • Okay, run number 12 completed.

  • You could see it's a different band,

  • fiddle all the way to oboe.

  • So I could just go back to Desktop and say hey,

  • give me now the file from run 12,

  • and then I could just go to the Desktop,

  • and we can say, right, fiddle down to oboe,

  • the new file has come in and replaced it.

  • It's up to me to manage my file system

  • if I want to like rename the other one,

  • keep that one, that sort of thing.

  • But all of this has been saved,

  • I can just go quickly back

  • and grab the file for number 11 now if I wanted.

  • It's all there on the Spell server for you.

  • So the next thing I want to show you which is pretty typical

  • of working with a machine learning algorithm

  • is that your data might be a big set of images

  • or a lot of text files,

  • that's not going to be something

  • that's part of your Git repo.

  • Your code and your algorithm is all there,

  • but the actual data is something

  • that you want to treat separately.

  • And this is where you want to use

  • two important commands, right?

  • So before, I just had my instruments.JSON file

  • right there in the repo so my Python code could pull it up,

  • but if I want to just have my code running on Spell,

  • but try with different data files,

  • I'm going to need to upload the file,

  • so I'm going to use spell upload.

  • And then what I want to make sure is that,

  • that's uploading it to Spell,

  • but there's another concept here of mounting.

  • So if I have a whole set of uploads,

  • I want to mount a particular one,

  • meaning I want it to be available

  • for this particular run.

  • So let's look at how that works.

  • To show you how that works,

  • I've modified the code to have it

  • just pull from any text file.

  • So when I run, when I now say python band.py,

  • I have to give it a data file.

  • So I can say data.json.

  • And data.json is not part of my Git repository,

  • it's just a data file.

  • So what I want to do first is actually say

  • spell upload data.json.

  • Enter a name for the upload.

  • Ah, so actually it's important that I reference a name

  • because I want to like organize my stuff

  • in a way that I can remember later.

  • So now it's going to tell me let's call it,

  • let's just call it a demo,

  • maybe I want to call it,

  • actually let me call it like json_bands,

  • like band or band,

  • I'll call it bands or instruments.

  • (laughing)

  • Let's call it instruments.

  • Okay so now you can see it's uploading it

  • to uploading instruments.

  • Let's go to the Spell website.

  • And when I look at the website,

  • I can now go to Resources,

  • and we're going to see here,

  • look at this, under uploads, instruments,

  • there's my resource, data.json.

  • So now I need to make sure

  • this resource is available, right?

  • Like is it available just by default?

  • Now I can say spell run

  • python band.py data.json,

  • and it's going to run it.

  • Casting spell number 14,

  • building machine, running, running,

  • ah, it couldn't find the file.

  • See, it threw an error, open file name,

  • it couldn't find this file that I want, data.json.

  • To add that file, for it to be available

  • as part of the run,

  • it needs to be mounted.

  • And this can be done actually by just modifying

  • with an argument the spell run command.

  • So in other words I want to say spell run,

  • and all this time I should have been

  • putting the command in quotes

  • 'cause it's going to like keep it together as a unit.

  • So I can say python band.py data.json,

  • and then -m, and now the path to that,

  • and the path to that is what?

  • uploads, what did I call it?

  • instruments I think,

  • instruments/data.json.

  • So if I had a bunch of different instruments files,

  • I could run it just by picking the one

  • that I want to mount for this particular run.

  • So let's give this a try,

  • and this is a very common thing, this -m,

  • there's lots of other dash this,

  • dash that, dash this,

  • so that I can modify,

  • I can request a GPU or I can ask it do this.

  • So I'll look at that

  • when I show you the documentation.

  • Okay let's give it a try.

  • Oh we're in number 15, this is so exciting.

  • Machine requested, mounting,

  • run is mounting, run is running, there we go.

  • So we can see it worked.

  • Now it used that file.

  • Oh that's wonderful.

  • I've kind of finished.

  • I mean there's so much more,

  • and as I said there's going to be

  • two upcoming live streams.

  • We're showing practical machine learning examples

  • that you could stay tune for or watch

  • if you're watching this sometime in the future

  • when those have already happened.

  • But I do want to mention a couple other things.

  • So let's actually let's run this on a GPU, why not?

  • Let's make use of a powerful GPU.

  • I've got $100 in credit.

  • So let's go here and see,

  • how do you find out all the things you can do?

  • The main thing you want to go to

  • is here under Documentation.

  • So I'm going to click Documentation.

  • There's a Quickstart guide.

  • Essentially the Quickstart guide

  • is what is in this video.

  • No more as succinct fact fashion.

  • There's some Core Concepts,

  • there's Guides for particular examples

  • like style transfer already,

  • translating text, recognizing numbers.

  • I encourage you to look at those.

  • But what I'm really concerned with right now

  • is this Command Line Interface Documentation.

  • That's what I really want to look at.

  • So I'm going to go there and you can see

  • this is all the stuff that I've been showing you,

  • spell whoami, spell upload, spell stop,

  • spell ps, this is a really useful one.

  • So spell ps display all user runs and their details.

  • Because one thing by the way you should know,

  • you might sort of think like, oh I'm going to just run this,

  • and then like, oh no, I didn't mean to do that,

  • let me quickly hit Control + Z.

  • Well, guess what I just did?

  • I stopped viewing logs with Control + Z.

  • That didn't actually stopped the process.

  • If I want to stop the process,

  • I've got to actually use probably spell.stop.

  • And spell.stop, I can stop a particular RUN_ID

  • if it's like taking forever and it has a mistake in it.

  • That one probably already finished 'cause I was talking,

  • but I could say spell -ps.

  • Oh sorry, just spell ps, no dash.

  • And now this is everything I've ever done basically.

  • You could see back like when I first was doing this thing,

  • I was running some test spell test.

  • It's always telling me the machine type, what the thing was.

  • This is a little bit awkward to look at because of,

  • it'll look really nice on your computer

  • especially when you have like a smaller font

  • and like it's all nice.

  • It'll look something nice like this (laughs)

  • but I have like a big font so you can see it,

  • and I don't know I'm like, my computer has gone crazy.

  • But this is the kind of thing that you want to look how to do.

  • So the one thing that you're going to want to really see is

  • I'm looking for, oh, spell.run,

  • and now the arguments for it.

  • So we could see that -m was for mounting,

  • and then also, ah, this is really important, --pip,

  • this is if a certain dependency that,

  • my project was just like a simple one

  • so it didn't really need any dependency,

  • but this is important so that when Spell runs it,

  • it uses the same Python packages that you're running.

  • And by the way Spell also supports other environments.

  • I'm using Python 'cause the future examples

  • and a lot of machine learning examples run through Python.

  • But you know me, I don't really know Python,

  • and maybe I'll come back and show you with JavaScript thing,

  • with node, all these other environment,

  • other environments are supported on Spell as well,

  • but here's the important one, --machine-type or -t.

  • And you can see these are the different machine types.

  • Probably the further you get to the right

  • is kind of the bigger the one.

  • And you're going to want to look through the Spell website

  • for more documentation on each one of these.

  • But let's just pick one for fun.

  • Let's try a v100.

  • I feel like that's kind of like

  • a reasonable GPU that's not too crazy.

  • To run it on the GPU,

  • I'm going to do exactly what I did before,

  • spell run python band.py

  • data.json -m,

  • argument for mounting

  • that data.json file,

  • and then also I need another argument, -t.

  • - t is for --machine,

  • it's to select the machine,

  • I'm going to pick v100

  • which is a machine that has a GPU

  • that I'm excited to try.

  • And now I'm going to hit Enter,

  • and we're going to run it on the GPU.

  • Okay, let's go.

  • Okay so here's the thing.

  • I am running this code now

  • on a machine that has a powerful GPU,

  • but I'm not actually making use

  • of any of the code libraries

  • that would actually make use of the GPU,

  • so technically this is really going to be the same thing.

  • And it already finished.

  • But when we get to the next tutorials

  • that are going to come in the future,

  • looking at LSTMs, and style transfer, Pix 2 Pix,

  • those will really make use of a machine with a CPU.

  • Okay it finished, it took seven seconds, there we go.

  • Alright so that's going to wrap things up.

  • I hope that you enjoyed this tutorial.

  • I hope you learned a little bit about

  • what it means to use a cloud computing resource

  • to run code that you're writing

  • on this computer or somewhere else,

  • you could do it in Python

  • and in other environments as well

  • that are supported by Spell.

  • Thank you so much to Spell

  • for the opportunity to do these tutorials.

  • I really appreciate it.

  • I've had a lot of fun

  • playing around with the platform

  • and hope to use it again in future projects.

  • So go over to the website, spell.run, give it a try.

  • Ask your questions in the Comment.

  • Look in the video's description

  • for two links to the upcoming live streams,

  • or if those have already happened,

  • those links will be replaced with the archive

  • of those live streams as well.

  • So I can't wait to see what you make,

  • how it works for you,

  • what kind of things I forgot to talk about

  • that I get to answer in the Comments.

  • And I'll see you again someday on The Coding Train.

  • (whistling)

  • Choo Choo!

  • (upbeat music)

(bell rings)

Subtitles and vocabulary

Click the word to look it up Click the word to find further inforamtion about it