learning frameworks out there right now to see what works best

the deep-learning space is exploding with frameworks right now ,it's like every

single week some major tech company decides to open source their own deep

learning library and that's not including the dozens of deep learning

frameworks being released every single week on github by cowboy developers how

*how many layers you have?*

let's start off with scikit-learn scikit it was made to provide an easy-to-use

interface for developers to use off-the-shelf general-purpose machine

learning algorithms for both supervised and unsupervised learning , scikit provides

functions that like you apply classic machine learning algorithms like support

vector machines logistic regressions and k nearest neighbor very easily but the

one type of machine learning algorithm he doesn't let you implement is a neural

network it doesn't provide GPU support either which is what helps neural

networks scale , since like two months ago

pretty much every single general-purpose algorithm that psyche learned

implemented has since been implemented in tensorflow sidekick you just got

LEARNED , there's also caffe which was basically the first mainstream

production grade deep learning library started in 2014, the cafe isn't

very flexible think of a neural network as a computational graph in cafe each

note is considered a layer so if you want new layer types you define the full

forward backward and gradient updates these layers are building blocks that

are unnecessarily big there's an endless list of them that you can pick from

intensive flow each note is considered a tensor operation like matrix add,

matrix multiply or convolution and a layer can be defined as a composition of

those operations so tender flows building blocks are smaller which allows

for more modularity ,cafe also requires a lot of unnecessary verbosity if you want

to support both the CPU and the GPU you need to implement extra functions for

each and you have to define your model using a plain text editor that is just

ghetto model should be defined programmatically because it's better for

modularity between different components , also caffe main architect now works on

the tensorflow team we're all out of caffe

speaking of modularity let's talk about keras : Keras has been the go-to

source to get started with deep learning for a while because it provides a very

high level API to build deep learning models Kera sits on top of the other

deep learning libraries like Theano and tensorflow it uses an

object-oriented design so everything is considered an object be that layers

models optimizers and all the parameters of the model can be access object

properties like model.layers[3].output will give you the output tensor

for the third layer in the model and model.layers[3].weights is a list

of symbolic weight tensors this is a cleaner interface as opposed to the

functional approach of making layers function that create weights when being

called great documentation it's all gucci yes i'm bringing that back but

because it's so general-purpose it lacks on the side of performance Keras has been

known to have performance issues when used with a tensorflow backend since

it's not really optimized for it but it does work pretty well with the Theano

backend , the two frameworks that are neck-and-neck right now in the race to

be the best library for both research and Industry are tensorflow and Theano

Theano currently outperforms tensorflow on a single GPU potential flow

outperforms piano for parallel execution across multiple gpus , Theano has got more

documentation because it's been around for a while and it's got native windows

support which tensorflow doesn't yet dammit windows in terms of syntax let's

just take a look at some code to see some differences

we're going to compare two scripts in tensorflow and beyond

they both do the same thing initializing phony data and then learn the line of

best fit for that data is it can predict future data points let's look at the

first step in both tensorflow and Theano for generating the data pretty much the

same way using numpy arrays so there's not really a difference there

let's look at the model initialization parts ,this is the basic "y=mx+b"

formula in tensorflow it doesn't require any special treatment of the x

and y variables

they're just they're natively but Theano we have to specifically say that

the variables are symbolic inputs to the function the tensorflow syntax for

defining the B&W variables is cleaner than we implement our gradient descent

function which is what helps us learn or trying to minimize the mean squared

error over time which is what makes our model more accurate as we train the

syntax for defining what we're minimizing is pretty similar then we

look at the actual optimizer which helps us do that will notice a difference in

syntax again

the flow just gives you access to a bunch of optimizers right out-of-the-box

things like gradient descent or Adam Theano makes you do this from scratch

then we have our training function which is again more verbose see the trend here

theano so far is making us implement more code than tensorflow so it seems to

give us more fine-grained control but at the cost of readability finally we'll

get to the actual training part itself they look pretty identical but

tensorflow methodology of encapsulating the computational graph feels conceptually

cleaner than pianos tensorflow is just growing so fast that it seems inevitable

that whatever feature it lacks right now because of how new it is it will gain

very rapidly

I mean just look at the amount of activity happening in the tension flow

repo versus the Theano repo on get up right now and while keras serves as an

easy use wrapper around different libraries it's not optimized for

tensorflow a better alternative you want to learn and get started

easily with deep learning is TF learn which is basically keras but optimized

for tensorflow so to sum things up the best library to use for research is

tensorflow the world-class researchers at both open AI and deep mine are now

using it for production best library to use is still tensorflow is it scaled

better across multiple GPUs and its closest competitor Theano .Lastly for

learning the best library to use is TF learn which is a high-level wrapper

around tensorflow that lets you get started really easily also shout out to

rahul do for being able to generate an upbeat midnight file badass of the

week please subscribe for more programming videos for now I've got to

go worship tensorflow some more so thanks for watching