Ifyou'reinterestedinlearningmoreaboutwhatitis, I knowthatthere's a fewothershereaswell.
Sothere's quite a fewcommunityofpeoplesharingknowledgethroughconferencesandlookpostsandworkshopsandthingslikethat.
But I'm notheretotalkaboutthat.
I'm heretotalkaboutmachinelearning, sousually I startbecause I don't knowwhatyouknowandwhatyoudon't know.
So I usuallystartwithanexampletotrytoputeverybodyonthesamelevelaboutwhatmachinelearningis.
Butyesterday I thinkMoneyCardid a veryniceintroduction, talkingabouthowtorecognizeourpicturesofcatsanddogs, and I thoughtthatwasreallysimpleandwelldone.
So I kindofremovesomeofmyslides.
And I'm justgonnabrieflysay, ifyouwerenothereorifyou, youknow, don't rememberthatit's basicallygivingtheabilitytocomputerstofindpatternsindatawithoutbeingexplicitlyprogrammed.
Soif I justtakebacktheexamplefromMonicayesterday, thefirsttimethat a computersees a pictureoff a cat.
Itdoesn't reallyknowwhatitis.
It's neverseenonebefore.
Butyouknow, bygivingitmillionsofsamplesofdataoffwhat a catlookslikein a doglookslikeusingalgorithmsandmachinelearningmodels, thecomputerisabletounderstand, tofindpatternsinwhatmakes a cat a caton a pictureandwhatmakes a dog a dog.
Soat a highlevel, thisisbasicallythat.
Soatthecoreofitismachinelearningmodels, andtobuild a model, youneedalgorithms.
Andusingtheselabelsandfeaturesorusinganalgorithm, youwouldtrytounderstandthecorrelationbetweenthefeaturesandthepriceforthecharacteristicandthepriceofthehousetogenerate a modelthatcanthenendupbeing a mathematicalrepresentationontheoutcomeofyourtraining.
Sousingeverythingthatyouknowaboutthehousesusealgorithmsthatendupbeinglike a functionthattakesnewparametersof a newhouse.
It's neverseenbefore, andit's abletogenerateahpredictionoff a pricethatthehouseshouldgofor, basedonallthehousesthatit's seenonthemarket.
Soit's importanttoknowthatsupervisedlearningneedslabelsandfeaturestogenerate a predictivemodel.
Andthen, ifwelookat a typeup, anothertypeoflearningtheunsupervisedlearning, it's theabilitytocreatepredictionsbasedonlyon a setoffeatures.
Sosounsupervisedlearningismorearoundclusteringratherthangetting a particularansweratoverquestion.
Sointhis, too.
Sonowthatwetalkedaboutthetypesoflearning I talkedabout, youknow, problemsthatyou'repredictingthepriceof a houseorcustomerbehavior.
Butitmightnotbesomethingthatyouwanttodo.
AndyesterdayinMonicastock, wesawthatmachinecanbeusedin a lotofmorecreativeways.
Solet's talkbrieflyaboutdifferentapplications.
I justputlike a shortleasetogether, andthe 1st 1 I'm goingtotalkaboutisthisexampleherewhereyouupload a pictureof a catandthenonalgorithmgeneratesforyou a pieceoftextthatyoucanuseasalltextforyourimagetagsinyourisJamil.
Anotheronethatwouldhavebeenreallyusefulforme a fewyearsago I was.
It's thisframeworkcallednotsafeforworkGswhereagainyouupload a pictureininthislittleexperimenthere, anditgivesyoutheprobabilityofbeinganimagethatisnotsafeforwork.
And I usedtoworkinadvertising, andwhenyouworkforbrands, sometimestheywanttoputcompetitionstogetherwheretheywanttoengagewithcustomers.
Sopeoplehavetoupload a selfieorwhatevertowin a contest.
Sopeopleapproved a lotofimages, andmostofthetimetheybrainwantstoseetheimageslivein a galleryorwhatever.
Andthethingisthatthecheckthatyouhavetodotomakesurethattheseimagesareokaytheyusuallydon't manually, eitherbybylike a productmanagerorahorbythedeveloperissousingmachinelearninginthefrontend.
Youcouldactuallydo a precheckofwhatpeopleareapplaudingbeforesavingintothedatabaseandbeforedisplayingitliveon a page.
Sothat's definitelysomething I couldhaveused a fewyearsago.
So I thoughtitwasreallyinteresting.
Andthenfightlikeanotherexamplethatisreally, really, like, totallylike, awesome, like I reallyloveitbecauseitmakesis a lotofdifferenttechnologies, anditcanactuallyhelppeopleintermsofaccessibility.
So I don't knowifyou'veseenthisone, butthisdevelopercalledBabyShakeSingcreated a prototypearoundtheconceptofkindoflikefutureinteractions.
But I thinkit's reallyinterestingtonowbeabletodothatinJavaScriptbecauseintermsoflearningcurve, it's a loteasierfordeveloperstoknowthattheyonlyhavetounderstand a fewconceptsofmachinelearningand, um, andunderstandthesyntaxoffinframeworkratherthanhavingtolearn a new, entirelynewlanguage.
Usuallyyouhavetoshowthemsomethingsomekindof M V P or a proofofconcept, andah, andyoubeingabletodomachinelearningAndJavaScriptmeansthatyoucouldvalidanideaandtheneventuallymovedtoPython.
Ifyouwanttopushitfurther.
But I thinkthat's oneofthereally, like, awesomethingthat I likeaboutyou.
Andthenfinallythereis a biggerandbiggerecosystemoftoolsthatyoucanuseinjail a script, andthere's moreandmoredocumentation.
There's a lotoftutorialsandcourses, so I thinkit's a goodtimetogetstartedifyouwanttogetintothatspace.
Sowhatcanyoudosoyoucandothreethings?
Firstofall, youcanimportonexistingpretrainedmodel, soit's a modelthathasalreadybeentrainedwith a lotofdatafor a specificpurpose, andyoucanjustimportedinyourappanduseitasis.
Um, so I wantedtotoseeif I couldusemachinelearningtohelpmedothat.
Youknow, itkindofdoitautomatically.
So I'm gonnademowritewhat?
I didn't and I showthecode.
Sothisdemoissupposedto I kindofbuiltitusuallytobeusedonthephoneso I couldjust, like, movearoundthedesktoplikeit's workingbecauseit's a website.
But, um, letmejustAllright, So I'm goingtostart.
It's goingtoneedthecamerafeed, and I have a reststickerhidingmycamera.
I needtoremoveit.
Ah, youneedtobuymyself a really, youknow, camerathing.
Putitback.
Okay, soifyou'veeverallright, so I havestartedthecamerafeed, Soyouhavetoimaginethatyouhaveitonthephoneandit's notpointingatyou, butso I justhave, like, a buttonthatherethatsays, youknow, easiestrecyclable.
Soitworkswellwithbottlesofwater.
So I haveabout a water s I'm goingtoputitmymicrophonedownbecause I needOkay, Getoutin.
Thankyou.
Yes, yes.
Soitis a photo, Uh, s sothenonmyphone, I wouldclick.
So, um, ofcourse, youknow, there's a bitmorecordaroundlikethe u I andbuttonsandthings, butjusttobeabletorecognizeimagesusingthatmodel, that's basicallyjustyouloadthemodel, yougetthedatafromthecamerafeedandthenyoutransform a littlebitandyouaskthemodel.
Youknow, canyoudetectthis?
Soatthefirstthingofimporting a pretrainmodelthatsonowlet's talkabouttransferlearning.
Sotransporting, as I saidbefore, isusing a pretrainmodelbutaddingyourownsamples.
Soforthisone s so I liketotryandplayrunwithnewtechnologiesandmixingthemwithaccessibility.
So I'm justgonnalaunchtheOh, myGod, I'm stillhere.
Okay, um, sothisoneisgoingtousethecameraaswell.
Sosee, thecamerafeedisgonnabereallysmall, butthere's a reasonforthat.
Likeyou'renotsupposedtoseeit.
I justshowedit.
So, um, soyouwouldseethewhat's goingon, Butokay, so I'm gonnagototheright.
I'm addingsomepolls.
I'm gonnagototheleft.
I'm gonnagodown.
I'm gonnapatrol.
Then I amtesting.
Okay, Lift.
Right.
That's prettygood.
Andthen I'm abletoactuallyinlessthanin, what, 30 secondstrained a modeltorecognizemyhandmovement.
I hadreallygestures.
And I canwriteanddothatandlikewhy?
Okay, umso I'm gonna, umallright, so I reallylikethisislike, myfavoritefeatureofusingJohnstrippedformachinelearningandinterms, ofcourse.
Sothere's a littlebitmorerecordthat I can't gotoomuchintoitbecause I'm lookingatthetime, and I think I'm gonnabeovertime.
But I amloadinganotherkindoflikeimagerecognitionmodelcalledMobileNet.
I'm againloadingtensorflow, Jess.
But I'm alsowriting a classifierbecause I needtokindofreclassifymymypersonalsampleswiththemodel.
Thenyoujustdeclaresomevariables.
I havefourclasses I'm tryingtoclassifybecause I hadfourgestures, butitcouldbe, youknow, thenumberthatyouwant.
Theimagesizeof 227 thatishardcoded.
I startedthatfromanexample.
I thinkit's because, um, thebiggertheimageofthelongerit's gonnataketotrain.
So I thinkifyouwantedtobefastthan 207 isfine.
Thetop K isthe K valueforKenya's neighbor.
Thatiskindofarbitraryaswell.
Youcanchangeit.
I can't gotoomuchintowhatthealgorithmactuallydoes.
I stillhaven't builtthatproject, but I'm stillhappywithoutumSonowlet's talkabouthow I putittogether.
Oh, Okay.
Sofirstofall, andbecausewhenyoudoYeah, I know.
Andthat's theonlythingpeoplerememberfromthetalk.
So, umsobecauseokay, sowhenyoubuild, whenyoudoeverythingfromscratchusuallyhavetokindofyouhaveanydata, right?
Soontheright, youhavethequickroadthat I saidthatwewillputtogetherWheretheyTherewas a gameonlinewherepeoplecouldwereaskedtodraw, I don't know, a strawberryand a doorandglasses.
Andactually, allthesesamplesweredownloaded, sotheywereactuallysavedin a database.
AndGoogleputthisthat I saidopensource.
Wecanuseit.
Thethingis, obviouslyGoogledidn't askpeopletodrawwillies, But I neededthemformy, uh, youknow, formyprototype, becauseifitdoesn't knowwhatitreallyis, youcantellmeifit's oneofthose.
So I drewsomeofthemdefinitelyknowasmanyasthedoctorsaid, becausethat's likemillions.
Um, butso I neededthat.
So I neededtohavedatatogether.
Thethingisthatwhen I drewmyown, I hatethislight.
Butsowhen I dreadonthecanvasoninthebrowser.
Itwastohurtby 200 pixels, buttheimagesandthen I said 28 by 28.
I don't reallythinkthere's a purposetochooseitbefore, but I inthisparticularcase, I usedwhatwecalltheCNNcompetitiononyournetworkbecausefromwhat I read, itdealswithwithimages.
Sointermsofcode, I'm gonnashowlike a reallysmallcouldsample.
I triedtoget, like, thecorebitsofit, butwhatyouhavetodointensivefloor, youhavetocreatetenses, whichisbasicallyliketheway a waytoshapethedatasothattensorflowwillunderstandhowtodealwithit.
Andas I said, youhavetocreate a trainingsetandtestsiton.
Thenyoucreateyourmodel.
So I have a sequentialmodeland I addlayersofahahconvolutionableto d layerandthenyouadddifferentlyisaswell.
I don't havetimetoreallygointothisindepth, butalsotheamountoflayersis a laboratory.
Soyouneed a largeamountofdataifyou'renotusing a pretrainedmodel.
Umyouknow, aswetalkedaboutbeforefor a computertounderstandwhatsomethingisusuallyneedsmillionsoffsamples.
Itcantake a lotoftimetotrainyourownmodel, dependingonwhatyou'retryingtodo.
Especiallyifyou're, I think, dealingwithimages.
Itcantake a lotoftime.
I remembertothinkaboutthemobileexperiencebecausewhen I wasplayingwithloading, likeimporting a pretrainmodel, somemodelscanbe a fewmegabytes, andthatkindofhurts.
Um, soifyou'reonlike a lownetworkisgonnabereallyimpossible, Andevenintermsofusingthebatteryofthemobiledevice, itcanbequiteexpensive.
Youhavetobesurethatifyoutellsomebodythattheycan't get a loanorthattheycangetacceptedandtheysueyouyoucan't justsay, Oh, weusemachinelearning, so I don't know.
Um, youhavetojustbeabletoexplainwhathappened, andmostofthetime, peopledon't knowwhat I definitelydon't know.
AndthenfinallyandreallyopponentLee, evenifyou'renotinterestedincodingmachinelearningthetopicofbiasandethicsin a I is, like, superinteresting.
Um, and I justliketomentionoftenthealgorithmsarenotbiased.
Weare.
I feellikewelovetopushtheresponsibilityon a pieceofcodeandnotjustlike, lookatwhatwedowrong.
AndifyoutranslatesomethingwhichisittoEnglish, thenallof a suddenitbecomesgenderedwhereyouhavethingslike, Heyishardworkingandshe's a lazy, likeallof a suddenit's gendered.
I'm notlazy.
I mean, youknowwhat?
She's a cook.
He's anengineer.
That's theworst.
Um, soitisthatkindofstuff.
It's likeandthenyouhadarticlesaround.
Oh, youknow, a Googletranslateissexist.
Well, whatdoyouthink?
It's thedatafrom, um, so I thinkit's justrememberthat I dothinkthat, uh, machiningcanhelpusrecognizesomeofourbiases, andthenhelpusYouwillbebetter, butthealgorithmitselfisnottheproblem.
Uh, okay, I'm gettingtotheendofmytalk.
So, um, thestoryaboutthatoneIsthatso?
I gavethistalk.
Um, I'm justgoingtotheunicorn.
Eso I gavethistalk a fewweeksagoinLithuania, andifyoulike, justthedaybeforeor a fewdaysbefore I hadfinishedtheprototypethat I wasbuildingthat I was, like, superexcitedaboutthatinvolvedmachinelearningandjavascriptAnd I wantedtoshowtheworld.
But I waslikeShorty.
Well, now, forsomepeople.
It's not a surpriseanymore.
But I thoughtthatif I didn't showithere, itwouldbeunfair.
So I'm gonnashowitaswell.
And, umandyourvisitorissupposedtobe a topicfor a talklaterthisyear, butitisrelatedbecauseitusesmachinery, so I hopeisgonnawork.
Sotheinspirationforthisissothisis a prototypebuiltby a developercalledCordishwhousedtheWebcamfeedtotraintheiralgorithmtorecognizesomegesturestothenapplyittoyourgame, too.
Sotobeabletohavethispersonalizedexperienceoffofplaying a gameThething I wasthinking, though, isthatwell, youhavetobeinfrontoftheWebcam.