Placeholder Image

Subtitles section Play video

  • What's going on?

  • Everybody.

  • And welcome to part two of our playing with Google takeout data video miniseries thing in this video we're gonna be doing is now building some like bar graphs.

  • Basically, from the top 10 were prodded and go with most common words with the sliding one year window.

  • And later we can try different windows, or you guys could just do it on her on your own time.

  • But for now, we'll do a one year window.

  • So with that, let's go ahead and get started.

  • So we should you should at this point have my life dot database and that should be populated with the, you know, the primary key UNIX and then all of the unique words.

  • So, uh, what I'm gonna do now is I'm just gonna copy search database.

  • I'm just gonna rename this to be, um I don't know, images, uh, and then open super close, that old one.

  • So it doesn't really matter anymore.

  • So now, and in fact, I kind of want to keep the original just so I don't have to re type some stuff.

  • Um, so we still want to connect to my life database.

  • We don't need the search activity directory anymore.

  • We don't need apart states, and that should be good.

  • So the next thing I'm gonna do is we want we need a directory that we're gonna dump all these images into.

  • So I'm just gonna make a new directory real quick.

  • I'm gonna call this images.

  • Uh, let's do word images, and we're gonna start with one year window one day slide.

  • It's kind of a long clump directory, but we'll go with it.

  • That's fine.

  • Um, so why, uh, well, wolf work that in in a minute.

  • So now what we're gonna do is we're going to specify Cem starting constants.

  • So a day we always knows 86 486,400 seconds.

  • Um, and then, like, a year would be equal to 365 day, um, and then for our slide, we're going to say that is equal to one day.

  • Okay, so with that, what we're gonna do is just start generating or word graph.

  • So we're say, define define words.

  • Graph.

  • Uh, now, what we want to do is we're gonna save everything into here, so I'm gonna copy that.

  • Come into here and I'm gonna say, saved their equals.

  • That so that's we're gonna stuff all the images now we want to do is first of all, we need to starting in unending time.

  • So we should be able to do, like a query.

  • I think so.

  • Something like I don't know what it is.

  • Let's look it up for a quick sq light Max column escalate, Max.

  • Okay, hold on.

  • Find sq light.

  • Find Max.

  • We could also try men.

  • I just want to see the expression man select Max from Okay, So?

  • So the first thing that we could do is we could Look, all we want to do is find the minimum time in maximum time.

  • So the words sports Kraft.

  • And then we're going to say, um well, we're just gonna do it, si dot Execute.

  • Um, I guess select Max UNIX Look of like, a hair.

  • Something over here.

  • Select.

  • You know, Max UNIX from, um from words.

  • And then let me just print.

  • Um I'm just gonna say d equals c dot Fetch all.

  • And then we'll front d.

  • Okay, so those are max time, so then we'll just say Max time equals D.

  • Uh a deep.

  • Well, we could just say this.

  • We can't call to fetch ALS, but we should be able to just do that.

  • And then 00 right.

  • 00 And then we should We're doing the stupid tab thing, huh?

  • What?

  • Why don't you let me do this?

  • There we go.

  • Thank you, sir.

  • Thank you.

  • That's so generous of you.

  • Okay, so now, Max time and then we'll do the same thing with minimum time.

  • So boom, change this to be men and then change this to be men, and then we'll print, um men Time, Max time.

  • Run that minimum time.

  • Okay, cool.

  • So that looks like it works pretty darn well.

  • So now we just need to start sliding over.

  • So are our start time will be equal to the men in time, and then our this is like our current.

  • So this is for our window.

  • So our start times, men time our end time for the specific window will be men Time.

  • Plus, I thought we had window.

  • What do we call this thing year?

  • Oh, we're just calling a year.

  • Okay.

  • I think I'll call.

  • I'll make a variable called window equals one ex year cool.

  • Meantime, plus window.

  • Okay, So, um, so that's our starting our end.

  • And then now what we want to do is we wanted just, like, iterated over.

  • Um, try to think of how exactly I would do this.

  • So I guess we'll go.

  • Wow.

  • Loop.

  • So, like, wow.

  • Um wow.

  • And is less than max time.

  • We want to make the query, um, for all of the words between start and end.

  • So, um si dot execute and we'll make a platoon f string.

  • Um, select word from words.

  • It looks like we lose our syntax highlighting when we go f string on a query.

  • That's okay.

  • Aah!

  • Select word from words where UNIX is greater, then start.

  • And UNIX is less than and beautiful.

  • And then we're gonna say, uh, data data equals c dot Fetch a ll, and then I'm gonna say break here, and then I'm gonna print data.

  • Fantastic.

  • So now that we have that, um, so this is like all our individual words in that timeframe now we want to do is take all of these words and throw them into some sort of list, and then we'll do.

  • We'll create a counter object and then do dot most common.

  • 10.

  • So So before I forget, Because that's the next thing we're gonna do from collections.

  • Import counter.

  • Done.

  • Okay, so, Data.

  • So then we wantto iterated through here and upend this two words.

  • So while this blah, blah, blah Okay, um, we'll do words is empty to equal to an empty list.

  • And then what we're gonna say is four well, summer yet is Rohan data.

  • So for road in data, word equals ro zero, um, words dot upend row zero.

  • Now, that's just on the fly.

  • But we really could probably What could we do here?

  • We could say words equals, um I zero right.

  • Four.

  • I skin data, right.

  • We should be able to get away with that.

  • So I am going to do this.

  • I 004 Okay.

  • What do you mean?

  • Okay, cool.

  • All right.

  • So that works.

  • So that gives us our words.

  • Now we can create a counter object.

  • So what we can say is, um, word underscore, er counter equals counter words.

  • And then what we could say is, um, we could say common words equals word.

  • Underscore counter dot Most underscore.

  • Common 10.

  • And then, rather than printing all of the words, we can print the common words.

  • Why did that just happen?

  • Because we're printing data, I guess.

  • Let's try one more time.

  • Yes, So you can see the top of the charts.

  • Python Not really too surprising.

  • Bitcoin pi probably with raspberry pine.

  • It is kind of weird that we have more pie than raspberry.

  • Not quite sure why that would be.

  • Maybe I type owed raspberry.

  • That's probably it.

  • Price forgot the Pee or something.

  • But pie, it's fascinating.

  • I almost kind of want to find out if that's right.

  • I bet I just type owed raspberry.

  • But anyway, cool Lennox S and P for the S P 500 again, probably because I don't tell.

  • Sometimes I type S P 500 sometimes S and P 500 but 500 always is the case map.

  • What?

  • Lib?

  • I don't know what's happening here with Google.

  • That would be an interesting one and then Satan himself.

  • So Okay, so that's our most common.

  • Now we'd want to graft those, so that would be on a single time.

  • So So how We're gonna make a bar graph here and We need a couple of things when we make a bar graph.

  • I wish bar graphs were easier.

  • And Matt plot lib.

  • Which, by the way, let's go ahead and import before I forget from Lizzie.

  • So, import Matt plus matt plot lived up.

  • I plot as put from Matt plot lib, import style and style that use a good plot.

  • Okay, great.

  • Beautiful.

  • Couldn't ask for a better set of imports there.

  • So now, um, we need to kind of set up how we're gonna do this.

  • So common words probable.

  • Okay.

  • Common words is good.

  • We don't need to print commoners anymore.

  • Uh, and instead, what we want to say is we're gonna set the y position.

  • So why oppose equals?

  • M p dot a range?

  • I don't think part, Len.

  • Common words in private.

  • 10.

  • But later, we might add that to be, like, constant that we change over time.

  • The important, um, pie Or have I have not import numb pie as in p.

  • Back to work.

  • Why?

  • Position?

  • Great.

  • Um, and then we're gonna say a word underscore counts is equal to so actually what we need to say, um, doing it common words.

  • What instead, What I'm gonna say is, um Topic zero for topic in this cool.

  • So it'll be common words.

  • And then the word counts will just kind of be the topic.

  • Uh, one.

  • So this will be the actual word itself in the counter, cause the counter object was a list of two bulls.

  • Um, where the first object in the to pull is tthe e uh, word.

  • And then the 2nd 1 was a count.

  • So cool.

  • Okay, so now that we have that we're ready to actually graft this and because we're gonna save it, we wantto adjust the size right away.

  • So we're gonna peel tea dot figure, and we're gonna say fig signs is equal to 12 and seven.

  • That should gives a 1200 by 700 pixel.

  • Um, graph.

  • So that's our appeal to dot figure.

  • Then we're gonna peel tea dot bar.

  • Why?

  • Pause.

  • Word counts and a line underscore her.

  • I'm sorry.

  • Not a line center, and then we're gonna give it in Alfa of not to pull 0.5 just to make it semi transparent.

  • Well, it will be beautiful.

  • And then p lt dot exe ticks.

  • Uh, wipo's common words.

  • So that way the ex ticks, rather than being like numbers, aren't said the actual word.

  • Because that's actually what we're trying to graph.

  • Here is the words.

  • And then how much like the Y axes will be volume.

  • So with that said, Peel, tea, why label people we'll call this volume and then finally, we also do a title and will display the date in the title.

  • So on a single frame it'll be somewhat ledge a ble.

  • But mostly what we want to do is make it somewhat readable while the frames are going at, like 30 or 60 frames a second.

  • So what I'm gonna do is make this an extreme, and I think we're gonna bring it yet another import, um, from daytime import date time.

  • And then we'll create a I'm trying to decide what I want to do there.

  • It's probably the dumbest way to do it.

  • Well, it's done this way, but I'm someone's gonna come up with a better way, but whatever.

  • So what I'm going to say here is F um, we're going to do Dave Month year because that'll be the easiest to follow.

  • I think so.

  • Just understand that's what's happening.

  • All non Americans are probably very happy from time stamps.

  • We're gonna get this daytime object from a time stamp.

  • Meaning you next time and we'll go with end because that's like it's like the last years of data for today.

  • Okay, so macro info up to today.

  • So we're gonna use the ending time stamp.

  • Um, so this and then and then what we can say is dot day, month, year dot day day.

  • There we go.

  • And that gives us our daytime objects that will give us a day like the number of the month.

  • And then we'll do month dash year.

  • So day, month and then year.

  • So April 1st 2019 would be 14 2019.

  • And then, as these things were changing really quick, we can kind of see it at least know where we are along the line.

  • So Okay, so we set the title, and then now we'll run.

  • We'll just quickly run appeal t dot show later.

  • We're gonna save the figure rather than show it.

  • But okay, And then, um I guess that's it.

  • So let's see what looks good to me.

  • Let's go ahead and run this bad boy that took a long time.

  • Okay, but here it is.

  • Hopefully won't always take that long.

  • Okay, so here's our graph.

  • So the 14th of June 2014 So we can see here.

  • Nice.

  • Pretty graph.

  • Now, it would be very tedious to go over this one at frickin time.

  • So instead, what we want to do is make it into a video.

  • So the way, the way we're going to do that first we have to save all the images.

  • So I'll probably save all the images and kind of like, pause while I do that, and then we'll we'll pick back up.

  • Um so let me I'm gonna clear this out of the way.

  • Clear this out of the way.

  • And rather than people to, you know, show we're gonna peel tea dot Save, save.

  • Underscore.

  • Fit.

  • Note.

  • Knowledgeware say fig.

  • Uh, and again to an F string.

  • And it's gonna be saved.

  • Her great.

  • See?

  • Is it going to save her?

  • Yeah, because that's where we want the images to go.

  • And then we're gonna make the video from this.

  • So we'll put both in the savior so saved her.

  • And then, um, and then we want to save the images with, like, a counter.

  • So we're gonna start a counter, equals one, actually started zero.

  • So arranges easier counter zero.

  • And then every time this is successful, we're gonna take up the counter.

  • Wow.

  • Probably say figs.

  • Oh, yeah.

  • And then we'll do counter, um dot PNG no longer show cause that'll take forever.

  • Um, And then when all that is done, it's counter plus equals one.

  • And then we would do start Plus equals the window No plus equals slide.

  • Right, Because we want to slide it by a day and then end plus C equals who?

  • That was rough.

  • Okay, Yes, I think we're good.

  • I think we're good.

  • Very good.

  • Let me come down into here G data and let's go ahead and run images dot Hi python python images.

  • Dupuy.

  • And let's see how that goes.

  • 0 100 for printing stuff we probably don't want to print.

  • We also don't know where we are along the way.

  • Let me go ahead and France print counter.

  • Save that.

  • Um, don't really know if I want to change it.

  • Let's just check the image is real quick and just see they're saving open.

  • Everything's going really slow on this machine.

  • Why would you just open an image brute?

  • I have no idea.

  • Why can't I open this image?

  • There we go.

  • Thanks, Idiot.

  • Okay.

  • Um cool.

  • Yes.

  • Oh, that looks good.

  • I'm just kind of shifting through the images.

  • So now we want to do is make a video out of those images.

  • So we'll just kind of let this run while we work on image video.

  • So I'm gonna go back to G data and I'm take images, Understand?

  • Copy and pay is just have a pie file.

  • Basically image video.

  • So now it is convert images to be, and I've already got code that does this.

  • So I'm just gonna go copy and paste it.

  • So I'm gonna go to the best python learning Web site in existence.

  • I'm gonna go type in unconquered.

  • Yes, sir.

  • There is.

  • And I'm gonna goto unconventional neural networks and then I'm gonna go down too deep dream deep dream frames.

  • Deep Dream video is what I'm after.

  • Thanks.

  • And I'm gonna just copy and pasta all this jazz and then we'll change what is necessary in a moment.

  • Just take note though I do have two different If you're on Windows, use this.

  • If you're on Lenox, use this.

  • If you're on Mac, I actually don't know.

  • Uh, Out.

  • We do need to modify out to be the proper resolution.

  • Um, this I'm just gonna probably hard code, so I'm gonna skip that.

  • That's just trying to find the length of the dream.

  • Really?

  • Um, out not release.

  • Done.

  • If you haven't watched that video, by the way, check out the a deep dream of a neural network.

  • Those were actually really cool.

  • It's just like every frame it runs a neural network.

  • That kind of like, uh, applies.

  • Um, the patterns it sees basically to an image.

  • It's It's super cool.

  • Definitely.

  • Just have to go see it.

  • Uh, okay.

  • So see how we're doing.

  • Still go in.

  • Still making the image is good to know.

  • Good to know.

  • Okay, so, um, great.

  • So now we're gonna do start modifying this stuff, so I don't need dream name up fudge.

  • What?

  • Failed to allocate.

  • Okay.

  • There's gonna be a problem, because this takes a while.

  • I really don't want to start that over again.

  • And I also don't know what that issue is feel so we should probably end up making about, uh, what, 356 times?

  • And it was, like, five years.

  • Well, a little less than five years.

  • So more than 3 56 I'd be more than 1500.

  • Uh, so about probably 1800 images.

  • So let's go, properties.

  • Deng, we're way off the mark.

  • And if we have to keep doing this every like a few times, um So here's what I'm gonna d'oh I'm gonna just hack it and I'm gonna say, uh, imports os, and then we're gonna check to see if this image exists.

  • So we're gonna say if o s stop path thought is file and then we're gonna check with an F string for this.

  • Like, if this image has already been saved, then forget about it.

  • Let's just move on.

  • Boom.

  • What is wrong with you?

  • Uh, pass else?

  • Then we need to create that image so we'll do all of this up to the counter tabbed over.

  • Boom, boom.

  • Okay, slow save and let's continue running.

  • Python images dot pie.

  • Great.

  • Picked up where we left off.

  • Perfect.

  • Well, but I have to do that like five more times.

  • But, hey, it's in the name of science.

  • Everybody.

  • So here, let's just say for I and range for now.

  • 300.

  • It's just however many images we have, so it's no big deal.

  • Image path is going to be Ah, basically this path where the images are right.

  • So we're gonna just copy pasta that into post a path that joint?

  • No, this for this will just be i dot PNG.

  • That's correct.

  • Good stuff.

  • Good stuff.

  • Saved her.

  • We don't have that, actually.

  • Let's grab that real quick.

  • Um, where is the nearest?

  • That's kind of gross.

  • I don't know if I want because I'm gonna change this.

  • That's gonna be annoying.

  • That's okay, though.

  • Um, just go weird.

  • I thought I deleted these already anyway.

  • Saved her.

  • Boom.

  • Done out.

  • Um, again, That'll be this dot and rather than I'll fix that.

  • Um, so formats this saved er slash video don a V i and then it needs to be We're gonna go with 60 frames a second.

  • A little quicker with 30 frames will take a while, and we just kind of want to see it over time.

  • So then it will be 1200.

  • Bye.

  • 700.

  • Save that.

  • And let's just see how many words we have now.

  • Our image is rather 607 so we'll just say 600 0 we will save that.

  • And while we're waiting for that other thing to run, let me just do this Python image video dot pie that will create the video.

  • Hopefully, let's see what happens.

  • Looks like it's doing it.

  • That's cool.

  • Uh, you know, go in here and it'll save the here when it's done.

  • Awesome.

  • And let's see what it looks like so far, I don't care.

  • Yes, it's cool.

  • So I wish video a V, I would go.

  • Go away.

  • Um, so yes.

  • So here you can actually see is like certain things like tick up and then go away.

  • So the python pandas got bootstrap flask and then cut out.

  • But anyways, it's kind of cool because you can, like, see all this stuff changing over time.

  • 2007.

  • Okay, we lost python.

  • No problem.

  • We've got some epic air handling over here.

  • Failed to Ella.

  • If anybody knows what's going on, I don't think like I don't not sure if we just ran like a clear fig.

  • If that would help.

  • Look, I'm just not sure Failed to allocate bit.

  • Map failed to allocate bit.

  • Map metal p l t dot clothes, fig.

  • But we don't have a fig.

  • We'd have toe, I guess, create a figure.

  • Or maybe redefine a figure.

  • I don't know.

  • By the time I do bug, that will be done.

  • So I still really feel like doing it.

  • So now I'm just gonna keep doing this until we get to, like, we're done with our, uh, you know, probably 1800 ish.

  • Maybe more, maybe less.

  • Um, okay.

  • And then when we're done, I'll show you the video, and then we can, and then I'll probably just leave it.

  • Maybe I'll do another one.

  • I just don't really see much point.

  • It's more interesting looking like I'm interested in seeing my data.

  • And I'm sure some people are interested in seeing my data, but too bad.

  • Do it with your own data.

  • So while we wait, shut out to some of mine channel members who have been long term channel members with me for six and seven months.

  • Joseph Coleman, Bill Fallen Sze be Louise Fernando F um, I'm gonna I'm gonna mess that up.

  • Reginald Roberts and met in oil.

  • Guys, thank you so much for your long term support.

  • You guys are freaking amazing.

  • I wish you tube gave me better options for, like, figuring out when people have re upped and even like, uh, some of the other.

  • I'm trying to think What the I'm just like blanking on the name right now, but there's, like, stream options like people use for streaming in like that.

  • Whenever someone re ups, it should be listed, right.

  • But I feel like the YouTube a p I is like, not helping people figure out when someone has, like, re upped.

  • So anyways, I just have to, like, manually go through that list, and that's kind of weird.

  • Um, okay, so I'm gonna pause here, and then I'll come back when this stuff is done, and we can kind of see at least my macro search.

  • Don't think I'm gonna do the micro stuff, but if you wanted to, you really would.

  • You would just go over here, change it, change the windows.

  • So this is just gonna be really big stuff that I did like lots of searches of and then it's gonna dominate this stuff that I did.

  • Um, you know, on a micro level in them, You know, if you go down to the month, you'll get just much more granular stuff going on.

  • So, um, definitely, really interesting.

  • I encourage you to check it out with your with your own data.

  • Anyway, I'll come back when this is done, all right?

  • And we're done.

  • I went ahead and converted to video, and we can watch in all its glory.

  • So, um, like, python tops the charts, but then you can kind of see are like they got pandas flask.

  • Whatever Google's doing there, you can all see elite dangerous like elite.

  • And then teii tress.

  • Ah, which is Ah, like a video game.

  • Um, and it's kind of interesting to see how that kind of goes up pretty high.

  • Well, I think it made it may be temporarily to like fourth and fifth spot.

  • Uh, and then that goes away.

  • You got 76126 That's Ah, zip.

  • Code poking.

  • Mine comes up Pride for Pokemon Go His presidency go here.

  • But whatever.

  • Um, then you got the s 2000 pops in climbs like almost topping the charts.

  • Really?

  • Battling python?

  • They're definitely low.

  • It shoots up and it Wow, it definitely passes.

  • Python and O S R s, uh, old school room escape shouts out, Um, Texas Tensorflow Corps and Grand Theft Auto.

  • Towards the end of 2017 um, s 2000 starts fallen off, which is pretty true.

  • I eventually moved away, and I still have these 2000 but I just don't really drive it very much anymore, then pops up 35404 Not yet Passing 76126 It is probably take awhile, like, because we're going back on entire year when I actually moved.

  • Is pregnant take over like a pretty significant time to surpass the other one, like probably about six months Would be, um, mathematically it and looking at that curious anyways, um, but yet there's 3544 now on the chart.

  • 76126 is gone.

  • Portia's coming up old school room escape stopping George.

  • Um, lips.

  • I don't know why that happened then.

  • Then we finish, um, April 4th, 2019 old school room escape, killing it.

  • It's because I'm a python expert.

  • I have to go pilot anymore.

  • And 35404 zip code Porsche Weather.

  • No surprise, Kare Aas.

  • I'm surprised.

  • Care us is that high in the charts?

  • To be honest with you, Alabama Tuscaloosa.

  • Guess where I lived.

  • Pan does.

  • All right.

  • Well, very interesting.

  • Okay, well, hopefully your data is interesting as well.

  • I strongly recommend you change some of the windows and then kind of like, look and see how your trends change.

  • Over time, we could totally, easily create an algorithm that, as something like you would just compare the previous top 10 to the new top 10 and anything that's changed, like his one comes out like any change means one left and one came in or more.

  • Could be two or three.

  • Um, but it would be cool to track that over time.

  • So you know exactly when those changes occurred and you could probably get it pretty like I said, because it's a year like any major event.

  • Like when I moved or something like that.

  • Any major event would definitely take, um, a decent period of time toe actually shift over.

  • So, um but I bet I just wonder what would happen if we went down to, like, a week or a month?

  • It would be almost perfect.

  • Like to figure out when major events changed Or like when I bought a new car and all that stuff.

  • So anyway, pretty interesting.

  • And using pretty brutal mentoring program, I think we could find out a lot about people.

  • So anyways, cool.

  • If you guys want to Seymour about playing with Google Data, let me know down below what you'd like to see in all that definitely want to dio Texas speech with our own voice.

  • I think that would just be really cool.

  • So, um, kind of hoping to do that?

  • But we'll see about that anyway.

  • Have you got any other ideas?

  • Feel free living below as always.

  • Thanks for watching things.

  • For all the support of subscriptions, the memberships, the donations, all the fun stuff.

What's going on?

Subtitles and vocabulary

Click the word to look it up Click the word to find further inforamtion about it