Everybody.

And welcome to a new tutorial.

Slash some coverage of a new package called requests Dash.

HTM L.

It's basically just a way for you to really quickly, easily parse, um, html.

So it's written by the same person who wrote the request library, so my expectations are high, but this is a very new package.

It's been on Get hub for about a month, maybe a little over, um, so I would expect some rough edges, but let's check it out.

So, first of all, to install it, let's scroll on down to the bottom to install it.

You just pipped install, request, stash HTML.

You will need python 3.6 or I'm assuming later.

But right now, the latest release of official Python is 36 So, uh, let's make sure you install it and then let's go ahead and check it out.

So check it out.

I'm gonna be using the following Web page, at least to start.

It's just python.

Parliament and Slash parse me, make par space and basically got some tax A list.

A table.

It's got imagery.

It's also inside of def tags, you see, kind of poke around That's got some Java script tests.

So if the job script loads, it says, look at you, shine in, and then it before it loads Its default is like, Why, you bad, though I think then we've got some pre tag information in here with the Zen a python, and we've got a link down here.

And then we've got some goofy looking characters here that it could throw you off.

So with that, uh, this is what we're gonna start off by parsing.

So coming back over here, it looks pretty simple.

I'm just gonna copy and paces and just remove the the interactive interpreter dash things.

So it should be that simple.

So you can request, not get I'm gonna pass in my own.

You, Earl.

You can feel free to use something else if you want.

Um, I'm just gonna be exemplifying some of the things I use this page because for this one page, I can control what happens to it.

It seems like every time I do a tutorial on literally anybody else's pages or with anybody else's AP, they all get deprecate ID.

They all change.

So this one should be a rock now eso that starts your session, we get some stuff, and then we can start actually interacting with it.

So again, I don't really know everything with this package, But generally when I check out a package, I mean, they might I'm not sure if he has, like, a documentation page.

It doesn't look like he does.

Let's see.

Oh, maybe he does.

I see what this looks like.

It's pretty much the same thing.

Um mmm.

For H team, this is interesting.

Pagination.

That's Ah, that's more magical than I would have expected.

That's incredible.

If that really is that simple to do pagination.

Wow, that's not written up here.

Okay, so anyway, that's interesting to say the least.

Um so then you requested Oh, Mike.

And then you could just generate over it.

Oh, Uh huh.

Um, that might work really well with red.

I'm curious to know if that works well on other websites.

Want to find well, to test that maybe at the very end, um, you can use it without request.

So if you downloaded maybe some HTML documents, you could then parse them.

Ah, yeah.

This has a lot more explanation.

Well, okay.

Cool.

um, I'm really that's that peyote pagination soon for.

That's just icing on the cake, man.

I would think you'd have to find the link to it and then go to the link.

The fact that that patch in age I'm sorry, I'm that's just that's just interesting.

So back to the simplified version of the docks, though all we need to do is once we get, we can just start referencing.

And I'm pretty sure as long as it's not Java script.

So yeah, immediately, we could just say, like, for example, let's just do our dot Well, first of all, let's print dir farce.

Let's just see all the things that we could do with it.

Let me just pull this up so immediately on the request.

Um, we could check against the encoding.

I get apparent in Cody.

We can close the request, uh, cookies and coding.

I'm not sure what the difference stream coating apparent encoding is.

Check the headers.

History weaken.

Reference.

The HTML.

We can check if it was a redirect.

That's interesting.

So just automatically does redirect you, but you can find out if you've been redirected.

That's interesting.

Uh, Jason, I wonder if you would just pull it.

I'm not sure if you say Dodge a song like it is like if it's like some sort of a p I or something in the responses.

Jason, if you could say Dodge a science that Jason object now, I'm not sure I'm guessing that's all the links.

That's if it's paginated, apparently, which is fascinating.

Also, be cool if it wasn't paginated like, I wonder if you could make some sort of crawler that just, like, automatically goes to any link and just keep slinking around.

If anybody that's trying to build just like a Web crawler, that's kind of an annoying thing to have to build.

Um, okay, cool.

So we've got text.

You are L.

And then the HTML.

So let's check out, uh, let's just check out dot html eso First of all, that's I guess we'll print out the dirt dot html as well, but let's see what we've got.

Text raw HTML search so we could search the HTML um, assuming links, find all of the links.

So, like, for example, it's print our dot html dot links, see what that looks like.

Okay, so it looks like it gives us.

Ah, set.

I guess that makes sense to give it a set.

Just in case there was multiple of the same.

I'm guessing that's a set.

I don't know.

Weaken, Uh, type.

Just looks like a set.

Yeah.

So it gives us a set of the links.

Um, what else could we print out?

Are that links?

We've got our dot html.

Wasn't that a dura of order?

Html Who gets all formatted and beautiful for us?

So what's the difference between a seamount and raw H de mille?

Oh, yeah.

Okay.

That gives us, like, the new lines in, like, all the tabs and spacing and stuff like that.

Okay, Okay.

I'm not sure exactly why you would need to do that, but, um Okay, so that's interesting.

The other things.

Let's see.

Let's check out.

Find you confined.

It looks like, uh, the idea.

Yes or not?

I'm not sure.

Let me see if if we have i d s here.

I know.

At least we have probably I d in the Yeah.

Yes.

No, J s.

I wonder if we can find that one.

I'm just gonna copy paste art up.

Find Yes.

No, Js print about even though it's kind of bad words.

Ah, darn, it was He, uh, probably got text.

I'm just guessing.

Nice.

Okay, so that's how we could find it.

I don't like what it's saying to me, but that's how we can find it, Which then brings me to the next curiosity I have, which is with the Java script rendering.

So what's happening here in first of all, let's just say, um I don't know Js test.

Uh, copy that.

Pace that Here.

Uh, save.

Okay, so at least, um, the whole point of this was to test if we could actually load Java script when we parsed things.

Um, since that was part of an older tutorial, but now with request html apparently, it's super simple, So let us check that out.

So if we scroll onto the documents, it looks like basically all you gotta do is just use dot render.

So s.

So let's go ahead.

Basically, you're gonna want to render it right after the request.

So let's just paste that in there.

So with Java script, the immediate return will be the base JavaScript, including, like the actual JavaScript code and in order to render Java script.

You actually have a browser that runs it like That's how JavaScript works.

Like you request the server and the server just responds in Just text, obviously, and then your browser's going to see.

Okay, here's some script.

We want to run this script.

So what you have to do in order, read elements inside of tags that are updated by job.

Script says you need to render it, and you also have to kind of wait a moment.

You have to wait until it's done rendering.

So that's kind of a tedious thing to build yourself.

But apparently it's in in here, ready to go.

Let's go and run it and let's see if it updates it.

So on the first run, it looks like you have to install chromium.

So I guess gonna download download may take a few minutes.

I hope it doesn't.

I don't really want to wait that long.

Maybe I'll posit what we go, Um, but I think it's done.

Okay, It's done okay.

And it's just telling you where it extracted to, um, only an heir unable to remove temporary user data.

Interesting.

It did work, though.

That's the That's this is the text right here that we're looking for.

So, like, we just kind of make some space here, since the air's in our way.

Um, probably got permissions there.

Honestly, uh, anyway, yeah, that's what we were looking for on that's indeed, you know, if we come over here, so it definitely works, but for some reason, we're getting an error when we attempt to remove the user data.

So let's open up this file the launcher.

So at least in sublime, it's really nice.

You could just double click on the thing that airs, and we could check it out.

Okay, So in this internal method, we're trying to clean up some data and they're using Shue till remove tree.

So is trying to remove a directory.

Basically, that's how you're gonna remove directory with contents.

Um, home.

So we already we saw where it went.

It goes into users h and then that, uh, let's see me see if I can find it.

Ah, yeah.

So it goes in, um, you know, here it is.

And then if we go into there, we can see the contents.

But for some reason, we're unable to delete them.

I'm gonna guess it's a permission.

Zehr.

One way we can confirm that is to not ignore Ares.

We'll set that to false, just in case anybody's like, Why would you ever ignore the errors?

I forget that.

I think if you don't ignore the error, I think one of the errors is that the file directory has contents, and that'll stop it from working.

If you ignore errors, it does work.

But we also don't get to see the air.

So let's go ahead and see, because things aren't working.

But once you once it is working ones Shue Till does work for you.

You actually kind of wanna access is denied.

You actually do wantto put ignore heirs to.

True.

So let me just set this to true again.

So we're seeing here access it tonight.

Just we don't have the permissions.

Um, I'm kind of curious.

I kind of want to open up an administrator to see if I Kenbrell that.

Let me pause for a moment.

Let me open it up and head to where we're working and see if we can use, uh, at least command prompt as administrator and see if this works.

Okay, let's give it a shot.

Where?

Administrator control panel.

Wow.

It's still can't remove it.

We did get the info here, though.

Um Mm hm.

That's gonna break any script that's running, too.

Though that's really quite the bummer.

Uh, least right now the show must go on.

So I'm gonna continue, uh, at least a short term fix that I can think of.

Um, why wouldn't that work?

I'm pretty sure if you used your own like Lennox and you pseudo, though, that this wouldn't cause any problems for you.

It's He beats the location.

I'm not really sure.

I really can't decide why.