Everybody and welcome to in R and D style video.

I don't normally do videos like this because it's basically either impossible are very difficult to structure things in such a way that people can follow along.

But in these times, I think people are pretty starved for content.

So I thought, you know, I could do a video on at least some of the stuff that I do that you guys really never see.

So Project, like what I've been working on here with this chap, is kind of so big and so just complex that probably there will never be a serious from start to finish of doing what I'm doing here.

So instead, I thought it would be interesting to try out kind of like a more or Indy style type video as well as at least towards the middle of may be the end.

What I can do is I can leave.

What I'm gonna try to work on here is scoring Chap Output, and I think I'm gonna do some very rudimentary scoring and then later kind of talk about next steps.

But then also, um, what I can do is kind of have an output of these chap bought outputs and then see if anybody can come up with some better ideas for scoring, of course, credit all given to anybody who does it if they do so.

Also on the note of content starvation.

Still, lots of people asking about neural networks from scratch.

Two things.

The videos.

My hope is to get them coming out sooner than I kind of had initially planned.

Just because again, I think so many people are kind of just stuck at home with nothing to dio.

And so so it would be nice to start putting out those those videos.

But if you are just dying for some content as a reminder, the draft for the book is is online.

Right now there's thousands of people with access.

If you pre order the book or even just the e book, you get access, usually same day access to that draft, and so you can read it.

You can highlight ask questions in line and so on.

So anyway, in that draft is basically everything from at the moment from, you know, coding a neuron to actually full forward past back propagation, actually training the model on, then coming out very, very soon is testing regularization like l one l two regularization and drop out.

So that is about to be pushed, probably within a week or so to the draft.

So anyway, definitely check those out and you can get access.

Like I said right now, Otherwise the video is my hope is soon, but I really don't know.

I really wanted to take my time on those videos and make them good, so I really don't want to rush them.

But I'm thinking of maybe moving some things around so we can kind of do those sooner than I had hoped.

So planned.

So anyway, back to what we're gonna do here.

What I've got is there's kind of so what?

This is is a chap pot, but it's based on the Googles and Mt, which is neural machine translation.

The kind of concept of Mt.

Was, or is to convert from one language to another, which is a highly complex task, especially depending on the language.

So something like converting, you know, English input to German or something like that.

What I thought of doing a very long time ago, though was.

What if we use it to English?

English?

So an English comment to an English response, And it turned out that actually kind of works.

It's definitely a hack.

I don't think this is how it was intended to be done.

But this is kind of what I've been playing with honestly years.

And so I really should just do something else with my time.

But I'm very interested in this concept.

So s so.

What we've been working with is Reddit Data.

So it's Reddit comments in responses.

If you there is a chap, a territory, if you go to Python permanent and just type chap by.

This is the same thing.

Basically, it's the MMT chap.

Now there's a couple of tweaks, but that's what we're dealing with.

So what?

The model that I've been training is a 20 total layers.

It's 10 layers encoder 10 layer decoder on the 1024 nodes per layer.

So just absolutely massive model.

And I've been able to do that with a batch size of like, I think it's at 1 28 right now, thanks to the two Artie X eight thousands.

So So, yeah, that that's the model.

I've been training, but I just can't get like, the perfect model.

So, like, you know, at Lets 100,000 steps.

The model answer some questions really well, but then still failed other questions.

And then at 120,000 steps, let's say answers those other questions that it wasn't answering well but then kind of screws up some other questions.

So then I started thinking, What if I treated this more like an ensemble where you take many models and you kind of put them together?

So historically, I've used ensembles in, like, a sort of voting classifier sort of way where you don't say you're trying to predict a class of a thing.

Yeah, it could be nice to have just one model, but almost always you do way better if you're willing.

Tohave like five models all predict, and then they run a vote on which class they think is most likely.

This tends to actually perform better than anyone model will, ever.

So then I was wondering whom well, you know, this isn't really a prediction.

It's not like a regression, necessarily.

But what I could do is just take all of the possible outputs and then take their scores or make new scores and then figure out which one of, like, really, what?

We could wind up with hundreds of outputs.

Then we can pick one is surely one of them is a good one.

Right?

So let me show you what a single model looks like, and then we'll jump into the ensemble.

So the ensemble code I had enlisted the help of Daniel because based on what I really wanted to do is have all these models running live at the exact same time.

Take that input, spit out the output and then make it really simple for me to start working with it.

So there's really no one better.

Definitely not mean to do that task.

So Daniel wrote this chap ensemble code, which, until I guess I'll put some input here.

Hello?

See if we get No, we don't get any output yet, but we'll just wait for that.

It'll come.

It will come.

Maybe also have Well, actually missed.

All the errors are gonna be output in the ensemble Could, um the way he sent it, Daniel sent it over.

It was silencing all the errors, but then I think actually opened it back up.

So we'll we'll probably see the errors.

I'm just waiting to see, like, a real air, but I think we're all set here anyway.

So while we're waiting for the single model tow run, Um, What?

I'm gonna I cannot hear it.

It sounds so good.

I should share.

I love the sound of a GPU at work a few times I've shared it.

I used to have it as my channel intro, and people literally complained about it.

Um oh, no.

Am I doing tyrant?

I think I ran trained.

Yeah.

Whoops.

I think I must have run trained.

I recognize that too much.

I meant to run Inference.

I've been constantly running train.

I think I was proud of what I did.

Flips.

Let me, uh, check.

Make sure that cleared out.

Memory good enough.

Okay.

Python.

Uh, inference.

Stup.

Well, okay, let's try that again.

I should That should have been my flag anyway.

Now wait for it to load.

Um, And while that's going and I've been severely disrupted, this is what you get with an orangey video.

My apologies.

People seem to like the Kaga one.

So anyway, I can't remember what I was talking about before.

Uh, Okay, so here's an example.

Output.

I literally said hello.

And this is the output it gives.

Let me zoom in for you guys.

Let's see how good you couldn't see that pretty good.

Okay, Um, how are you?

So that's obviously a terrible output.

And so the outputs, basically each chat bought model has I want to say it's called being.

I forget what it stands for, but you'll get to paying on how big your beam width is.

I think again.

It's been so long since, like we've had so many abstractions on top of the original and MT.

Code that I honestly forget.

But basically the model output 10 responses.

And in general, I want to say that default one chosen by the model is always the top response and then the pointing at it.

You can't see what I'm pointing it.

This here is a score that I believe mostly, Daniel wrote.

And again, this was written probably two years ago or a year ago.

I don't remember how he was scoring them, but there's some problems with score and then again, just like?

Well, I'd liketo have is possibly multiple scoring mechanisms.

And those could also be ensemble type method.

So anyway, as you can see, that was no good.

Let's try.

How are you doing?

Okay, I'm good, Thanks.

That's pretty good one, but and then you can see these other ones.

They're color coded by how bad they are.

So So this is the best one.

And then these air all kind of pretty bad.

As you'll see, they're just not finished sentences.

It's I'm good.

I'm just trying.

That's I'm good.

I'm just, you know, these aren't done, so we could use packages.

I'm not too familiar with Spacey, but I don't know if I mean saying that, right, But anyway, this it's spelled Spacey.

Um, I could try that.

I just don't know enough about that packaged enough.

If I could do but I'm thinking of, but I'm sure I can, But I know it with an L t k.

I could use an lt k to know.

Is this a complete sentence?

Is it a coherent thought?

So that would be a really good way to score outputs my curiosity's help quickly.

I could do that, but that will probably be the next step after I do when I'm thinking of doing today, because there are far easier ways to judge these outputs.

So one way that I'm pretty confident is actually included in this Daniel score that I'm gonna call it for that one is Does it end with punctuation?

So none of these end with punctuation.

One thing that I'm not sure is included, but I would like to include is any repetition.

So if you'd say, if it's like I'm good and then I'm something else, like, generally, those are pretty bad, like, just from what I've seen.

But we can continue.

Uh, what's your favorite?

Blue eyes?

Black and white, Black and black.

Okay, um, what?

Your job on the software is here.

Okay, these are pretty good.

I mean, this one's doing pretty good.

This is the This will be the largest, the latest model.

So this is the one that has been training the longest.

So as time is going on, this one is actually had pretty good responses.

But one of the things, like so just happily having a valid response and then having like a really thorough response, are like two totally different things.

So some of the models have had longer responses that were just really interesting responses, like three.

Not always the same, I guess.

Like a lot of times, the chap wants to gear itself.

Torrents.

Very short, quick responses.

So that's another thing.

I would like to kind of have some sort of length value as well length, but without repetition.

Okay, so something like that again, I'm just going to be trialling airing again.

This is just pure RND.

But then here, you can see Yes.

So, like, the problem here is Yes, it's long, but it's all repetitive.

Like we don't want repetitive stuff.

Um, yeah.

So I think you get the idea.

Uh, so So this is just one model, and we could continue asking questions, but, you know, some of them are good.

Some of them are bad.

Um, what's your GP?

Yeah, So I mean, like, these aren't This is pretty good.

I mean, that's not bad at all.

And like, really a lot of these air.

Pretty good responses.

Try to think of like a decent where do you?

I'm in Singapore, but these are pretty good responses to be honest.

I haven't really played with this model too much, but I know there are certain outputs that are not that great.

So, like, for example, um, there's our ensemble.

Let me go into No, um, would be model.

And then usually we can load by the latest clips.

Not that way.

This way.

There we go.