Um, it's really you could consider that component of industrial organizational psychology, but it's, ah, component that has a strong emphasis on individual differences, and so you could consider it first, an exploration of the practical utility of psychological concepts in predicting real world phenomena.

But he could also consider it a part of the validation of the psychometric models of personality that we've described because one of the things that you want to do to determine whether or not a construct So it's a hype of the hypothetical psychological phenomenon or entity whether or not that's riel, depends to some degree on what it's good for right, part of the aim of sciences, prediction and control.

And so it's useful to take a look at important life outcomes among human beings and to determine whether or not you can predict them.

Because that's one way of testing.

Whether or not the phenomena that you've derived, say, from your statistical analysis, both in the case of fluid intelligence or intelligence more broadly and in the case of the Big Five actually has the capacity to manifest itself in a meaningful way in in the world outside of inside the lab.

That's that's one domain of potential validation, but then outside the lab as well.

And so what we're gonna do today is to look at the factors that determine performance in the real world.

And so you might think, well, what are some important real world outcomes on which performance might vary?

And we're going to stick to those that are rather obvious today?

We talked about creative achievement already, for example, but you could think of creative achievement, entrepreneurial achievement, those actually clumped together.

They're part of the same phenomena you think about academic achievement, and you could think about success in the workplace.

And then those could be subdivided because success in the workplace, for example, needs to be analyzed in terms of success, say, at simple jobs and success that complex jobs like there's different ways of fraction ing fraction ating ah phenomena to increase your capacity to understand it and to measure it and then to predict it.

But those air rough domains, where there's self evident individual differences and where the capacity to predict performance would at least be of some utility performance in the workplace.

This is an interesting one.

Performance in the workplace is a is a tricky measure, a tricky issue.

I've been working on performance prediction tests for a long time because I was interested in because I'm a clinical psychologist and because I have a practical side, I'm often interested in what the utility of psychological measurements might be outside the lab.

It's interesting to consider that from an ethical perspective, too.

So if you're hiring people, you have to conundrums that lay themselves in front of you.

And one is tthe e ethical necessity to give each person a fair chance, and the other is the ethical necessity to place the proper person in the proper position.

And, you might say, Well, you could do that randomly and like There are countries in Europe, for example, that have a quasi random approach to the selection of university students.

So Holland is like that they have a pretty open admission policy now.

What that means is that everyone has a chance to go to college.

But the down side of it is that the failure rate in the first year, for example, is extraordinarily high now, you might say that's a perfectly reasonable price to pay for the open admission and for the opportunity to give everyone a chance.

But you could also say, Well, that's a hell of a waste of time for the people who go into the first year of college or university and fail.

It's not a pleasant experience for them.

It's It's very expensive in terms of times, time and resources.

And perhaps it would have been better for them and for broader society had they been able to determine beforehand whether they had two sets of of qualities that were necessary to increase the probability of success.

Because if I could tell you well, you know, you have, ah, 80% chance of success in this domain and only a 20% chance in this domain, you still might want to take the risk in the 20% success domain.

But you might also think, well, I might as well go off and function where my particular combination of Proclivities has the best opportunity to manifest itself and because why not position yourself for success rather than failure?

Now?

I'm not saying that the ethical conundrum between those two alternatives is something that's easy to Uh uh, What would you say?

An easy thing to map your way through.

It's by no means an easy way to map your way through because there's our strong arguments to be had on both sides of the equation.

So but, um, there's still the scientific question that remains.

Which is to what degree can you accurately predict people's performance?

And to what degree does that reflect positively or negatively on your psychological concepts?

And then there's the actual practical utility of potentially offering to schools, to universities and to work places in general, the probability of selecting people with an above average chance of succeeding.

It's even more complicated, say, if you're selecting not so much entry level employees in a company, because maybe you tell yourself there more towards the possibility of bringing more people in and letting them fail or succeed in the job.

But let's say you're replacing a management team at a medium to large size corporation.

You know, if you bring in managers that are incompetent, not only are they going to fail, which is obviously not very good for them, and the failure rate among managers is very high.

It exceeds.

It exceeds 65.

They figure.

I think it's 65% of managers.

I believe that's correct at zero or negative net value to the company's.

It's something like that.

The failure rate managerial positions is overwhelming.

And the problem with bringing someone into a managerial or an executive position even worse, who isn't competent to play that role is that they can wipe out the careers of everyone that there supervising and, in the case of, say, large companies that could bring the whole damn company doubt.

And so it's not like the the ethic of people deserve an equal chance.

Let's say to fail and succeed isn't very practical ethics when you're putting someone in a high demand position, where the consequences of failure are can be overwhelming not only for that person, but for all the people that they happen to be whose destiny they happen to be involved in determining.

And so then you might say, if you could come up with this election process, that would increase the probability of hiring an above average manager from 50 50 to 60 40 or 70 30 maybe you're actually ethically compelled to use that.

And in fact, by law, this is particular to the case in the United States.

You're required by law to use the most effective, valid, reliable and non discriminatory selection process that currently exists in order to select your employees.

And one of the things that's going to happen to employers in the next 10 years is they're going to get a very nasty shock for using interviews, because the data on unstructured interviews indicates quite clearly that a that they're discriminatory partly because they, for example, if you're tall and, uh, assertive and good looking and charming, then you're much more likely to do well in an interview.

But that doesn't necessarily have any bearing whatsoever on the probability that you succeed in the position.

And so, and not only that, the predictive validity oven structured interviews is very low.

It's about 0.12 It's something like that, which means that if you just pick people randomly for success or failure, let's say you'd have a 50 50 chance of predicting whether someone was going to be a success or a failure is just a coin toss.

And then, if used in unstructured interview, you'd get that up to 56 44 which is slightly better, but it's by no means it's not even close to the accuracy that you could get.

For example, if you just used to standard test of conscientiousness, which would give you a correlation of about 0.25 with with, say, manager with managerial productivity.

And there's this interesting, this is an interesting thing to know.

It's called the binomial effect.

Size display effect sizes the magnitude of in effect, right?

And and it's not an easy thing to get a handle on all that.

You really need to if you're gonna be a psychologist, because in any study there's an effect size indicator correlation off often Orin R.

Squared, which is the correlation squared, or cones D, which is the effects eyes expressed in standard deviations or something like that.

But you kind of have to understand that at a basic level, to understand what statistics actually do.

And there's this phenomenon called the binomial effect size display that can help you understand what, like in an embodied sense, what the magnitude of a correlation means.

So here's how it works.

Imagine that you have a predictor of 0.20 So the correlation is our equals 0.20 between phenomena, one will say conscientiousness and phenomena to workplace performance 10.20 correlation.

The question might be, Well, how much?

How much would you improve your predictive capacity over chance levels if you applied that predictor?

And the answer is that we are is the difference between the odds ratio.

So let me explain that so 0.50 point 50 If you subtract one from the another, you get zero.

So the predictability of selection by chance is 0.50 minus 0.50 equals zero.

That's the predictive validity of chance.

If you have a predictor of 0.20 which is approximately, that's sort of the low end estimate for conscientiousness, then that would change your odds ratio from 0.50 point 50 right random 2.60 point 40 because 400.60 minus 0.40 is 0.20 And so the correlation coefficient turns out to be the the difference between the odds between the odds, so and so it gives you a quick rule of thumb.

So, for example, so if you have a 0.20 predictor that gives you 60 40.

We have a 400.30 predictor that gives you 65 35 because 350.65 minus 0.35 point 30 And if you have a 0.6 predictor, which is really up on the high end, right, you really start to push your limits of statistical predict prediction validity.

At that point, that gives you 0.0.80 minus 0.20 And so what you have done if you use a predictive, uh, predictor that has a correlation coefficient of 0.60 which you could get, for example, if you took conscientiousness and combined that with a good test of I.

Q.

For predicting complex jobs, you might be able to get up 2.6.

That moves your odds ratio of selecting an above average person for the position from 0.50 point 502.80 point 20 So it cuts here.

Your failure rate by more than half right brings it down from 200.50 point 20 because 0.80 minus 0.20 point 60 So that's a really good thing to know that's called the binomial effect size display.

It's really good thing to have in your mind.

It's very simple.

It's just a it's just subtraction, and it gives you some sense of the power off of of statistical predictions.

Now the question might be, Well, let's say you had a predictor of 0.20 conscientiousness, you might say.

Well, if you square they are, that gives you 4% of the variance.

Who the hell cares?

4% of the variance you've left 95% of the variability between people in terms of their performance.

Unexplained, you might say.

Well, why even bother?

Well, the answer to that question is how much difference in productive output is there between people.

Because if there's a tremendous degree of productive of difference in productive output between people, then increasing your ability to predict someone's performance even by some relatively small increments might have massive economic utility.

You know, if if, let's say the top 10% of your people are 50 times as productive as the bottom 10% of your people than shifting your ability to predict up so that you have more of those extremely high performing people or less of the extremely low performing people might more than pay off might more than pay for itself from an economic perspective, even though your prediction your predictor isn't doesn't have that massive amount of power well, and that actually happens to be the case.

So back in 1968 there was a guy named Walter Michelle and he had reviewed he's a social psychologist.

He reviewed the personality leader to up to that point and concluded that the typical personality measure on Lee predicted the typical performance measure at about 0.2, and that's actually remained relatively stable.

I would say it's a little higher than that's probably 0.25 especially if you do things like correct for measurement, error and so forth.

And what Michelle said was because it's only 0.25 Let's say you square that that's 5% of the various.

You leave 95% of the phenomenon unexplained.

You might as well not even bother measuring personality.

And so that actually killed the field of personality from a second metric perspective for about 25 years, really until about the early 19 nineties, when people woke up and thought, Wait a minute, What are the typical effect?

Sizes in other domains of prediction?

And then they found out that while the 0.20 correlation that was typical of personality prediction was actually pretty damn good bye social sciences or health sciences standards.

But it doesn't sound good when you just think about it as an absolute measure because it leaves 95% of the phenomenon unexplained.

But when you compare it to other things that people consider of reasonable magnitude than it turns out that personality psychologists are doing just fine.

And then also in the 19 nineties.

And I'll show you some of this, their economic calculations done.

And so one of the calculations would be, well, imagine that you have.

Maybe it took 20 companies, and you did it.

Ah, distribution of the productivity of their employees.

It's hard thing to do because you have to measure their productivity.

How the hell do you do that right?

With sales people, you can measure sales that's pretty straightforward.

With lawyers, you can measure hours billed like there's some.

There are some occupations where the performance measure is sort of built into the job But if you're a manager in the mid level of a large corporation, how the hell can you tell how productive you are?

It's so there's a measurement problem on the productive ity measurement and as well as the performance prediction end.

And it's a very intractable problem.

And the way that people often do that is by saying, Well, let's say you're a manager in the mid level of a corporation.

How do we determine how productive we are?

You are.

Well, we might ask you to compare your we're work productivity with your peers, maybe construct up a questionnaire asking about your efficient use of time and so forth, and then we might get your peers to do the same thing to you.

We might get your supervisors to do the same thing to you, and we might get your your subordinates to do the same thing to you and then aggregate across all those measures and in for that.

That aggregate opinion actually constitutes a valid measure of productivity.

You actually don't know right, because that assumes that what you're doing that your peers and your supervisors and so forth our rating is actually related in some positive manner to the bottom line of the company, and you actually don't you actually can't figure that out?

This is actually I think, why Why large companies start to become unstable is because if there's enough layers between the operations of the people in the tears of the corporation and the rial outcome measure, which is basically profit because that's what we've got, then the relationship between your activity as a manager and the productivity of the company starts to become increasingly blurred.

And that might mean that you're working as hard as you can on something that's actually going to cost the company money.

So you would actually be much more productive from a profit perspective if you just didn't go to work at all.

And that that happens a lot in large corporations because you never know, especially if there's a lot of steps that have to be undertaken in a process.

Before you contest the product in the market, you have no idea if you're wasting time and resources, you just can't tell.

So the performance measurement issue is a very, very complicated one.

We haven't talked about it that much, but I give you kind of a brief overview of it.

Now what you really want to do is have multiple sources of information about performance and aggregate across them.