0:04

Hi! In this module,

Â I'm going to introduce the phenomenon of regression to the mean.

Â Regression to the mean is a phenomenon that you may have heard about perhaps in

Â the context of the discussion of parents and their offspring.

Â It turns out that, you know,

Â parents that are extraordinary on some measure,

Â unfortunately for them, their children,

Â on average, will be less extraordinary than they were.

Â They will be closer to the mean than they were.

Â For example, the children of parents with very high IQ, on average,

Â will be closer to the mean IQ of their generation than their parents were.

Â If average IQ isn't changing over time, on average,

Â the offspring of these high IQ parents will actually have,

Â again on average, lower IQ than their parents did.

Â And this applies to almost anything that you can measure if you're talking about

Â the association from one generation to the next in something where,

Â in the first generation,

Â we're looking at people that were out at the end of a distribution,

Â out at the tail of a distribution.

Â On average, for example,

Â if you think about high-income parents,

Â on average, their children will have incomes that are closer

Â to the mean of their generation than their parents did.

Â Now, this phenomenon of regression to the mean

Â actually shows up in a lot of different places in sometimes unexpected ways.

Â And so, I want to sensitize you to it

Â and talk about some of the ways we have to

Â think about it when we're designing a research study.

Â So let's look in detail about what it is.

Â So, let's go back and think about what we

Â talked about with regard to correlations in the previous module.

Â Correlations are always between -1 and one.

Â When we talked about standardized scores or z-scores,

Â we learned that this means that a change

Â of one standard deviation in one variable must always lead to

Â a change in the other variable that is

Â less than one standard deviation or greater than -1 standard deviation.

Â In other words, a one standard deviation change in one variable can't be

Â associated with a change of more than one standard deviation in another variable.

Â So, a one standard deviation change in one variable must be associated with

Â a change in the other variable of between -1 and one standard deviations.

Â In the example in our previous module,

Â these two hypothetical variables for the example that we constructed,

Â a one standard deviation change in X was

Â associated with a 0.79 standard deviation change in Y.

Â Now, that may sound very technical,

Â so we have to walk through how that relates to the examples that we just talked about,

Â the IQ of parents and the IQ of children and so forth.

Â Now, when we go back from standard deviations

Â and standardized scores to actual original values,

Â the implication is that if we associate two variables through correlation,

Â if we pick one of those variables and we pick one of the observations, on average,

Â the value for the other variable in that observation,

Â on average, will be closer to the mean of its distribution.

Â In this case, on average,

Â if the observation is one standard deviation away from the mean for X,

Â on average, it will just be 0.79 standard deviations away from the mean on Y.

Â Now, this has some subtle implications

Â for research design that you have to think about carefully.

Â And in particular, it has a specific implication for studies that use samples or

Â that look at populations that have been selected for extreme values on some distribution.

Â This is more common than you might think.

Â Imagine that in our example, our made-up example,

Â X and Y are test scores at two points in time.

Â So each observation is one person,

Â we measure their test scores at one point in

Â time â€“ that's X â€“ and then six months or a year later,

Â we test them again and we record their score.

Â And the correlation between these two is 0.79.

Â Now, again, we're gonna assume that the correlation is 0.79,

Â so one standard deviation change in one of the variables actually leads to a

Â 0.79 change in the standard deviation of the other variable.

Â So imagine, as an example,

Â that we seek to test an intervention for poorly performing students.

Â I'm gonna walk through this example to show you some of the unusual ways in which,

Â again, the regression to the mean can insert itself or creep into our research.

Â So, again, we're gonna test an intervention to help out poorly performing students.

Â And we'll assume that the hypothetical data that we've been using already

Â represents the test scores of these students at two points in time,

Â time one and time two,

Â and that, in general,

Â test scores for the same person across time are correlated at the 0.79-level.

Â So we, because we're interested in

Â helping these students that are performing the least well,

Â we select students whose scores the first time we administer

Â the test are more than 0.5 standard deviations below the mean.

Â So we're picking the students at the bottom of the distribution,

Â the lower tail, because they're the ones that we want to help.

Â Their average score, if we think about it,

Â is gonna be one standard deviation below the mean.

Â So, in effect, I'm talking here in terms of

Â z-scores or standardized scores that we talked about in the last module;

Â we're talking in terms of standard deviations above or below the mean.

Â So we take these students,

Â the ones that are not performing well,

Â and then we conduct our intervention â€“ perhaps it's a new curriculum or something else.

Â So we'll call Y the score that we measure at time two.

Â Well, if test scores, in general,

Â are correlated at the 0.79-level across time,

Â just because people, even though their IQ isn't changing,

Â how well they do on a test may bounce around because on one day they may feel better,

Â one day they may not feel as well, etc.

Â So their scores may bounce around,

Â be correlated, at again,

Â an assumed level of 0.79 even if at some level their IQ,

Â their ability, nothing else is changing.

Â So we go and measure this group of students that we

Â selected because they were more than 0.5 standard deviations below the mean.

Â Go back and measure them after the completion of the intervention.

Â On average, actually even if the intervention did nothing,

Â the mean of their test scores,

Â that is the test scores at time two,

Â the Y in this analysis,

Â will be 0.79 standard deviations below the mean,

Â so it will actually be 0.21 higher than it was the first time we administered the test,

Â even if at some level the intervention achieved nothing.

Â So this is an important point that relates to regression to the mean.

Â If you have before-and-after comparisons where

Â you have a population that you're studying that you

Â selected because it had some sort of extreme value on some measure

Â that you're interested in â€“ perhaps test scores

Â or could be heart rate or cholesterol levels.

Â You measure them again at some future point in time.

Â So long as the correlation in the measurements are not one,

Â you're almost guaranteed that

Â those people's averages the second time you measure them will

Â be closer to the mean even if your intervention achieved nothing.

Â So here, as we just showed you,

Â even in the absence of an effective intervention,

Â if an intervention did nothing at all,

Â we would expect just based on the phenomenon of regression to the mean and, say,

Â the assumed correlation in test scores from one time to another,

Â that people that were initially selected for being at

Â the bottom of the distribution in terms of test scores.

Â Measure them again a few months later,

Â a few years later;

Â on average, they're gonna be closer to the mean.

Â In other words, their average score will be

Â higher than it was the first time we measured them.

Â So, we'll go from an average of -1 standard deviations to -0.79 standard deviations.

Â To recap, an apparent improvement,

Â even if actually the intervention didn't work or we did nothing at all, of 0.21.

Â Now, I'll go back to some examples of other ways that this creeps in,

Â but I'll talk about why we have to deal with

Â this especially if we're conducting some sort of intervention study,

Â an experiment where we're selecting people into a sample based on,

Â again, their extreme values â€“ in this case, test scores.

Â So, this means that even for a before-after comparison,

Â when we want to test some kind of intervention,

Â we'll still need a control and a treatment design so that we can

Â compare the trajectories of the people that

Â received the intervention and the people that did not.

Â So, if we divide our group in the figure â€“ we've identified the people in

Â our treatment group with circles around

Â their dots and then the control group is left without circles,

Â so we've divided the original students into

Â two groups at random â€“ then we can follow them up.

Â We can give the intervention,

Â the teaching intervention, to the treatment group,

Â leave the control group alone,

Â and then six months or a year later we can test them again.

Â If the intervention has an effect,

Â it'll be apparent in a difference between the control and the treatment groups.

Â They will have differing trajectories.

Â So the control group should see

Â some improvement in scores just because of regression to the mean.

Â If the intervention works,

Â we should see a larger or better improvement in scores.

Â If the intervention is ineffective,

Â the change in the scores in the treatment group will be the same as in the control group.

Â So, again, if they differ,

Â the scores at time two differ between control and treatment,

Â then we have an effect.

Â So, where does this come in?

Â Well, there are a lot of settings where a simple before-and-after comparison

Â will suggest that an intervention or a policy change had an effect,

Â especially again if the population for the study was selected based on extreme values.

Â This will be because of regression to the mean.

Â Let me give you some common examples.

Â One is medical treatments.

Â So, on average, if you think about somebody's medical condition,

Â perhaps how well they feel,

Â whether or not they're experiencing pain.

Â If you measure people at different points in time in

Â terms of how much pain they are feeling,

Â it will not be perfectly correlated.

Â There will be some value,

Â some correlation less than one,

Â that the amount of pain that they report will vary from one time to another.

Â So if you think that you've got some new way of helping people with pain and

Â you select a bunch of people that report being

Â in great pain at a particular point in time,

Â on average, if you look at them six months later,

Â 12 months later, on average,

Â that group that was selected for having, again,

Â extreme values will probably be better off than they

Â were at the first point in time because of regression to the mean.

Â There may be new people that are experiencing more pain,

Â but the original group that we selected

Â for their presence in the tail of the distribution,

Â will probably be better off than they were.

Â So if we did anything in that time,

Â it will appear to have an effect.

Â That's why medical studies that test

Â interventions can't just be before-and-after comparisons.

Â If we're testing some relief for pain,

Â we have to take the people that are suffering at time one,

Â divide them into two groups,

Â leave one of them alone, and then administer the treatment to the other.

Â The reason I'm singling out medical treatments here is because they're

Â a rich source of examples of this sort of problem because, of course,

Â most of the time when we want to find a treatment for somebody,

Â we're doing so because people are at the extreme of some distribution on

Â some measure that we care about â€“

Â their blood pressure is very high or their cholesterol is very high.

Â These are all variables that are not

Â perfectly correlated from one measurement to another.

Â They are correlated well below the level of -1 or one;

Â they're much closer to zero than that.

Â So again, if you are trying to test some intervention for high blood pressure

Â and you start out by selecting a bunch of

Â people with very high blood pressure at a particular point in time,

Â and then actually even if you do nothing at all,

Â but you measure their average blood pressure six months later,

Â on average, they'll be closer to the mean blood pressure.

Â So it'll look like anything that you did worked

Â unless you have adopted a control and treatment design.

Â Another way that regression to the mean creeps

Â in is with people assessing the effects of policy changes.

Â Policy changes, often like medical treatments,

Â are not distributed at random.

Â They often occur when people feel that a particular phenomenon,

Â a social phenomenon, is out of control or at some extreme.

Â So, people introduce new laws to prevent crime when crime rates are high.

Â If crime rates are low or just typical,

Â why do anything about it?

Â So, extremes in the prevalence of some phenomenon that concerns

Â people tend to trigger the interventions via policy and so forth.

Â So if you think about it, if you think about the, for example,

Â the crime rate as a variable that if you measure,

Â at multiple points in time, it drifts around.

Â It's correlated and it's not correlated at one or -1,

Â might be correlated at some much lower level.

Â So, if you think about the crime rate at some point in

Â time â€“ and it's an extreme value within

Â the overall distribution of crime rates and this is what led people

Â to panic and introduce new laws â€“ on average,

Â if you measure the crime rate at some later point in time,

Â even if at some level the intervention did nothing, on average,

Â the crime rate will probably be closer to the mean, the long-term mean,

Â for the crime rate, and an intervention may appear to have an effect.

Â Now, it's much harder to do control-treatment designs for things like policy changes,

Â and so we end up simply with arguments about whether or not a policy induced

Â some change or whether it was simply an example of regression to the mean.

Â Now, in some cases,

Â economists and others do look for,

Â as we talked about in previous lectures,

Â natural experiments â€“ situations where policies were

Â changed at different times in different places and so forth.

Â Those sometimes can give some insight into cause and effect and

Â get us away from the problems with regression to the mean.

Â Now, of course, the other example I would like to draw to

Â your attention is educational interventions,

Â like in the example that I just gave you.

Â So, we're all interested in finding ways to make

Â things easier or make things better for our least well-performing students.

Â And so if we conduct studies where we select students because of their lower performance,

Â their poorer performance, on average,

Â they're gonna get better anyway next time we measure them.

Â So we have to think about how to do something about that,

Â most likely with a control and treatment design.

Â That's a little easier to do in an educational setting.

Â Now the flip side of this is is that we're trying to introduce,

Â say, intervention to help out gifted kids,

Â where we pick a bunch of kids that are at the top of the distribution,

Â and we think we're gonna do some kind of enrichment that will help them even further.

Â Well, I hate to break the news but, on average,

Â it's always gonna look like it's gonna make the kids do less well because,

Â on average, their scores are probably gonna be

Â closer to the mean next time you measure them.

Â And it's not necessarily a reason to be disappointed in the intervention,

Â it just means, it means that there is a need for a more carefully designed study,

Â again, a control and a treatment.

Â So, overall, I hope that I've convinced you that regression to the mean,

Â even though it sounds somewhat paradoxical and it's related

Â to the structure of correlations,

Â is actually an important phenomenon that you have to

Â think about when you're designing a study

Â or interpreting the results of other studies presented to you by other researchers.

Â So keep regression to the mean in mind especially if you're

Â trying to assess the effects of interventions or other changes.

Â