Does Retention Really Help?
Measuring the effects of retention would naturally involve performance before and after retention. Regression to the mean alone would move the average of low performers higher and closer to the mean on the post test and vice versa, move the average of high performers lower and closer to the mean on the post test. With the above in mind, let us take a look at a study on how retention in algebra affects student's performance in a large high school district in California:
Lower performing students experience improvements while higher performing students experience declines. These findings should sound quite similar to regression to the mean. Statistics is indeed tricky. The way the above findings are stated makes retention look more favorable, but digging deeper into the study and looking at the actual data shows a much less favorable outlook:
A more accurate presentation of the above study is therefore the one below (from Jill Barshay at Hechinger Report):
To avoid pitfalls of statistics, it is often helpful to avoid correlation studies and simply directly interrogate how a particular intervention may in fact influence learning. When a student fails in a subject, there is often a reason, perhaps even a number of reasons. A student who has not mastered arithmetic often finds algebra extremely challenging. In this case, for example, giving such student a repeated dose of algebra does not really address the problem.