"Bear in mind that the wonderful things you learn in your schools are the work of many generations, produced by enthusiastic effort and infinite labor in every country of the world. All this is put into your hands as your inheritance in order that you may receive it, honor it, add to it, and one day faithfully hand it to your children. Thus do we mortals achieve immortality in the permanent things which we create in common." - Albert Einstein

Friday, January 2, 2015

Does Retention Really Help?

Great caution is necessary to address whether requiring students to repeat a grade or subject is helpful or harmful. A correct analysis demands adequate attention to the multivariate character of learning outcomes. First, there is the statistical phenomenon called regression to the mean. This is nicely explained in Social Research Methods via the following:

Measuring the effects of retention would naturally involve performance before and after retention. Regression to the mean alone would move the average of low performers higher and closer to the mean on the post test and vice versa, move the average of high performers lower and closer to the mean on the post test. With the above in mind, let us take a look at a study on how retention in algebra affects student's performance in a large high school district in California:

Lower performing students experience improvements while higher performing students experience declines. These findings should sound quite similar to regression to the mean. Statistics is indeed tricky. The way the above findings are stated makes retention look more favorable, but digging deeper into the study and looking at the actual data shows a much less favorable outlook:

 "F" and "D" students improve their grades (0.94 - equivalent to improvement by one letter grade) while "B" and "A" decline (-0.93 - equivalent to a lowering by one letter grade). Everyone is simply moving to the mean! About 400 students did not pass Algebra the first time in this school district. After retaking Algebra, 541 students are still between "F" and "D", about 600 are still scoring "below basic" in an algebra standardized test, with 160 scoring "far below basic".

A more accurate presentation of the above study is therefore the one below (from Jill Barshay at Hechinger Report):

To avoid pitfalls of statistics, it is often helpful to avoid correlation studies and simply directly interrogate how a particular intervention may in fact influence learning. When a student fails in a subject, there is often a reason, perhaps even a number of reasons. A student who has not mastered arithmetic often finds algebra extremely challenging. In this case, for example, giving such student a repeated dose of algebra does not really address the problem.

No comments:

Post a Comment