Standardized Exams Cannot Measure Excellence
We would like to measure the fruits of our effort. We would like to gauge how our schools are doing. Thus, we have standardized exams that are supposed to be much more objective than grades given by teachers. Consistency unfortunately does not automatically translate to accuracy and there are obviously limits in how much information test scores can actually provide. First, a single exam given to all students has an intrinsic limitation in coverage. Consequently, these exams are often geared toward determining deficiencies and not mastery. It is indeed a huge mistake to equate standardized test scores with excellence. In fact, recent research has shown that a standardized exam like ACT (originally American College Testing) cannot reliably predict college graduation rates.
The above graphs show how college graduation rates correlate with high school grade point averages (left) and scores in the ACT exam (right). Each of the gray lines is a high school. The ACT graph has much less variability and its spread is especially narrow at the low end of the ACT scores. This suggests that the ACT exam, like other standardized exams, is written to spot deficiencies. This naturally comes from a significant number of questions in the test that only probe fundamental topics or skills in basic education. What is important to note is how the curves (almost for every high school) dives as one goes to higher range of ACT scores. The problem with any standardized exam is that preparation can significantly alter the score. Students can enroll in classes designed specifically to prepare for the ACT test. Of course, some students can do well if the preparation matches the exam. An excellent performance in ACT thus measures either privilege, access or resources and begins to perform poorly in predicting college graduation rates. At the high end, the ACT exam is measuring something else. And I drew a red curve to emphasize this trend. It is amazing that almost each and every gray line shows this trend.
ACT does have something to say regarding this recent research but it is definitely clear that using standardized test scores on an individual basis is deceptive.
Tests should never be intended for ranking, sorting or selecting our children. Tests are assessments. These should only inform instruction. And we must keep this in mind especially when dealing with standardized tests. Here is a short video from Ken Williams reminding us, educators, that we must stop labeling our children.
 |
| Above copied from Allensworth, E. M., & Clark, K. (2020). High School GPAs and ACT Scores as Predictors of College Completion: Examining Assumptions About Consistency Across High Schools. Educational Researcher. https://doi.org/10.3102/0013189X20902110 |
The above graphs show how college graduation rates correlate with high school grade point averages (left) and scores in the ACT exam (right). Each of the gray lines is a high school. The ACT graph has much less variability and its spread is especially narrow at the low end of the ACT scores. This suggests that the ACT exam, like other standardized exams, is written to spot deficiencies. This naturally comes from a significant number of questions in the test that only probe fundamental topics or skills in basic education. What is important to note is how the curves (almost for every high school) dives as one goes to higher range of ACT scores. The problem with any standardized exam is that preparation can significantly alter the score. Students can enroll in classes designed specifically to prepare for the ACT test. Of course, some students can do well if the preparation matches the exam. An excellent performance in ACT thus measures either privilege, access or resources and begins to perform poorly in predicting college graduation rates. At the high end, the ACT exam is measuring something else. And I drew a red curve to emphasize this trend. It is amazing that almost each and every gray line shows this trend.
ACT does have something to say regarding this recent research but it is definitely clear that using standardized test scores on an individual basis is deceptive.
Tests should never be intended for ranking, sorting or selecting our children. Tests are assessments. These should only inform instruction. And we must keep this in mind especially when dealing with standardized tests. Here is a short video from Ken Williams reminding us, educators, that we must stop labeling our children.
Comments
Post a Comment