Naughty Or Nice?

Naughty or nice? It is not that easy when it comes to evaluating teachers.

Early this year, the Philippines' Department of Education (DepEd) through Order No. 12 s. 2013 established a performance-based bonus system for public school teachers. This system is based primarily on students' test scores in the National Achievement Exam, the school's dropout rates, and proper liquidation of maintenance and other operating expenses. Perhaps unknown to DepEd officials, evaluating a teacher's performance using students' test scores is very much prone to error. Here are the reasons why such evaluation system is flawed.

Weeks later after the DepEd issued the order, a lecture was given by distinguished psychometrician Edward H. Haertel of Stanford University. DepEd should have listened to this lecture. The lecture can be nicely summarized in the following figures (All of these are copied from RELIABILITY AND VALIDITY OF INFERENCES ABOUT TEACHERS BASED ON STUDENT TEST SCORES):

First, research has shown that albeit teacher effectiveness does impact learning. The difference between a good and a poor teacher does not really influence students' test scores that much. Out-of-school factors such as socio-economic status have a much greater effect on students' performance. Teachers are therefore being evaluated on something they are hardly in control and that is simply unfair.

Second, students' test scores are not on a linear scale. For example a 10 point gain associated with an improvement from 90% to 100% is much more difficult compared to an increase from 50-60%. The former requires that a student gets all the answers correct, not even one mistake. 

Third, year over year comparisons are likewise prone to errors because there is learning loss during school breaks. This loss has nothing to do with what the teacher does. In this case, out-of-school factors are of greater significance.

Lastly, the scores are not stable. This is illustrated by data from elementary schools in five different counties in Florida. In the following figure, 10 percent of the lowest scoring teachers came from the top scoring ones a year ago, and vice versa, 10 percent of the highest scoring teachers came from the bottom quintile in the previous year. The scores are very unstable, which should not be surprising since the measures being used are not really valid for evaluating teachers.

This is what research is clearly saying. Yet, DepEd continues to ignore what teachers' groups like the Alliance of Concerned Teachers have been saying:

The Performance Based Bonus of Aquino really belongs to only one place, the trash.