"Bear in mind that the wonderful things you learn in your schools are the work of many generations, produced by enthusiastic effort and infinite labor in every country of the world. All this is put into your hands as your inheritance in order that you may receive it, honor it, add to it, and one day faithfully hand it to your children. Thus do we mortals achieve immortality in the permanent things which we create in common." - Albert Einstein

Monday, May 2, 2016

What's In A Score?

Students take exams. This is one way of gauging how much a student has learned. In addition to specific assessment tools that a teacher may use in his or her class, there are standardized tests. In the United States, there is the National Assessment of Educational Progress while in the Philippines, schools have the National Achievement Test. These standardizes exams are indeed powerful tools but one must understand how to use scores in these exams appropriately.

More than ten years ago, W. James Popham wrote in Educational Leadership an article entitled "Why Standardized Tests Don't Measure Educational Quality". Popham pointed out one reason why extra care must be taken in interpreting scores from a standardized exam: These exams were not just testing what a student had learned. Scores on this exam were really determined by at least three factors:
  1. what was taught in school, 
  2. a student's native intellectual ability, and 
  3. a student's out-of-school learning.
Popham also likened standardized exam to measuring temperature with a tablespoon. The scope of a standardized exam would always be limited. Popham did cite that test developers chose questions based on how discriminating the items were. Thus, only those questions that could be answered by about half of the test takers often managed to survive in the final version of a standardized exam. After all, the main point of a standardized exam was to spread and differentiate between students. 

With the above in mind, it clearly becomes obvious that comparisons similar to the ones shown below are really pointless.

Above copied from The Nation's Report Card
There is really nothing meaningful in stating that the 2015 score in Mathematics is lower than the score two years ago for one good reason. This exam is not really that precise or accurate. 

Yet, even with standardized exams that are quite imprecise, incomplete and inaccurate, there are useful pieces of information one could get from the scores. The fact that the fourth grade students from the Philippines scored near the bottom of the science exam in the Trends in International Mathematics and Science Study (TIMSS) 2003 is not something that can be dismissed.

Above copied from
Using a tablespoon cannot tell us the temperature within the nearest degree on a Celsius scale, but it sure can be used to find if something is hot or cold. When the average score is too low and the spread is too wide (in the case of the Philippines), it speaks volume regarding the quality of science education in the country as well as the degree of inequity in the educational system.

The Nation's Report Card just released a few weeks ago in the United States may not tell us so much how schools are doing compared to two years ago, but the scores still tell an important story. That consequential story is seen when scores are cast in the following fashion:


Above copied from The Nation's Report Card


Above copied from The Nation's Report Card

Racial gaps remain in schools in the United States. There is still a gender gap in reading. The level of education parents have are strongly correlated with scores in these exams. These are much more salient points. For one, it clearly shows why standardizes exam scores should not be used to determine teachers' salaries.

No comments:

Post a Comment