Do We Know How to Evaluate Teaching
|How my students evaluated me|
|Above copied from Studies in Educational Evaluation|
Student course evaluations. There are literally thousands of articles on student course evaluations, and within them one can find supporting evidence for all possible opinions. Student evaluations do provide useful information: They are often used to flag instructors who have anomalously low scores in order to understand and address the reasons for the negative student opinions, which is appropriate. However, they have several critical failings with regard to the criteria listed above that should prevent them from serving as the primary method for evaluating the quality of teaching.
First, they have some fundamental limitations that transcend any details. It is impossible for a student (or anyone else) to judge the effectiveness of an instructional practice except by comparing it with others that they have already experienced. If all they have experienced are lectures, they cannot meaningfully evaluate the effectiveness of lectures relative to other practices. This prevents student evaluations from encouraging or rewarding the adoption of more effective research-based teaching methods when such practices are seldom used at an institution.The criteria Wieman listed are validity, meaningful comparisons, fairness, practicality, and chance for improvement. The validity criterion is clearly not met as Wiemen pointed out, "There is another basic limitation of student evaluations that ask how much was learned in the course, which most do. People are poor at evaluating their own learning, because it is difficult to know what you do not know. The accuracy of this evaluation is also sensitive to the level of expertise of the respondent."
Wieman therefore proposes with Sarah Gilbert a more effective way of evaluating teaching. This proposal is published in CBE Life Sciences Education. This evaluation scheme basically looks at practices implemented by an instructor and gives points to those practices that have been shown by research as beneficial to student learning.
Table 1. Teaching practices inventory categoriesThe following are examples provided by Wieman and Gilbert:
I. Course information provided (including learning goals or
II. Supporting materials provided
III. In-class features and activities
V. Feedback and testing
VI. Other (diagnostics, pre–post testing, new methods with
VII. Training and guidance of TAs
VIII. Collaboration or sharing in teaching
Looking at the above table, one can already anticipate a problem. The third column lists references that demonstrate benefits of a particular teaching practice. This is research. The main problem is that there is a huge disconnect between research and practice when it comes to student evaluation of teaching. How much more if one considers the entire field of teaching practices.
Thus, for years to come, we can only expect student evaluation of teaching to continue....