Classroom Instruction and Tutoring, Human versus Machine

Annie Murphy Paul recently wrote an article in the New York Times (NYT) entitled "The Machines are Taking Over". The article talked about the works of a husband and wife team, Neil Heffernan, a professor of computer science at Worcester Polytechnic Institute, and Cristina Lindquist Heffernan, a mathematics middle school teacher. Their program called Assistments is an online tutoring platform that provides immediate feedback to teachers, students and school administrators.

Video introducing the Assistments online tutoring program

Paul's NYT article starts with the time when Neil and Cristina were just engaged. Neil was a graduate student at Carnegie Mellon studying ways in which software could be used to help students while Cristina was a successful math tutor. Neil recorded Cristina's interactions with students in order to distill just makes a teacher a good teacher. Paul shared the following from Neil's notes:

"As the session continued, Lindquist gestured, pointed, made eye contact, modulated her voice. “Cruising!” she exclaimed, after the student answered three questions in a row correctly. “Did you see how I had to stop and think?” she inquired, modeling how to solve a problem. “I can see you’re getting tired,” she commented sympathetically near the end of the session...."

Clearly, the communication between the teacher and a student is a dialogue, a two-way street that is not strictly paved and is open-ended. While Neil maybe interested in extracting the qualities of a good teacher to write a computer program that will emulate these traits. For schools without access to technology, highlighting what these qualities are may already be helpful to teachers who desire to become more effective in helping their students. There is a good reason why class sizes must be limited. Teaching requires a knowledge of where the students are. This task becomes impossible as the size increases and teaching then becomes a one-way street. Tests can only be given a limited number of times as these exercises take time, leaving less time for lectures, activities and recitation. Recitations in large classes are usually daunting that students become terrified to speak out or raise a question. For this reason, Paul cites the work of educational psychologist Benjamin Bloom, who published the landmark paper:

Figure downloaded from  http://web.mit.edu/bosworth/MacData/afs.course/5/5.95/readings/bloom-two-sigma.pdf

The above figure is self-explanatory. It highlights how effective tutoring is compared to classroom instruction, so effective that the Asian Development Bank had recently made a study on "shadow education", pointing out how much private tutoring has spread out over Asia and its far reaching economic and social implications. (See "When Teachers are Underpaid...") Paul cites that in New York City's elite schools, tutors can charge as much as $400 per hour. That is almost equivalent to a one month salary of a public school teacher in the Philippines.

Assistments' goal is to provide a software that would mimic private tutoring thereby avoiding the expensive and inaccessible tutors. The following paper describes how Assistment can help in a student's homework:

Mendicino, M., Razzaq, L. & Heffernan, N. T. (2009). Comparison of Traditional Homework with Computer Supported Homework. Journal of Research on Technology in Education, 41(3), 331-359.
http://teacherwiki.assistment.org/wiki/images/1/13/Jrte_layout.pdf

Abstract

This study compared learning for fifth grade students in two math homework conditions. The paper-and-pencil condition represented traditional homework, with review of problems in class the following day. The Web-based homework condition provided immediate feedback in the form of hints on demand and step-by-step scaffolding. We analyzed the results for students who completed both the paper-and-pencil and the Web-based conditions. In this group of 28 students, students learned significantly more when given computer feedback than when doing traditional paper-and-pencil homework, with an effect size of .61. The implications of this study are that, given the large effect size, it may be worth the cost and effort to give Web-based homework when students have access to the needed equipment, such as in schools that have implemented one-to-one computing programs.

Assistment can likewise supplement instruction while providing assessment of the student as described in the following:

Koedinger, K. R., McLaughlin, E., & Heffernan, N. (2010). A quasi-experimental evaluation of an on-line formative assessment and tutoring system. Journal of Educational Computing Research, 43(1), 489–510.
http://teacherwiki.assistment.org/wiki/images/1/18/Quasi.pdf

Abstract 

ASSISTment is a web-based math tutor designed to address the need for timely student assessment while simultaneously providing instruction, thereby avoiding lost instruction time that typically occurs during assessment. This paper presents a quasi-experiment that evaluates whether ASSISTment use has an effect on improving middle school students' year-end test scores. The data was collected from 1240 seventh graders in three treatment schools and one comparison school. Posttest (7th grade year-end test) results indicate, after adjusting for the pretest (6th grade year-end test), that students in the treatment schools significantly outperformed students in the comparison school and the difference was especially present for special education students. A usage analysis reveals that greater student use of ASSISTments is associated with greater learning consistent with the hypothesis that it is useful as a tutoring system.  We also found evidence consistent with the hypothesis that teachers adapt their whole class instruction based on overall student performance in ASSISTments.  Namely, increased teacher use (i.e., having more students use the system more often) is associated with greater learning among students with little or no use suggesting that those students may have benefited from teachers adapting their whole-class instruction based on what they learned from ASSISTment use reports.  These results indicate potential for using technology to provide students instruction during assessment and to give teachers fast and continuous feedback on student progress.

Heffernan does acknowledge that technology has its limitations. A software program can be designed in precise terms. It simply follows instruction. Although smileys can be used, a computer screen can hardly emote. A computer cannot truly put a frowning face. A computer indeed has limitless patience so it never gets annoyed. But this also means that a computer lacks the real drive, the emotions of excitement and enthusiasm. And as a previous article of this blog  "Overcoming the Constraints of Poverty on Education", highlights, skills, such as perseverance, curiosity, conscientiousness, optimism, and self-control, are very important. These are human traits. These are what you see in the eyes of a real teacher.