RYSK: Erlwanger's Benny's Conception of Rules and Answers in IPI Mathematics (1973)

This is the second in a series of posts describing "Research You Should Know" (RYSK).

In 1973, Stanley Erlwanger was a doctoral student at the University of Illinois at Urbana studying under Robert Davis (who taught many of us math as an advisor for Sesame Street) and Jack Easley when he published his landmark "Benny" article in Davis's new Journal of Children's Mathematical Behavior. (Now simply the Journal of Mathematical Behavior.) This and other Erlwanger articles became known as disaster studies (Spieser & Walter, 2004, p. 33) because they painfully reveal learning gone wrong, and they continue to impact the way we think about learning math and how we do research in mathematics education.

During the back-to-basics movement of the 1970s there was a push for programs that supported individualized instruction. One such program was Individually Prescribed Instruction, or IPI. IPI was designed for students to "proceed through sequences of objectives that are arranged in a hierarchical order so that what a student studies in any given lesson is based on prerequisite abilities that he has mastered in preceding lessons" (Lindvall and Cox, as cited in Erlwanger, 1973, p. 51). To measure that mastery, IPI relied heavily on assessments that were checked by the teacher or an aide, who would then have the opportunity to conference with the student and check for understanding. Erlwanger, however, saw a conflict inherent in the program: while the goals of IPI were "pupil independence, self-direction, and self-study" (Erlwanger, 1973, p. 52), teachers were supposed to have "continuing day-by-day exposure to the study habits, the interests, the learning styles, and the relevant personal qualities of individual students" (Lindvall and Cox, as cited in Erlwanger, 1973, p. 52). So is a teacher, with a class of students each working at their own pace, supposed to continuously monitor each individual student? How? The logical way to do this is to monitor assessment results and focus attention on strugging students. After all, if a student is passing the assessments and "mastering" objectives, how much could go wrong?

Benny was a twelve-year-old boy with an IQ of 110-115 in a 6th grade IPI classroom. Benny had been in the IPI program since 2nd grade, and the teacher identified Benny as one of her best students. By sitting down and talking to Benny about the math he was learning, Erlwanger discovered that Benny's conception of math was not only very rule based, but in many cases Benny's rules yielded wrong answers. For example:

  • Benny believed that the fraction \(\frac{5}{10} = 1.5\) and \(\frac{400}{400} = 8.00\) because he believed the rule was to add the numerator and denominator and then divide by the number represented by the highest place value. Benny was consistent and confident with this rule and it led him to believe things like \(\frac{4}{11} = \frac{11}{4} = 1.5\).
  • Benny converted decimals to fractions with the inverse of his fraction-to-decimal rule. If he needed to write 0.5 as a fraction, "it will be like this ... \(\frac{3}{2}\) or \(\frac{2}{3}\) or anything as long as it comes out with the answer 5, because you're adding them" (Erlwanger, 1973, p. 50).
  • When Benny adds decimals, he adds the number and moves the decimal point the total number of places he sees in the problem. So \(0.3 + 0.4 = 0.07\) and \(0.44 + 0.44 = 0.0088\). Benny's rule for multiplication is very similar: \(0.7 \times 0.5 = 0.35\), \(0.2 \times 0.3 \times 0.4 = 0.024\), and \(8 \times 0.4 = 3.2\). Because these are correct answers, that only served to reinforce Benny's rules about the addition of decimals.
  • Benny thinks different kinds of numbers should yield different answers: "2 + 3, that's 5. If I did 2 + .3, that will give me a decimal; that will be .5. If I did it in pictures [i.e., physical models] that will give me 2.3. If I did it in fractions like this [i.e., \(2 + \frac{3}{10}\)] that will give me \(2\frac{3}{10}\)" (Erlwanger, 1973, p. 53).

As you might guess, Benny got a lot of wrong answers and sometimes failed to achieve the 80% mastery mark on his assessments. It's clear that Benny isn't simply guessing and getting wrong answers -- his methods are consistent and he can confidently explain his reasoning. When Benny is wrong, he tries to change his answers until he gets ones that match the answer key, a process he called a "wild goose chase" (Erlwanger, 1973, p. 53). Because Benny's teacher/aide is only looking for answers that match the key (and trying to do so quickly), the emphasis is on the answer, not the reasoning. It was only Benny's persistence that resulted in him mastering more objectives than most of his classmates.

This style of learning led Benny to believe that math is little more than a collection of arbitrary rules and singularly correct answers: "In fractions, we have 100 different kinds of rules" (Erlwanger, 1973, p. 54). Erlwanger asked Benny where he thought the rules came from. "By a man or someone who was very smart. ... It must have took this guy a long time ... about 50 years ... because to get the rules he had to work all of the problems out like that..." (Erlwanger, 1973, p. 54). For both reasons of scholarship and concern for Benny, Erlwanger returned to the school twice a week for 8 weeks to work with Benny one-on-one. Unfortunately, despite Benny's eagerness to learn, Erlwanger found this to be too little time to change Benny's firmly-established view of mathematics and little progress was made.

What Benny Means to Theory, Research, and to Khan Academy

(It might be helpful to read yesterday's post about constructivism and the Khan Academy before reading this section.)

Erlwanger summed up the theoretical aspect in his conclusion:
Benny's misconceptions indicate that the weakness of IPI stems from its behaviorist approach to mathematics, its mode of instruction, and its concept of individualization. The insistence in IPI that the objectives in mathematics be defined in precise behavioral terms has produced a narrowly prescribed mathematics program that rewards correct answers only regardless of how they were obtained, thus allowing undesirable concepts to develop. (1973, p. 57)
Looking back at Benny in 1994, Steffe and Kieren summarized that
Erlwanger was able to demonstrate how Benny's understanding of mathematics conflicted with any "common sense" understanding of what would be regarded as "good mathematics." This was a crucial part of Erlwanger's work, because by demonstrating what a "common sense" view of mathematics should not be, Erlwanger was able to falsify (naively) the behavioristic movement in mathematics education at that very place where behaviorism has its greatest appeal -- at the level of common sense. (p. 72)
Prior to Benny, the large majority of research in mathematics education depended on quantitative methods -- using statistics to summarize and compare the performance of treatment and control groups. Erlwanger had opened the door to qualitative research, which essentially meant that researchers could now see the value of interviews, case studies, and similar methods. In other words, Benny showed researchers that they can, and should, talk to children.

Although we're approaching the 40th anniversary of the Benny study, anyone who has been paying attention to the debates regarding Khan Academy should be able to draw parallels between it and IPI and realize we're retreading a lot of the same water. In a recent Wired Magazine article about Khan, stories are told of students working individually, at their own pace, with their progress measured by a computer that judges answers right or wrong. The article highlights Matthew Carpenter, a fifth grader who has completed "an insane 642 inverse trig problems" (Thompson, para. 2). Carpenter has earned many Khan Academy badges, a sign of progress that pleases his teacher and amazes his classmates. Unfortunately, the article provides no evidence that Matthew Carpenter is not Benny. I, and hopefully everyone, sincerely hope he is not Benny. I hope he's developing a proper view of the nature of mathematics and developing solid mathematical reasoning and understanding. But I can't be sure, and maybe Carpenter's teacher can't be sure, either. While we sometimes can and do use behaviorist programs of instruction to learn, we can't rely on them to be sure that learning is happening the right way. That's Benny's lesson, and that's why we need to be critical (but not necessarily dismissive) of Khan Academy. People who fail to do so might be surprised with the results they get for all the wrong reasons.

References

Erlwanger, S. H. (1973/2004). Bennyʼs conception of rules and answers in IPI Mathematics. In T. P. Carpenter, J.A. Dossey, & J. L. Koehler (Eds.), Classics in mathematics education research (pp. 48-58). Reston, VA: NCTM.

Speiser, B., & Walter, C. (2004). Remembering Stanley Erlwanger. For the Learning of Mathematics, 24(3), 33-39. Retrieved from http://www.jstor.org/stable/40248471.

Steffe, L. P., & Kieren, T. (1994/2004). Radical constructivism and mathematics education. In T. P. Carpenter, J. A. Dossey, & J. L. Koehler (Eds.), Classics in Mathematics Education Research (pp. 68-82). Reston, VA: NCTM.

Thompson, C. (2011, July). How Khan Academy is changing the rules of education. Wired. Retrieved from http://www.wired.com/magazine/2011/07/ff_khan/all/1.