This is the 13th in a series describing "Research You Should Know" (RYSK).
In her presidential address at the 2000
AERA conference,
Lorrie Shepard revealed a vision for the future of educational assessment. That message turned into an article titled
The Role of Assessment in a Learning Culture, and its message is still very much worth hearing today. Lorrie Shepard remains a globally-respected expert in assessment, psychometrics, and their misuses, and I'd think she was totally awesome even if she wasn't my boss.
Shepard is often present for debates about large-scale testing, but this paper focuses on classroom assessment -- the kind, says Shepard, "that can be used as a part of instruction to support and enhance learning" (p. 4). Shepard does this by first explaining a historical perspective, then describing a modern view of learning theories, then envisioning how new assessment practices could support those theories. Impressively, she does this all in just 11 well-written pages. (In fact, given that
the paper is available on the web, I wouldn't blame you at all for skipping this summary and just reading the article for yourself.)
History
Shepard highlights several major themes from history that have continued to drive our assessment practices. One is the social efficiency movement, which "grew out of the belief that science could be used to solve the problems of industrialization and urbanization" (p. 4). While this movement might have helped our economic and educational systems scale rapidly (think about Ford and the assembly line), social efficiency carries with it a belief that people have a certain innate (and largely fixed) set of capabilities, and our society operates its most efficiently when we measure people and match their capabilities to appropriate education and employment. For example, students were often given IQ tests to determine if their future path should lie on a particular academic or vocational track.
The dominant learning theories of the early and mid-1900s were associationism and behaviorism, both of which promoted the idea that learning was an accumulation of knowledge that could be broken into very small pieces. Behaviorism was also tied closely to theories of motivation, as it was believed learning was promoted when knowledge was made smaller and opportunities for positive reinforcement for learning were made greater. Much of the assessment work related to these beliefs can be traced back to
Edward Thorndike, considered to be the father of scientific measurement and earliest promoter of "objective" testing. It's been 100 years since Thorndike was elected president of the American Psychological Association, and decades since his ideas seriously influenced the leading edges of learning theory. Still, as most anyone who works in schools or experienced a traditional education can attest, ideas of social efficiency and behaviorism are still evident in schools -- especially in our assessment practices.
Together, the theories of social efficiency, scientific measurement, and beliefs about intelligence and learning form what Shepard sees as the dominant 20th-century paradigm. (See
page 6 of the paper for a diagram.) It's important to begin our discussion here, says Shepard, because "any attempt to change the form and purpose of classroom assessment to make it more fundamentally a part of the learning process must acknowledge the power of these enduring and hidden beliefs" (p. 6).
Modern Theories
In the next section, Shepard describes a "social-constructivist" framework that guides modern thought on learning:
The cognitive revolution reintroduced the concept of mind. In contrast to past, mechanistic theories of knowledge acquisition, we now understand that learning is an active process of mental construction and sense making. From cognitive theory we have also learned that existing knowledge structures and beliefs work to enable or impede new learning, that intelligent thought involves self-monitoring and awareness about when and how to use skills, and that "expertise" develops in a field of study as a principled and coherent way of thinking and representing problems, not just as an accumulation of information. (pp. 6-7)
These ideas about cognition are complimented by
Vygotskian realizations that the knowledge we construct "is socially and culturally determined" (p. 7). Unlike
Piaget's view that development preceded learning, this modern view sees how development and learning interact as social processes. While academic debates remain about the details of cognitive vs. social (and vs. situative vs. sociocultural vs. social constructivist vs. ...), for practical purposes these theories can coexist and are already helping teachers view student learning in ways that improve upon behaviorism. However, Shepard says, since about the 1980s
this has left us in an awkward state of using new theories to inform classroom instruction, while still depending on old theories to guide our assessments.
Improving Assessment
If we wish to make our theories of assessment compatible with our theories of learning, Shepard says we need to (a) change the form and content of assessments and (b) change the way we use and regard assessment in classrooms. Some of the potential changes in form are already familiar to most teachers, such as a greater use of open-ended performance tasks and setting assessment tasks in real-world contexts. Furthermore, Shepard suggests that classroom routines and related assessments should reflect the need to socialize students "into the discourse and practices of academic disciplines" (p. 8) as well as foster metacognition and important dispositions. Shepard does not go into much more detail here because others have already given attention to these ideas, but gives us this simple yet powerful idea (p. 8):
"Good assessment tasks are interchangeable
with good instructional tasks."
Next Shepard pays special attention to negative effects of high-stakes testing. Shepard could be called a believer in standards-based education, but recognizes how "the standards movement has been corrupted, in many instances, into a heavy-handed system of rewards and punishments without the capacity building and professional development originally proposed as part of the vision (McLaughlin & Shepard, 1995)" (p. 9). Unfortunately, Shepard's predictions have held true over the past 12 years: we've seen test scores distorted under political pressure, a corruption of "teaching to the test," and a trend towards the "de-skilling and de-professionalization of teachers" (p. 9). What's worse might be a decade of new teachers who've learned to "hate standardized testing and at the same time reproduce it faithfully in their own pre-post testing routines" (p. 10) because they've had such little exposure to better forms of assessment.
For the rest of the article, Shepard focuses on how assessment can and should be used to support student learning. First, classrooms need to support a learning culture where "students and teachers would have a shared expectation that finding out what makes sense and what doesn't is a joint and worthwhile project" (p. 10). This means assessment that is more informative and reflective of student learning, one where "students and teachers look to assessment as a source of insight and help instead of an occasion for meting out rewards and punishments" (p. 10). To do this, Shepard describes a set of specific strategies teachers should use in combination in their classrooms.
Dynamic Assessment
When Shepard wrote this article, formal ideas and theories about formative assessment were still emerging and the field had yet to settle on some of the language we now use. But if you're at all familiar with formative assessment, Shepard's description of "dynamic" assessment will sound familiar: teacher-student interactions continuing through the learning process rather than delayed until the end, with the goal of gaining insight about what students understand and can do both on their own and with assistance from classmates or the teacher.
Prior Knowledge
The idea of a pre-test to see what students know before instruction begins is not new, but Shepard says we should recognize that traditional pretests don't usually take account of social and cultural contexts. Because students are unfamiliar with a teacher's conceptualization of the content prior to instruction (and vice versa), scores might not accurately reflect students' knowledge as well as, say, a conversation or activity designed to elicit the understandings students bring to the classroom. Also, as Shepard has frequently observed, traditional pre-testing often doesn't significantly affect teachers' instruction. So why do it? Instead, why not focus on building a learning culture of assessment: "What safer time to admit what you don't know than at the start of an instructional activity?" (p. 11)
Feedback
The contrast in feedback under old, behaviorist theories and newer, social-constructivist theories is clear. Feedback under old theories generally consisted of labeling answers right or wrong. Feedback under new theories takes greater skill: teachers need to know how to ignore student errors that aren't immediately relevant to the learning at hand, while crafting questions and comments that force the student to question themselves and any false knowledge they might be constructing. (See Lepper, Drake, and O'Donnell-Johnson, 1997, for more on this.)
Transfer
While it is our hope that our students will be able to generalize the specific knowledge they have learned and apply it to other situations, our ability to accurately research and make claims about knowledge
transfer turns out to be a pretty tricky business. Under a strict behaviorist perspective, it was appropriate to believe that each application of knowledge should be taught separately. Many of our current theories support an idea of transfer, and evidence shows that we can help students by giving them opportunities to see how their knowledge reliably works in multiple applications and contexts. So while some students might not agree, Shepard says teachers should not "agree to a contract with our students which says that the only fair test is one with familiar and well-rehearsed problems" (p. 11).
Explicit Criteria
If students are to perform well, they need to have clear guidance about what good performances look like. "In fact, the features of excellent performance should be so transparent that students can learn to evaluate their own work in the same way their teachers would" (p. 11). This reinforces ideas of metacognition and, perhaps more importantly, fairness.
Self-Assessment
There are cognitive reasons to have students self-assess, but other goals are to increase student self-responsibility and make teacher-student relationships more collaborative. Students who self-evaluate become more interested in feedback from others, are more aware of standards of excellence, and take more ownership over the learning process.
Evaluation of Teaching
This is another idea now heavily intertwined with formative assessment, but Shepard takes it one step farther than I normally see it. Instead of just using assessment to improve one's teaching, Shepard recommends that teachers be transparent about this process and "make their investigations of teaching visible to students, for example, by discussing with them decisions to redirect instruction, stop for a mini-lesson, and so-forth" (p. 12). This, Shepard says, is critical to cultural change in the classroom:
If we want to develop a community of learners -- where students naturally seek feedback and critique of their own work -- then it is reasonable that teachers would model this same commitment to using data systematically as it applies to their own role in the teaching and learning process. (p. 12)
Conclusion
Shepard admits that describing this new assessment paradigm is far easier than it is to implement in practice. It relies on a great deal of teacher ability and confronting some long-held beliefs. Shepard recommended a program of research accompanied by a public education campaign to help citizens and policymakers understand the different goals of large-scale and classroom assessments. Neither the research or educating the public is easy, because both are built upon a history of theories and practice that a new paradigm needs to discard. Perhaps we haven't taken on this challenge with the effort and seriousness we've needed, and I worry that now we're more apt to talk about "learning in an assessment culture" rather than the other way around, as Shepard titled this article. I sometimes wonder if she's considered writing a follow-up with that title, or if she's hoping she'll never have to. I guess the next time it comes up I'll have to ask her.
Math note: This is an article about assessment and not specific to mathematics, but I'd be remiss if I didn't share Shepard's inclusion of one of my all-time favorite fraction problems:
References
Lepper, M. R., Drake, M. F., O'Donnell-Johnson, T (1997). Scaffolding techniques of expert human tutors. In K. Hogan & M. Presley (eds.), Scaffolding student learning: Instructional approaches & issues. Cambridge, MA: Brookline Books.
McLaughlin, M. W., & Shepard, L.A. (1995). Improving education through standards-based reform: A report of the National Academy of Education panel on standards-based educational reform. Stanford, CA: National Academy of Education.
Shepard, L. A. (2000). The role of assessment in a learning culture. Educational Researcher, 29(7), 4–14. doi:10.2307/1176145
Thompson, P. W. (1995). Notation, convention, and quantity in elementary mathematics. In J. T. Sowder & B. P. Schappelle (Eds.), Providing a foundation for teaching mathematics in the middle grades (pp. 199-221). New York: State University of New York Press.
