You might have read my recent post about Lorrie Shepard's 2000 article The Role of Assessment in a Learning Culture and assumed she focused on classroom assessment because changing large-scale, standardized assessments was a lost cause. Think again. By that time, an effort to integrate new theories of learning and cognition into the NAEP was already underway, traceable back to a 1996 report titled Implications for NAEP of Research on Learning and Cognition written by by James G. Greeno, P. David Pearson, and Alan H. Schoenfeld. For years Greeno has been recognized as one of education's foremost learning theorists, while Pearson and Schoenfeld are highly-regarded experts in language arts and mathematics education, respectively.
The National Assessment of Educational Progress, sometimes called "The Nation's Report Card," has been given to students in various forms since 1969. Unlike the high-stakes assessments given by states to all students, the NAEP is given to samples of 4th, 8th, and 12th grade students from around the country, and the use of matrix sampling means no student ever takes the entire test. The goal of the NAEP is to inform educators and policymakers about performance and trends, and details about how different NAEP exams try to achieve this are described in depth at the NAEP website.
Greeno et al. tried to answer two main questions in their report: (a) Does the NAEP inform the nation "about significant aspects of the knowing and learning" (p. 2) in math and reading, and (b) What changes in NAEP would make it a better tool for informing the nation about the performance and progress of our educational system? The authors acknowledge the tradition with what they call differential and behaviorist perspectives on learning, and focus more of their attention on the ability to assess cogntiive and situative perspectives, which have strong theoretical foundations but hadn't been reflected in most large-scale assessments.
Concisely, the report says the "key features of learning in the cognitive perspective are meaningful, conceptual understanding and strategic thinking" and that the "key feature of learning in the situative perspective is engaged participation with agency" (p. 3, emphasis in original). Greeno et al. say that if students are engaged in learning activities that reflect these perspectives, then the NAEP should try to capture the effects of those experiences.
One of the main reasons I'm writing about this report is because it gives me another chance to describe current learning perspectives that go beyond the simpler "behaviorism vs. constructivism" argument I knew as a teacher and heard from others. This report does this well without burdening the reader with all the gory details that learning theorists grapple with as they try to push these theories even further. So here's my summary of their summaries of each perspective:
Differential
This perspective accepts the assumption that "Whatever exists, exists in some amount and can be measured" (p. 10). For knowledge, that "whatever" is referred to as a trait, and different people have traits in different amounts. Evidence of traits can be detected by tests, and the correlation of different tests supposedly measuring the same trait is an indication of our confidence in our ability to measure the trait. Because the person-to-person amount of a trait is assumed to be relative, it's statistically important to design tests where few people will answer all items correctly or incorrectly.Behaviorist
Behaviorism assumes that "knowing is an organized collection of stimulus-response associations" (p. 11). To learn is to acquire skills (usually and best in small pieces) and measuring learning is seen as an analysis of behaviors which can be decomposed into responses to stimuli. Behaviorism's influence on curriculum is seen when behavioral objectives are organized as a sequence building bigger ideas out of smaller, prerequisite objectives.Cognitive
The cognitive perspective primarily focuses "on structures of knowledge, including principles and concepts of subject-matter domains, information organized by schemata, and procedures and strategies for problem solving and reasoning" (p. 12). Learners actively construct their knowledge rather than accept it passively, and conceptual understanding is not just the sum total of facts. The early part of the cognitive revolution was reflected in the math and science reforms of the 1950s and 1960s, while Piagetian ideas and research on student understanding have pushed the perspective further. Assessments need to determine more than right and wrong answers, and research involving think-aloud protocols, student interviews, eye-tracking studies, and patterns of responses have yielded better theories about how to assess for student understanding.Situative
The situative perspective is a social view of learning focused on "interactive processes in which people participate in practices that are organized by the societies and communities they belong to, using the technologies and natural resources in their environments" (p. 14). Knowing is no longer in the head -- instead it is seen as participation in a community, and learning is represented by increased and more effective participation. John Dewey took parts of this perspective in the early 20th century, but we owe much of the theory to Lev Vygotsky, whose work in the 20s and 30s in the Soviet Union eventually emerged and has heavily influenced learning science since the late 1970s. The situative perspective is more readily applied to interactions between people or between people and technology (which is seen as a cultural artifact with social roots), but even solitary learners can be assessed with the situative perspective if we focus on "the individual's participation in communities with practices, goals, and standards that make the individual's activity meaningful, either by the individual's adoption of or opposition to the community's perspective" (p. 14). The influence of the situative perspective on curriculum and classrooms is most easily seen in the focus on student participation, project work, small-group discussions, and authentic work in subject-area disciplines.In summary, achievement in each perspective can be described as:
- Differential/Behaviorist
- - "progress a student has made in the accumulation of skills and knowledge" (p. 16)
- Cogntive
- - a combination of five aspects (pp. 16-18):
- Elementary skills, facts, and concepts
- Strategies and schemata
- Aspects of metacognition
- Beliefs
- Contextual factors
- Situative
- - a combination of five aspects (pp. 19-21):
- Basic aspects of participation
- Identity and membership in communities
- Formulating problems and goals and applying standards
- Constructing meaning
- Fluency with technical methods and representations
What Does This Mean for the NAEP?
Greeno et al. declared that the NAEP was "poorly aligned" (p. 23) with the cognitive perspective. It hadn't captured the complexity of student knowledge and they recommended a greater focus on problems set in meaningful contexts and tasks that reflected the kind of knowledge models and structures theorized in the research. As for the situative perspective, Greeno et al. went so far to say that what the NAEP had been measuring was "of relatively minor importance in almost all activities that are significant for students to learn" (p. 27). Whereas the situative perspective focuses on participation in a particular community or knowledge domain, it's impossible to escape the reality that on the NAEP, the domain is test-taking itself, a "special kind of situation that is abstracted from the variety of situations in which students need to know how to participate" (pp. 28-29). Measuring learning from the situative perspective would require a complicated set of inferences about a student's actual participation practices in an authentic domain, and the technical limitations of the NAEP limits our ability to make those inferences.The report continues with specific details about how we might measure learning in language arts and mathematics with the NAEP from both a cognitive and situative perspective. In the conclusion, the authors first recommend some systemic changes: First, NAEP needed more capacity for attending to the long-term continuity of the test and its design. Given how important NAEP is for measuring longitudinal trends, we can't change it without a careful study of how to compare new results to old. Second, the authors wanted a national system for evaluating changes in the educational system. The NAEP alone can't tell us everything we need to know about the effectiveness of educational reforms.
As for recommendations for the test itself, Greeno et al. emphasized the need to align the assessment with ongoing research, especially in the cognitive perspective. Instead of planning for NAEP tests one at a time and contracting out various work, the development process needed to become more continuous with particular sustained attention given to progress in the cognitive and situative dimensions. More ambitiously, the authors recommended a parallel line of test development to begin establishing new forms of assessment that might capture learning in these newer perspectives. This is a critical challenge because while we know the least about assessing from the situative perspective, the situative is often the perspective that frames our national educational goals. The NAEP can't measure progress to situative-sounding goals without better measurement of learning from a situative perspective.
It has now been 12 years since the release of this report. I don't know how Greeno et al.'s recommendations have specifically been followed, but there is good news. If you read most any of the current NAEP assessment frameworks, you can find evidence of progress. The frameworks have changed to better measure student learning, particularly from the cognitive perspective. Some frameworks honesty address the difficulty in measuring the situative perspective using an on-demand, individualized, pencil-and-paper (but increasingly computer-based) test. (See Chapter One of the science framework, for example.) Will we see any radical changes any time soon? I doubt it. The information we get about long-term trends from the NAEP requires a certain amount of stability. Given the onset of new national consortia tests based on the Common Core State Standards, I think the educational system will get its fill of radical change in the next 3-5 years. With that as the comparison, we all might contently appreciate the stability and attention to careful progress reflected in the NAEP.
References
Greeno, J. G., Pearson, P. D., & Schoenfeld, A. H. (1996). Implications for NAEP of research on learning and cognition (p. 84). Menlo Park, CA.