NCTM Denver 2013: Saxe et al's Engagement in Mathematical Discussion: Linking Practices and Outcomes

Research Pressession - Wednesday, April 17, 8:30 am

Geoffrey B. Saxe - University of California, Berkeley
Maryl Gearhart - University of California, Berkeley
Ronli Diakow - University of California, Berkeley
Nicole Leveille Buchanan - University of California, Berkeley
Jennifer Collett - University of California, Berkeley
Bona Kang - University of California, Berkeley
Kenton De Kirby - University of California, Berkeley
Marie Le - University of California, Berkeley
Discussant: Deborah Loewenberg Ball - University of Michigan

This group from Berkeley presented their findings from the use of Learning Mathematics through Representations, or LMR. LMR is a research-based curriculum unit for the teaching of integers and fractions in the elementary grades. Despite only being 19 lessons long (at the time of their study), it carefully attended to students' definitions of number, unit intervals, subintervals, and early fraction sense. When compared to similar coverage by Everyday Math, LMR produced significantly higher learning gains at all stages.

I'd have been hard-pressed to take a worse picture than this.

The group did find variability in LMR results. After checking for curriculum coverage differences and not finding anything significant, they developed measures for both content and participation. The group found that communication was key, especially for the lowest-achieving students.

This research group also did intensive classroom observation and video collection. They looked at interesting teacher moves designed to disrupt student thinking in ways that elicited student protest, where students became motivated to express their understandings of concepts in ways that corrected the teacher's intentional mistakes. The group paid specific attention to the trajectory of student understanding about unit intervals. Over six weeks, with pre-, interim, and post-assessments, they showed how students' understanding of the fraction 8/7 grew (for most students) over time.

Saxe's anthropological approach to studying shifts in understandings over time add some theoretical nuance to this work, examining the semi-durability of ideas as they are reproduced and altered. This kind of detail is often lacking in research, but can provide some key insights about teaching and learning.

The Discussant, Deborah Ball, began her comments with "Wow." I think that says a lot. She commented on the project at a meta-level, about the project itself, and "being greedy," she asked questions about what else we can get out of this body of work. Ball appreciated the connectedness of the project, both to other projects and across the history of Saxe's work. She also appreciated how the work was situated across all aspects of instruction, including teaching, curriculum, and class discussion. Ball called the work "programmatic" in the ways it carefully broke down the issues of the study and carefully applied the right methods, the care taken with definitions, and the depth with which instruction was analyzed. Ball also asked about "correcting the teacher," wondering more specifically what they perceive that move/strategy to be, and how it might not fit into either direct or dialogic instruction. For her "greedy" questions, Ball asked:

  1. What are you learning about teaching?
  2. What are you learning about the challenge of "drop-in" curriculum?
  3. What are you learning about the assessment of student learning?
  4. What did you learn about who was talking in class? What were the relationships with social or identity markers? What are you learning about the use of problems that weren't situated in the real-world of the students?
  5. How can the rest of us learn how to do this kind of programmatic work?

To answer #4, the nature of the work and issues with Human Subjects precluded them from collecting demographic information about students. In the interest of time, the panel decided to take most of the other questions under considerations while allowing time for Q&A from the audience. The most interesting answer in the Q&A was in regards to the availability of the LMR curriculum. Saxe said they tried getting it published commercially, but commercial publishers want to sell K-5 series of textbooks, not a 19-lesson replacement unit. So instead, the group is planning to post all the materials online and make them free for teachers to use in their classrooms.

NCTM Denver 2013: Herbst et al's Methods to Study Decisions in Mathematics Teaching

Research Pressession - Tuesday, April 16, 3:00 pm

Pat Herbst - University of Michigan
Daniel Chazan - University of Maryland
+Karl Kosko - Kent State University
+wendy rose aaron - Oregon State University
Justin Dimmel - University of Michigan
Orly Buchbinder - University of Maryland
Ander W. Erickson - University of Michigan

There are many things about teaching that can be described, but some very important things can be very difficult to measure. This group is working on creating an infrastructure and set of methods to make quantitative claims about phenomena in teaching, such as teacher decisions, recognition of norms and obligations  mathematical knowledge for teaching, and teacher beliefs. Developing these methods should help get the field beyond "what works" to a place of "how things work," ideally resulting in a model where we can better justify certain teaching practices and predict the results of their use. In addition to exploring interventions in classrooms, or working with teachers watching videos of classrooms, the group is using LessonSketch, a cartoon-like technology that allows users to quickly create and/or watch classroom scenarios.

There are several aspects to this research. First is looking at the breaching of norms. Using LessonSketch, teachers watched animations of classroom practice and "pinned" instances of norms being breached. The group found that participants were more likely to pin and remark on actions that breached norms, with more experienced teachers being more adept at identifying the breaches.

Wendy Aaron, Pat Herbst, and Justin Dimmel (right to left)

Using similar tools, the group also looked at teachers' preferences toward different student strategies, evidence for hypothesized instructional norms, and different kinds of professional obligations. The group found that when a norm was breached, teachers' justifications for the breach could be categorized as: (a) disciplinary, (b) institutional, (c) individual, or (d) interpersonal. The group considered these categories as part of the collective knowledge of the teaching profession, even though teachers' obligations may not be indicative of personal acceptance.

+Karl Kosko dug into some of the measurement of decision making. Teachers were provided scenarios of teaching geometry that ended at key points, followed by four potential actions designed to assess the likelihood of norm compliance. Using multinomial regression, the group found that teachers' pedagogical content knowledge for geometry didn't have any measurable effect on decision making in this study, but years of experience and perceived autonomy did have an effect.

In the conclusion, Dan Chazan hoped that work such as this will move teaching practice away from being perceived as an independent variable that determines student achievement. Instead, teaching will be an "achievement verb," something that is dependent on a host of input variables that shape teaching practice.

NCTM 2013 Denver: Superfine's et al's Supporting Underprepared Algebra Students: Results from a Design-Based Research Program

Research Presession - Tuesday, April 16, 1:00 pm

Alison Castro Superfine - Learning Sciences Research Institute, University of Illinois at Chicago
James Lynn - Learning Sciences Research Institute, University of Illinois at Chicago
Timothy Stoelinga - Learning Sciences Research Institute, University of Illinois at Chicago
Mara Martinez - Learning Sciences Research Institute, University of Illinois at Chicago
Cynthia Schneider - Charles A. Dana Center, University of Texas at Austin
Diane Briars - Pittsburgh, Pennsylvania
Discussant: Phil Daro, Public Forum on School Accountability, San Francisco

Algebra 1 continues to be a make-or-break-it course for many high school students. Some 75% of students who fail Algebra 1 will also fail on a 2nd attempt, or to get them through they will be placed into a pseudo-algebra course that grants credit without the rigor and support.

This project arose from the need to find a different way to deal with struggling algebra students. With NSF, Chicago Community Trust, and Gates Foundation funding, this group worked as a design team consisting of 25 researchers, curriculum developers, and practitioners to develop and oversee an intensified algebra program where students take algebra two periods each day. The key partner was the Dana Center at Texas at Austin, and you can find the project website at http://www.utdanacenter.org/intensifiedalgebra/index.php. Much of what was described in the presentation can be found in the key design features and course and lesson structure.

Themes from the literature

The model for curriculum selection is one that I believe is quickly becoming familiar. Tasks are rated by cognitive demand (Stein, Smith, Henningsen, & SIlver, 2009) and teachers are evaluated on how they implement the task, including the "launch" (I'm guessing they're using the work of Jackson, Shahan, Gibbons, & Cobb, 2012) and investigation and debrief phases of the lesson.

As design research, revisions are frequently made and evaluated. This group placed a focus on fidelity, or the extent to which practice follows the intent of the curriculum. Making a total of 58 observations of five teachers in three schools, the group looked for ways they could better support teachers' enactment of tasks with high cognitive demand.

Alison Castro Superfine presenting

Two trends stood out to the researchers. First, they were far more likely to see teachers explain ideas instead of students. Also, use of the AgileMind technology (which was part of the curriculum) varied significantly across the teachers.

While the project and revision process is ongoing, there are signs of positive results. On problems related to linear functions, students in the project showed significant growth, although the percentage correct on these problems still hovered around 50%.

As for how the research has informed the design process, the group observed three challenges: (a) the needs and time cycles of curriculum developers and researchers, (b) working in settings of implementation, and (c) contributing to theories of practice. This group appeared to be applying Gravemeijer's (2004) concept of a local instruction theory to inform their design process, although no author was cited in the presentation and they used local instruction theory in a somewhat different context.

The discussant, Phil Daro, praised the group for their decision to take a design approach to this problem. He questioned a statement in the presentation about not being able to expect high quality enactment of a low quality task, saying this has been documented from successful classrooms in Japan. Daro also liked the group's not-to-strict view of fidelity: "Implementation research is cursed by the idea of fidelity. I'd like to see fidelity go away." Lastly, Daro said this kind of work and partnership is the kind of work that should go on for 20-30 years, creating quality from within. "Don't design something and tell teachers that it's good for them. Develop empathy for teachers first and then give them something they'll think is cool."

References

Gravemeijer, K. (2004). Local instruction theories as means of support for teachers in reform mathematics education. Mathematical Thinking and Learning, 6(2), 105–128. doi:10.1207/s15327833mtl0602_3

Jackson, K. J., Shahan, E. C., Gibbons, L. K., & Cobb, P. (2012). Launching complex tasks. Mathematics Teaching in the Middle School, 18(1), 24–29.

Stein, M. K., Smith, M. S., Henningsen, M. A., & Silver, E. A. (2009). Implementing standards-based mathematics instruction: A casebook for professional development (2nd ed., p. 182). New York, NY: Teachers College Press.

NCTM Denver 2013: Foegen et al.'s Building Progress Monitoring Measures for Algebra: Exploring Items and Scores

Research Presession - Tuesday, April 16, 8:30 am

Anne Foegen - Iowa State University
Barbara Dougherty - University of Missouri, Columbia
Vickie Spain - University of Missouri, Columbia
Jeannette Olson - Iowa State University
Subha Singamaneni - Iowa State University

One of the essential challenges in measurement is to design an instrument that captures the maximum amount of information using a minimum amount of time and resources. (Hint: You can't have both.) While I agree with Lorrie Shepard (2000) that "good assessment tasks are interchangeable with good instructional tasks" (p. 8), there can be room for other kinds of assessments for specific purposes.

This collaboration between researchers at Iowa State University and the University of Missouri, Columbia, is attempting to develop a set of algebra assessments for progress monitoring that students can take in a matter of minutes and can be graded by a teacher equally quickly. Part of the goal is to create the kind of progress monitoring instrument that would be useful for Response to Intervention (RtI). It's definitely something that is needed, as I remember once being told I had to use a simple arithmetic assessment for RtI (for high schoolers!) because an algebra version did not exist for the RtI system the district was using.

We spent time in this discussion session talking about the tasks on both the procedural and conceptual progress-monitoring instruments. While some of the problems were certainly interesting in the way they tried to address student thinking, most of the instruments looked very, very traditional. There is a heavy focus on symbol manipulation and almost nothing comes associated with even a hint of context. Yes, a math student with strong formal skills would do well on these assessments, but I couldn't help but think they're missing something in their approach. That feeling was probably best summarized with the question I asked them: "This assessment looks like something designed for students of Saxon textbooks. How would you feel if that is the case and your instrument is used to promote Saxon texts?"

I left reminding myself two things: (a) projects at the end of the first of a federally funded project are still very much at the learning/big revisions stage and (b) they wanted a test that could be taken in 5-7 minutes, and they got one. If there's something impressive about this effort, it's probably that this team seems to have come to grips with the reality of sacrificing some quality for efficiency. They're pushing themselves to make the best <10 minute algebra tests they can, and I hope their end result is useful, or, failing that, helps us better understand practical limits of assessment.

Monday Night at NCTM: Poster Session and Kenneth Zeichner's Plenary Address

Braving the snow, Ryan Grover and I slogged our way via RTD from Snowy Boulder to Almost-as-Snowy Denver. We arrived in time for a poster session. If you're unfamiliar with poster sessions, I'd describe them as a sort of academic mingling punctuated by awkward moments of silence as you read people's posters before deciding if the person is worth talking to. Yes, it's as fun as it sounds. I had one extended conversation with Sara Lohrman Hartman from Ohio University. Her recent dissertation work in rural Appalachia studied how a small school utilized a math coach. That's a rare luxury for a small rural school, at least in my limited experience.

As 7:00 neared I made my way to the plenary session and happily found Amanda Jansen standing outside the door. I'd never met Amanda in person, but I've conversed with her regularly on Google+ for more than a year. In that time she's made time for me and put up with my questions and rambling; it may not mean much to her, but as a grad student it means a lot to be welcomed by others in the field. We took some seats midway back in the room and settled in for a talk from Kenneth Zeichner, a well-known expert in the field of teacher education. His talk was not specific to mathematics education, but still of great interest for anyone who cares about how teachers are prepared and the institutions that prepare them.

My horrible picture of Ken Zeichner

As a student in Jennie Whitcomb and Dan Liston's Research on Teaching and Teacher Education class this semester, Ken's talk was very familiar. (If you've ever been told of the importance of reflective teaching, Ken and Dan literally wrote the book on the subject.) Ken began by framing three perspectives in the current debate about teacher education. First are the defenders, those who want to preserve the function and status of college and university-based teacher education (UBTE). Next are the reformers, those who seek to deregulate teaching and let the quality of teachers and teacher preparation be dictated by free market principles. Ken frequently cited Rick Hess as a proponent of this perspective, and shared George Will's belief that closing ed schools was the surest path to improving education in the U.S. Last are the transformers. This group, and where Ken places himself, wants reform of teacher education, but in a way that builds upon the capacity and expertise of UBTE and preserves a democratic, public system not driven by market forces.

From here, Ken shared what he sees as the two visions for preparing quality teachers. One is a system that produces professionals for a teaching profession; the other is a system that is technocratic and highly skill-driven, with teachers focused on actions that raise test scores in a narrow curriculum. Teaching skill is important, admits Zeichner, but professionals go beyond the management of a classroom to understand the social and political context of schools and their communities, and have skills that allow them to better reflect in ways that positively impact their teaching.

There are about 3.6 million teachers in the U.S., and alternative pathways for training those teachers (TFA, teacher residency programs, etc.) have grown since the early 1990s. Still, somewhere between 70-80% of the nation's teachers have earned licensure through UBTE at one of the almost 1400 college- and university-based programs around the country. Even if we were to deregulate teacher education, the market does not (and probably would not) have the capacity to prepare teachers without the help of UBTE.

Zeichner admits that UBTE has gotten a lot of non-critical, negative attention from the media. We've come to believe that too many teachers are not getting good results and efforts are now underway to trace poor student results back to the programs that prepared the teacher in those classrooms. When a recent bridge collapsed, said Zeichner, "The engineer who designed the bridge was exposed in the media but I don't remember anyone asking what engineering school he attended." Using another analogy, Zeichner said judging teacher education this way was similar to the idea that we should judge medical schools by patient outcomes. The result, says Zeichner, would be doctors avoiding treating those who need treatment most, just as teachers will avoid teaching in the tough schools with histories of poor test scores. This will lead to further inequality in how quality teachers are distributed, something the U.S. -- which was recently ranked 26th out of 29 nations in a UN report about child well-being -- cannot afford to have happen. The reformers' cries of "No Excuses!" does not rid us of the ails of poverty, no matter how good the teachers might be.

Zeichner addressed some key contradictions in the teacher education argument. For example, the Obama administration has pushed for higher standards for students while simultaneously pushing for alternative pathways to teaching that have lower standards. Also, as Mike Rose has pointed out, the logic of removing teacher education requirements and regulation, or to disregard teaching experience as a measure of quality, makes little to no sense in any other field. (Nobody would trust a neurosurgeon who hadn't been properly trained and lacked experience.) Further, no high-achieving country has deregulated their system of teacher education. Countries like Finland and South Korea have set their standards high, not lowered them.

Zeichner admits that messages like this, and his affiliation with schools of education, have led many to see him as a defender of UBTE. Zeichner insists that he's not a defender, but a transformer. Instead of deregulating UBTE, he sees a great need to change, yet not replace, our current system. This transformation would shift UBTE closer to the world of practice, providing a clinical experience for preservice teachers in a hybrid approach. This move would require the sharing of responsibility and accountability for teacher preparation with schools and their communities, a shared space with values that would prioritize democratized knowledge. Zeichner never advocated for this as a cure-all, but it is the approach most likely to support a system of effective, professionally prepared teachers.

Q&A

(Note: These quotes from Zeichner are rough approximations. Caveat emptor.)

Q: An attendee from Chicago asked about the dismantling of public education in Chicago and the closing of public schools in lieu of charters.

Zeichner: "I'm not against charter schools. I'm against disempowerment of parents and local stakeholders." Zeichner says he was seeing similar school closings in his home of Philladelphia, where public schools are being closed and replaced by charters run by charter management operators. A new study should be out soon that shows the ineffectiveness of these approaches so far. "There's a lack of democratic debate. I'm not for the alternatives to go away, but I'm for an honest debate."

Q: Why do educational entrepreneurs think they're going to make a lot of money?
A: Zeichner: "Some alternative routes use non-profit status for the tax breaks and they outsource their services. From a distance it's hard to see where they're making money, but they are. ... I'm not implying anything about morality. Greedy, self-serving attitudes are everywhere, including universities. But there is a lack of transparency about what's going on. It astounds me how the NewSchools Venture Fund has so much influence while flying under the radar."

Q: I've done UBTE and "quick fix" teacher education programs. Quick fix doesn't work, and online programs are dangerous. Still, too many teachers don't really their profession like a true profession.
A: Zeichner defends online education to a degree, having been involved with some online programs that offered a lot of quality. "I'd be careful in discounting all online programs. Some universities are using them along with more traditional approaches. They can be improved and we still have a lot to learn. We need to be careful about demonizing anything just based on their sponsorships or affiliations."

Q: What are things a new generation of mathematics educators should fight for in public education? How do we take our conversations outside the education community?
Zeichner: "For me, I've decided that ed schools alone will not accomplish much. The ownership of teacher education needs to be broader and shared with communities. I recently met with a lot of stakeholders and listened to their stories. There's a lot of community activism in places like Chicago and Philadelphia, and we can join those efforts already underway instead of coming in from above with solutions to their problems. I wish I'd been more in the public sphere in my career, writing more op-eds and participating in forums, even if many would have disagreed with me. Now, the polarization we're left with is harmful."

Q: How do we measure low-performing teacher education programs?
Zeichner: "I agree with the National Research Council's report on improving teacher education, and that the profession itself needs to take care of its own accreditation and monitoring. I'm afraid that test scores and value-added models will fill this role at great cost and distraction, as simply ranking as a way to improve quality has never really worked. To anoint these people who know little about the field as saviors is idiocy. Twenty-five years from now people will look back and wonder how we could have been so stupid."

(Mic drop. Not really, but I wish.)

My Thoughts

For me, Zeichner's serious (and some might say somewhat depressing) talk was balanced by a sense of validation. Not only did I recently write a paper that expressed many of these same ideas, it gives me hope that the system I'm a part of is recognizing a need and possibility for significant change. For some people, I'm sure a reformed/deregulated system of teacher education seems like a remote possibility, but right now in Colorado we're anticipating the introduction of legislation proposing to do exactly that, removing the requirement for teacher education as we know it. If that bill appears and makes progress, you'll be sure to hear more from me.