NCTM Denver 2013: Lester's Whatever Happened to Problem Solving in the Math Curriculum?

Annual Meeting - Saturday, April 20, 9:30 am

Frank K. Lester - Indiana University

Frank Lester was the editor of the 1234-page Second Handbook of Research on Mathematics Teaching and Learning. This was an interesting talk, not so much for any specific content but for how it was put together. Lester began the talk by demoing Algebra Touch, an iOS app that promotes fluency with symbol manipulations in solving equations. He asked, "What will the math classroom of the future be like?"

Frank K. Lester

Then Lester went into problem solving, something he feels has slowly slipped out of most mathematics curricula. Problem solving, says Lester, is "What you do when you don't know (or aren't sure) what to do." That leads to the teacher's role in the classroom:


From here, Lester showed some of his favorite problem solving tasks. The first is probably familiar to most of you:

A snail is at the bottom of a well that is 10 meters deep and it wants to get out. Every day it climbs up 4 meters. It then slides back 2 meters when it rests at night. If it does this day after day, how many days will it take the snail to reach the top of the well?

As part of the discussion, Lester related his problem solving heuristics, harkening back to Polya's How to Solve It. Suddenly a talk that began with a discussion of technology and the future was using a (good?) problem that felt like it was from the 1980s and (good) strategies that were from the 1940s. Lester's choice of heuristics to apply here were "Draw a picture/diagram" and "Be skeptical of your solutions," since many initially reason that the snail reaches the top of the well in 5 days.

Lester then looked at finding the square root of 12,345,678,987,654,321. His heuristic -- one he called a "super heuristic" -- was to look for a pattern. I couldn't help but feel like this was a trivial problem with a trivial answer.

Things got better with the next problem: "On a European river cruise, 2/3rds of men are married to 3/5ths of the women. How many men and how many women are on the cruise?"

Lester joked that this problem predated talk of same-sex marriage, and I found it to be a bit out of touch. Lester said the problem could be adapted to involve pairings of animals or objects. Another heuristic here was "make reasonable guesses, not as final answers, but to get you started." After discussion of this problem, we moved on to one more: "A club has 500 members. At the Spring dance, tickets for new members were $14 but $20 for longtime members. All of the new members attended but only 70% of the longtime members attended. How much ticket revenue was collected?" It seems like there isn't enough information, but solving this plays off the fact that $14 is 70% of $20. That might elicit some interesting reasoning, but again I think this trivializes the problem.

Lester returned to technology at the end, mentioning strategy games like Math Dice and Rush Hour. He advised that teachers have an important role to play when students play games. Prior to the play, teachers need to help students be clear about the rules for playing, model how to play, and discuss special situations. I think that depending on the game and the goals, this could turn into too much guidance. During student game play, teachers need to watch students play, attend to their thinking, help and point out misunderstandings. It's important, says Lester, to not suggest strategies for playing. Kids should be left to figure those out for themselves. After gameplay, reflection is important, just as with other classroom activities.

Lester's slides can be found at the conference planner.

claimtoken-51bce6f7eea47

NCTM Denver 2013: Danielson's They'll Need it for Calculus

Annual Meeting - Saturday, April 20, 8:00 am

Christopher Danielson - Normandale Community College, Bloomington, Minnesota

As Steve Leinwand noted in his Thursday talk, math teachers are a relatively conservative, risk-averse bunch. Perhaps our conservatism comes from the perceived slow but steady progress of math over millennia where it's easy to take comfort in the old because the new can seem so difficult to obtain. Some of this rubs off in the way we teach, the activities we choose for students, and our judgement about what's important for students to know.

Chris Danielson's session kicked off by calling out some mathematics that gets taught in the name of "needing it for calculus," despite no widespread need for it anymore. Simplifying radicals. Rationalizing the denominator. Simplifying rational expressions. Factoring quadratics. Composition of functions. The binomial theorem. It's not that someone, somewhere doesn't have a use for these things, but what is increasingly becoming the exception should not prove the curriculum rule. Mediocre proficiency with these topics is not what leads students to be successful in calculus. What students really need for calculus is a deep understanding of slope as a rate of change and accumulation.

Christopher Danielson

This is a familiar story for some of us. We cringe when we ask students "What's slope?" and they parrot back, "rise over run" without knowing much beyond that. Yes, that might be one way to describe slope, but there are other, and arguably more important ways to describe slope. Danielson's focus on slope as a rate of change not only is most fundamental for calculus, but it is in alignment with the research on teaching slope (Lobato & Thanheiser, 2002; Peck & Matassa, 2012; Stump, 1999, 2001).

Danielson led the well-attended workshop through a number of middle-school appropriate tasks involving rates of change. Because the tasks were set in informal contexts, students would be most likely to work in terms of "dollars per bicycle rental" or "enjoyment per piece of candy," depending on the context of the problem. Time was spent not just looking at rates, but doing simple calculations to compare changes in rates over time, a fundamental conception needed for calculus. The problems in the workshop were adapted from tasks found in Connected Mathematics, a popular NSF-funded curriculum for the middle grades.

For Christopher's presentation, related tweets, and participant notes, see his post at http://christopherdanielson.wordpress.com/2013/04/21/the-goods-nctmdenver/.

References

Lobato, J., & Thanheiser, E. (2002). Developing understanding of ratio-as-measure as a foundation for slope. In B. H. Litwiller (Ed.), Making sense of fractions, ratios, and proportions (pp. 162–175). Reston, VA: NCTM.

Peck, F., & Matassa, M. (2012). Beyond “rise over run”. RME in the classroom. Workshop at ICME-12, Seoul, South Korea. Retrieved from http://rmeintheclassroom.blogspot.com/2012/07/icme-12-workshop-and-sharing-group-on.html

Stump, S. L. (1999). Secondary mathematics teachers’ knowledge of slope. Mathematics Education Research Journal, 11(2), 124–144. Retrieved from http://www.springerlink.com/index/R422558466765681.pdf

Stump, S. L. (2001). High school precalculus students’ understanding of slope as measure. School Science and Mathematics, 101(2), 81–89. doi:10.1111/j.1949-8594.2001.tb18009.x

NCTM Denver 2013: Fennell and Wray's Math Specialists Get Ready Now: Common Core Assessments Are Coming

Annual Meeting - Friday, April 19, 3:30 pm

Francis (Skip) Fennell - NCTM Past President; McDaniel College, Westminster, Maryland
Jon Wray - NCTM Board of Directors; Howard County Public Schools, Ellicott City, Maryland

Skip Fennell, Jon Wray, and Beth Kobett (who was absent for this presentation) are the leads on ems&tl, the Elementary Mathematics Specialists & Teacher Leaders Project. As the name implies, the focus here is on supporting math specialists, such as district-level curriculum directors, instructional coaches, and anyone who is in a position to support mathematics teachers.

For this presentation, Fennell and Wray looked at the upcoming Common Core assessments, PARCC and Smarter Balanced (SB), and suggested ways math specialists can help teachers prepare for the tests.

Francis (Skip) Fennell

The challenge Fennell and Wray presented was essentially to focus on the upcoming assessments and respect the influence they will have on curriculum and instruction, without focusing too narrowly on the assessments and cause instruction and learning to suffer. This means, for example, not turning classroom practice into test prep, and using sample items from both PARCC and SB wisely.

Fennell and Wray used the concept of assessment literacy to describe the ability for teachers and specialists to understand a testing program. Many teachers have no formal training in assessment, so math specialists must be able to help them build their assessment literacy. Part of this is simply becoming more familiar with the schedules and formats of the upcoming PARCC and SB assessments. Both consortia offer more than just an end-of-year test, and teachers are going to need to help students interpret new kinds of technology-enabled assessment tasks.

Jon Wray

Fennell sees great potential in the CCSSM, but said, "If the Common Core becomes political, it's dead." Teachers and specialists need to work with the standards in ways that doesn't reduce them to a checklist of vaguely connected ideas. Using a number of items and task prototypes, Fennell and Wray showed examples of sample items from PARCC and SB and showed the many ways these could be used richely in lessons if the teacher provides the right support and instruction. "There are a lot of ways sample items can be used as instructional gems, " said Fennell. A list of potential questions and strategies for various tasks can be found in their slides.

The presentation wrapped up with an urging to better understand the role of formative assessment around these sample tasks. Also, encouragement was made to use materials from both PARCC and SB, regardless of the test your state has adopted. More task resources were linked to, including Illustrative Mathematics, the Institute for Mathematics and Education (especially the progressions documents), The Mathematics Common Core Toolbox, the PARCC Educator Leader Cadre Portal, and the Smarter Balanced Scientific Sample Pilot Test Portal.

The slides for this presentation are available here.

NCTM Denver 2013: Building Mathematics Learning Communities with NCTM Reflection Guides

Annual Meeting - Friday, April 19, 2:45 pm

NCTM Professional Development Services Committee
Chonda Long, Director of Professional Development at NCTM

The take-away from this session is pretty simple. To help facilitate professional development, NCTM has produced a series of free, online reflection guides that leverage lessons in NCTM journals.

Chonda Long

The idea is that teachers in professional learning communities can use these reflection guides to help focus their efforts around particular lessons. As an example, we worked through a problem presented in a 2005 issue of Mathematics Teaching in the Middle School. I left to attend another session before the ending, but the reflection guide and a link to the lesson can be found at http://www.nctm.org/profdev/content.aspx?id=23531.

To see all the reflection guides, visit http://nctm.org/reflectionguides.

NCTM Denver 2013: Hirsch's Mathematical Modeling: The Core of the Common Core State Standards

Annual Meeting - Friday, April 19, 2:00 pm

Christian R. Hirsch - Western Michigan University

Hirsch might claim that modeling is at the core of the Common Core, but at a glance it looks like a standard without standards. Yes, the fourth Standard for Mathematical Practice is "Model with mathematics," but the high school content standards chooses to mark standards in other domains as related to modeling instead of grouping the modeling standards together. This makes it more difficult to see the modeling connections across the high school standards, but that shouldn't reduce their importance.

Christian Hirsch has been at Western Michigan for 40 years and he is probably best recognized as the principal investigator for the Core-Plus Mathematics Project. Along with IMP, Core-Plus is one of the most recognized secondary, integrated, NSF-funded curricula to come out of the post-Standards curriculum development period in the 1990s.

Christian Hirsch

Hirsch opened his talk by detailing how all of the mathematical practices can be addressed with a modeling-focused framework of curriculum and instruction. "Real world problems, if even solvable, take a lot of time and perseverance." To Hirsch, Standards for Mathematical Practice 1 and 4 are the focal points of the entire process, at least in classrooms with good instruction. "I'm talking about classrooms where classes begin with problems. I'm not talking about classrooms where the problems are saved until the end."

The key to modeling and making mathematics problematic, says Hirsch, is to identify problems in context, study those problems through active engagement, and reach conclusions as the problems are at least partially solved. The learning lies not only in the solutions to the problems, but the new mathematical relationships that are discovered along the way.

Hirsch used several examples of problems involving modeling in this presentation. The first dealt with the business prospects of a climbing gym. Assuming a survey had been conducted that found the number of expected climbers is related to price \(x\) by the equation \(n(x) = 100 - 4x\), how many daily climbing wall customers should the gym expect? I didn't catch all the details of this problem, but the next question involved finding the optimal and break-even revenue points for the gym, which is nicely modeled by a quadratic. Hirsch advocated using a computer algebra system to assist with the calculations, and advised to help students realize that rounding to the nearest cent, if necessary, also slightly moves answers away from their true zeroes or maximums.

Hirsch's next problem dealt with finding the optimal location for an oil refinery with wells 5 km and 9 km from shore. I sense that this and the previous problem are in Core-Plus, but unfortunately that wasn't made clear and no handouts or downloads for this talk have been provided. While I don't like leaving presentations early, at this point I had a pretty good sense for this one and left to catch an overlapping presentation starting at 2:45. The problems Hirsch chose and the approaches to solve them were pretty solid 30 or more years ago and are still pretty solid today, and I wasn't feeling like the presentation was suddenly going to break new ground. (For me, at least. I totally understand that problems and approaches like this might be new ground in many classrooms.)

NCTM Denver 2013: Business Meeting

Annual Meeting - Friday, April 19, 12:30 pm

I had no idea what to expect at the NCTM Business Meeting. It was scheduled to be in one of the larger rooms, so I was surprised to see only a dozen or so people there, huddled up and down the sides of the center aisle. Kichoon Yang, NCTM's soon-to-retire executive director, was about the only one standing. I walked in, then out, then back in the room again, and had to make sure the meeting was open to the public before sitting down.

The meeting was quick. There was some review of prior business and some quick votes on some proposed motions. The item of most interest to me was discussion of possibly moving the NCTM Annual Meeting to the summer. NCTM sent out a survey a few months ago to gauge interest in the move, as attendance at the Annual Meeting has declined as more states require testing in April. According to Yang, NCTM has many contractual agreements for the Annual Meeting stretching out five years or more, making rescheduling the Annual Meeting a possible, but non-trivial matter.

The person seated in front of me asked if the new Common Core-based consortia tests, PARCC and Smarter Balanced, would also have conflicting test dates in April. Yang said that would be a good thing to investigate. Shortly after I talked to the person who asked the question and realized it was Shirley Frye, Past President of NCTM in 1989, the year the first Standards were released. I think she was impressed that I knew that, and I explained to her my connections to Bonnie Litwiller (author of the Addenda project) and Ed Rathmell (co-author of the elementary standards, and someone I was happy to spend time with at the conference) at Northern Iowa. She was proud of how successful some of that work had been, particularly in the Addenda's ability to generate revenue for NCTM that was needed after taking a financial risk to produce the Standards. We didn't talk long, but as someone with an interest in the history of math education I think meeting Shirley Frye was among the true highlights of my week.

NCTM Denver 2013: Thrasher and Perry's Supporting Beginning Teachers through Online Social Communities

Annual Meeting - Friday, April 19, 11:00 am

Emily Thrasher - North Carolina State University
Ayanna Perry - North Carolina State University

At NC State University there is a program called the Noyce Mathematics Education Teaching Scholars, funded by the National Science Foundation. Along with providing preservice preparation, this program experimented with using online tools to support their graduates (called "Scholars" below) in their first years of teaching.

Thrasher and Perry

Thrasher and Perry cited the poor retention statistics for teachers, and hoped that this program would keep more of their Scholars in the classroom. Their approaches for continued contact and collaboration were organized into synchronous and asynchronous modes of communication.

Synchronous communication was mostly facilitated through real-time chat and Skype. Sometimes Skype included an expert panel or guest speakers. The group found that by being online everyone could participate, and Scholars liked keeping in touch with their former classmates. The synchronous meetings also provided time to discuss research-based practices and how the Scholars might use them in their classrooms. Unfortunately, scheduling synchronous communication was a persistent problem. Scholars didn't feel comfortable prioritizing a "virtual" meeting on their calendar and requested more face-to-face meetings. These face-to-face meetings were difficult because Scholars had taken jobs in various places across the country, but an effort was made to meet for on-campus professional development or to meetup at professional conferences several times throughout the year.

The group primarily used five tools to collaborate asynchronously: a wiki, a website, Dropbox, Facebook, and Google+. The initial plan was to use the wiki heavily, but Scholars were slow to adapt to the technology and needed facilitators to model how the tool could be used to share. Also, the presenters felt it was important to make the wiki private, as there were things the Scholars probably didn't want to be made public. The website served more as an archive of newsletters and activities, and did not facilitate much collaboration itself. Participation in Dropbox was perhaps the most successful, as Scholars found it very easy to create subfolders and submit lesson ideas and resources. The project didn't see much use for Facebook at first, judging it to be too informal, but it emerged as a place where Scholars and their former professors could keep in touch. Lastly, Google+ emerged as useful when several of the Scholars started using it to distribute and discuss self-made YouTube videos and links to lessons, as well as using Google's instant messaging features.

To judge the effectiveness of these efforts, the researchers observed classrooms and asked Scholars to reflect and discuss the data that was collected. While valuable, observations were limited by time and budget, and the presenters suggested that reviewing videos of lessons might have been a better strategy.

Of the 24 scholars in the program, two never taught and 19 of the remaining 22 are still teaching beyond their first year, so retention looks good thus far. Interestingly, some of the Scholars have taken considerable interest in discussing the research shared online, and that's caused some of them to consider leaving teaching for graduate school. While seeking more education is certainly not bad, it wasn't part of the plan to increase retention.

NCTM Denver 2013: Leinwand's Essential Mindsets for Tilling the Soil for the Common Core State Standards

Annual Meeting - Friday, April 19, 9:30 am

Steve Leinwand - American Institutes for Research, Washington, D.C.

There is perhaps nobody better at shouting math education's rallying cry than Steve Leinwand. Knowing that my notetaking could not keep up, I recorded Steve's talk for later review. Graciously, Steve has granted me permission to post it here. (Which saves me a ton of typing!) You can find slides for Leinwand's "Tilling the Soil" talk on his website.


Check this out on Chirbit

Dan Meyer covered the tweeting duties during the talk:























My takeaway? Math teachers need to push for more and better collaboration. No longer can teachers just teach what they enjoy, or pretend teaching is mostly improvisational. If we are truly professionals, we need to do serious work around our new standards and curriculum, including critiquing the teaching of colleagues, reviewing and refining lessons over time, and recognizing the body of knowledge about teaching mathematics that can be built upon and further contributed to. But listen for yourself.

NCTM Denver 2013: Abels, Matassa, & Johnson's Making Sense of Algebra with Realistic Mathematics Education

Annual Meeting - Thursday, April 18, 2:45 pm

Mieke Abels - Freudenthal Institute for Science and Mathematics Education, University of Utrecht
+Michael Matassa Jr. - Freudenthal Institute US, University of Colorado Boulder
+Raymond Johnson - Freudenthal Institute US, University of Colorado Boulder

When it came time to propose session for the 2013 NCTM Annual Meeting in nearby Denver, we at the Freudenthal Institute US at CU-Boulder knew we should have some kind of "Intro to RME" workshop. Because I was already proposing to be a lead speaker on another session, I needed to find someone else to take the lead. Michael Matassa said he would do it, but then +David Webb had a better idea: Why not ask Mieke Abels from the Freudenthal Institute to do it? Mieke would be a perfect choice - she's been involved in FIUS from the beginning and she continues to be involved in curriculum development for Mathematics in Context and curriculum in the Netherlands. Happily, Mieke agreed and Michael and I were happy to back her up as co-presenters.

The picture at the top is Nederland, CO, which is amusing to our Dutch colleagues

Our goal in this presentation was to bring out the curriculum design features and give attendees a sense for informal and preformal approaches to algebra for the middle grades. Too often it seems "early algebra" gets interpreted as "algebra early," as if a school could just box up their high school algebra textbooks and ship them down to the middle school. Making big jumps to formal mathematics is risky, and that's one reason Realistic Mathematics Education (RME) adheres to a principle called progressive formalization. To illustrate, we started with a task you could give to 6th graders, or perhaps even younger students.

Tug-of-war, taken from Mathematics in Context

Those of us who have mastered formal algebra tend to want to write equations for this and solve. But for young students, RME design principles suggest we support students by relying on a "realistic" context. While "real-world" contexts are certainly realistic, RME's use of "realistic" means it can be imagined by the learner. The power of the context is not necessarily its authenticity, but its capacity to be mathematized.

On the tug-of-war task students will inevitably find different ways to substitute different animals for each other until it becomes clear which side would win the tug-of-war. Some students will likely try to redraw the animals, while others might use letters ("E" for elephant, etc.) as a substitute. Even though a formal equation might use "E" to represent the pulling strength of an elephant, it's fully expected at this stage for students to try writing things like "E = O + 2H" to represent the animals in the middle of the above slide, and interpret "2H" simply as the abbreviation "2 horses."

The Iceberg Metaphor

The concept of progressive formalization is often represented with the iceberg metaphor (Boswinkel & Moerlands, 2003; Webb, Boswinkel, & Dekker, 2008), which places formal mathematics above the water line. The tip of the iceberg is only supported because of the iceberg's "floating capacity, which is where informal and preformal mathematics is placed. Examples like the tug-of-war problem are informal because they rely almost entirely on the realistic context with little or no mathematical abstraction.

Here's another example of an informal task:

Three Frogs, taken from Mathematics in the City

Again, the frog jumping doesn't have to be something replicable in the real world. It need only exist in the imagination of the student, and to solve it students need to find ways to represent the jumps and steps in their work. At this point students will have worked often with number lines (including open number lines) and easier problems involving frog jumping, making number lines a natural model for this problem, like this:

Using an open number line to represent Sunny's jumps

The nature of this problem and the need to draw double number lines that end in a particular place helps students consider what it means to be variable in this problem, versus what quantities remain constant. Other types of problems with other contexts use other kinds of models. For example, the familiar model of a balance is used in RME-based curricula (the 1 and 5 represent weights):

The balance model, found in the Digital Mathematics Environment

Student work for these kinds of tasks can be an indicator of where in the formalization process students might be. If students are redrawing pineapples and lemons, they are still working at an informal level. For convenience they might replace pineapples and lemons with letters, suggesting a small amount of formalization, and eventually they'll be using those letters to write equations and not need to think in terms of the fruit and the balance. Just like tug-of-war, the balance model suggests an understanding of equals that is relational, which helps students who tend to interpret equals as operational.

The use of models is at the heart of the preformal level of the iceberg. Nearer the bottom we would place models of informal contexts. For example, a student who draws sectors of a circle to represent a fraction of a pizza is using the circle as a model of the pizza. Nearer the top of the preformal level we would place models for mathematical abstraction. The student who uses sectors of a circle to represent a fraction of seats occupied in a bus is using the sectors of the circle as a generalized representation of a part and whole, and not a specific representation of a bus. In RME students become familiar with many models, such as number lines, open number lines, double number lines, ratio tables, five frames, rekenreks, area models, and balance models.

Models are key at the preformal level of the iceberg

Another preformal model useful for systems of equations is notebook notation. Here a problem shows two combinations of long and short candles. Working informally, students would find some combination of candles that makes the problem solvable, such as doubling the second combination to make two long candles and two short candles for $6.80. Since the top arrangement has one more short candle and is a dollar more, then short candles must cost $1.00. A preformal way of working with these combinations is notebook notation:

Notebook Notation, taken from Mathematics in Context

From the notebook it becomes easier to see how students will learn to write and manipulate formal systems of equations. The column headings become the variables, and equal signs are placed in front of the total. Formal matrix notation is reachable from notebook notation as well.

Although progressive formalization is often presented as a direct informal-to-preformal-to-formal process, it is not expected that students will learn this way. Students who can work formally or preformally with easier problems are likely to reach to a lower level when problems become more difficult. Because they have achieved the formalization with easier problems, they become more likely to formalize more difficult problems when they can reason with less formal strategies when necessary.

Another way to view progressive formalization is with a learning trajectory, which connects specific contexts and representations along a path towards formal mathematics. Creation of both iceberg models and learning trajectories can be a productive activity for professional development and curriculum planning and alignment.

A learning trajectory for equations and systems of equations, with connecting links

RME isn't meant to be deeply complex, but contexts, models, and the connections between them need to be carefully chosen. Curriculum developers at the Freudenthal Institute take a design research approach to this work, testing and revising in iterative cycles to improve the curriculum over time. FI (formerly IOWO) was founded by Hans Freudenthal in 1971, giving the Netherlands over 40 years to gradually improve their mathematics curriculum and teaching. This type of adherence to a core philosophy for so long is generally unknown in education in the U.S., but schools in the Netherlands have used it to score near the top of international rankings on the mathematics portion of the PISA assessment.

If you'd like more information about RME, the following might be of interest:
References

Boswinkel, N., & Moerlands, F. (2003). Het topje van de ijsberg [The top of the iceberg]. De Nationale Rekendagen, een praktische terugblik [National conference on arithmetic, a practical view] (pp. 103–114). Utrecht, The Netherlands: Freudenthal Institute. Retrieved from http://www.fisme.science.uu.nl/publicaties/literatuur/5467.pdf

Van Reeuwijk, M. (2001). From informal to formal, progressive formalization an example on “solving systems of equations”. In H. Chick, K. Stacey, J. Vincent, & J. Vincent (Eds.), The future of teaching and learning of algebra: The 12th ICMI study conference (pp. 613–620). Melbourne, Australia. Retrieved from http://repository.unimelb.edu.au/10187/2812

Webb, D. C., Boswinkel, N., & Dekker, T. (2008). Beneath the tip of the iceberg: Using representations to support student understanding. Mathematics Teaching in the Middle School, 14(2), 110–113. Retrieved from http://www.nctm.org/publications/article.aspx?id=20793.

NCTM Denver 2013: Johnson & Thomas's Statistical Reasoning in the Middle School

Annual Meeting - Thursday, April 18, 1:00 pm

+Raymond Johnson (Me!) - University of Colorado Boulder and Freudenthal Institute US
+Susan Thomas - University of Colorado Boulder


My colleague Susan Thomas and I took an interest in middle school statistics during our first year in our PhD program. When we read the statistics standards in the Common Core State Standards (CCSSM) for middle school, we both thought they were a major shift from prior standards, and we wondered how teachers would perceive them. For our qualitative research project, we found two willing middle school teachers who offered us access to their classrooms and time for interviews. The data we collected suggested that these teachers, both well-educated and in a high-achieving school, had some difficulty interpreting some of the standards, were unclear about some of the bigger ideas those standards might lead to, and did not have curriculum to support the level of thinking and reasoning called for in the CCSSM.

The push for more statistics is reflected in each iteration of major standards documents (NCTM 1989, 2000), the GAISE report, and the CCSSM. Some of this comes from the relative youth of statistics as a field. I've heard teachers say, "Why do we teach boxplots? We didn't learn them when I was in school," not realizing that John Tukey didn't invent them until the 1970s -- in many cases, the teachers are older than the plots. How often as math teachers do we get to teach anything younger than we might be?

More obviously, the push for statistics is part of the explosion of data in our world. Google's Eric Schmidt recently claimed that "There were 5 exabytes of information created between the dawn of civilization through 2003, but that much information is now created every 2 days, and the pace is increasing" (Kilpatrick, 2010). Even when the topic isn't "big data," increasingly we collect more and more information about our own activities. Making sense of data requires statistical thinking and reasoning.

To understand statistical reasoning, it's helpful to have a way of thinking about how statistics is different from mathematics. In 1962, John Tukey wrote, "Statistics is a science in my opinion, and it is no more a branch of mathematics than are physics, chemistry, and economics; for if its methods fail the test of experience -- not the test of logic -- they are discarded" (pp. 6-7). A more recent and helpful perspective of statistics comes from Michael Shaughnessy, who said "The twin sister of the 'certainty' in mathematics is the 'uncertainty' in statistics. We must prepare our students to deal with both types of quantitative reasoning as they grow in the mathematical sciences" (2010).

So what is statistical reasoning? According to delMas (2004), statistical thinking is knowing when and how to apply statistical procedures, while statistical reasoning explains why results were produced or why a conclusion is justifed. That means that things like stating implications, justifying conclusions, and making inferences are all part of statistical reasoning.

For this workshop, the strategy Susan and I used was to: (a) identify in each grade level of the middle school CCSSM a theme related to statistical reasoning, (b) find something in the research literature that might help teachers better understand what kind of reasoning to elicit and look for in their classes, and (c) evaluate some statistical tasks for their ability to elicit that reasoning. Susan got snowed out on the wrong side of Vail Pass, but thanks to our preparation together I was able to soldier on without her.

Grade 6: Variability and Distribution


A look across 6th grade standards

When we looked at the 6th grade standards, we saw a common theme of reasoning with variability and distribution. Yes, there's focus on measures of center and procedural things like knowing how to calculate the IQR and MAD, but all that is supported by reasoning with variability and distribution.

The work of Bakker and Gravemeijer (2004) stuck out to me in how they approached student reasoning with variability. "An underlying problem is that middle-grade students generally do not see 'five feet' as a value of the variable 'height,' but as a personal characteristic of, say, Katie" (Bakker & Gravemeijer, 2004, pp. 147-148). The suggestion was for students, instead of always assembling data points into distributions, had opportunities to deal with distributions that obscured individual points. By focusing more on distributions and variability, students will be more prepared later to reason with standard deviations, margins of error, and other topics in high school and beyond.

The tasks we chose to inspect were:

Often the best part of leading a workshop is simply giving teachers time to talk and share ideas. If you look at the tasks and want to share ideas, please do so in the comments.

Grade 7: Sampling and Inference


A look across 7th grade standards

Across the 7th grade standards we see a strong theme for reasoning with sampling and inference. The focus on informality here is key, as a standard like 7.SP.B.4 is not trying to suggest hypothesis testing and t-tests, but instead a basic understanding for how variability in samples indicates uncertainty about population parameters, either as a single sample or in a comparison of samples.

Two items from the research stood out to me here. First, Rubin, Bruce, and Tenney (1991) claimed that "Over-reliance on sample representativeness is likely to lead to the notion that a sample tells us everything about a population; over-reliance on sample variability implies that a sample tells us
nothing" (p. 315). Getting more specific about the notion of sampling variability, Saldanha and Thompson (2003) found that higher-performing students "developed a multi-tiered scheme of
conceptual operations centered around the images of repeatedly sampling from a population, recording a statistic, and tracking the accumulation of statistics as they distribute themselves along a range of possibilities" (p. 261). Together with reasoning of variability developed in 6th grade, these standards address the crux of inferential statistics: sampling distributions.

The tasks we inspected were:
There was a lot of good discussion about the Counting Trees task and speculation around student strategies. There's a clear intersection here with students' reasoning with ratio and proportion, and one teacher suggested instead of using the diagram in the task, using aerial or satellite photography with real trees. The key idea, however, is giving students opportunities to sample in different ways and reason why different approaches yeild different, yet similar estimates for the population.

Grade 8: Covariation


A look across 8th grade standards

The clear unifying theme across 8th grade CCSSM standards is covariation of two variables. There is some real opportunity to have this support 8th grade algebra standards related to graphing lines and linear equations, and I'm anxious to see curriculum that really ties the two together.

From the research, I noticed two key things. First, Konold (2002) judged that "It seems unwise, for example, to specify ... that by middle school, students will learn how to 'make conjectures about possible relationships between two characteristics of a sample on the basis of scatterplots' (NCTM 2000, p. 248)" (p. 5). Konold isn't saying that 8th graders can't understand scatterplots, but rather that middle school students understand covariability in a variety of ways, and will develop their own alternative representations before reasoning with scatterplots. In similar work, Moritz (2005) found that students with incomplete understanding will often focus on the variabilty of one variable but not the other, or the variability of both without the association between the two. Some examples of alternative representations:

An informal and creative bivariate table

Paried and sorted case value plots

Scatterplot slices

All of the above are non-scatterplot examples that appropriately show reasoning with covariation. The tasks we inspected that deal with covariation included:
We ran short on time for much of a discussion on these tasks, but participants were looking at how scatterplots were used and thinking about how else students might reason with covariation. Of course, none of the tasks used in the session were meant to be perfect, and none teach themselves. Instead, it's important to find the most promising tasks we can, understand them as part of a greater curriculum, and attend to the student thinking and reasoning they might elicit.

Lastly, here are the resources we recommended at the end of the workshop. By no means is this an exhaustive list. It's just a place to start for teachers looking to supplement or revise their curriculum.
(Link to our presentationsource files, and handouts.)

References

Bakker, A., & Gravemeijer, K. (2004). Learning to reason about distribution. In D. Ben-Zvi & J. Garfield (Eds.), The challenge of developing statistical literacy, reasoning and thinking (pp. 147–168). New York, NY: Kluwer.

delMas, R. C. (2004). A comparison of mathematical and statistical reasoning. In D. Ben-Zvi & J. Garfield (Eds.), The challenge of developing statistical literacy, reasoning and thinking (pp. 79–95). New York, NY: Kluwer. doi:10.1007/1-4020-2278-6_4

Kilpatrick, M. (2010, August 4). Google CEO Schmidt: “People aren’t ready for the technology revolution”. Readwrite. Retrieved from http://readwrite.com/2010/08/04/google_ceo_schmidt_people_arent_ready_for_the_tech

Konold, C. (2002). Alternatives to scatterplots. Proceedings of the Sixth International Conference on Teaching Statistics (pp. 1–6). Cape Town, South Africa: International Association for Statistical Education. Retrieved from http://www.stat.auckland.ac.nz/~iase/publications/1/7f5_kono.pdf

Moritz, J. (2005). Reasoning about covariation. In D. Ben-Zvi & J. Garfield (Eds.), The challenge of developing statistical literacy, reasoning and thinking (pp. 227–255). New York, NY: Kluwer. doi:10.1007/1-4020-2278-6_10

Rubin, A., Bruce, B., & Tenney, Y. (1991). Learning about sampling: Trouble at the core of statistics. In D. Vere-Jones (Ed.), Proceedings of the Third International Conference on Teaching Statistics (pp. 314–319). Voorberg, The Netherlands: International Statistics Institute.

Saldanha, L., & Thompson, P. (2003). Conceptions of sample and their relationship to statistical inference. Educational Studies in Mathematics, 51, 257–270.

Shaughnessy, J. M. (2010). Statistics for all -- the flip side of quantitative reasoning. NCTM Summing Up. Retrieved from http://www.nctm.org/about/content.aspx?id=26327

Tukey, J. W. (1962). The future of data analysis. The Annals of Mathematical Statistics, 33(1), 1–67.

NCTM Denver 2013: Thoughts on Mayim Bialik's "The Power of Just One Teacher"

Mayim Bialik, known mostly for her role as Amy Farrah Fowler on The Big Bang Theory, gave the opening talk at this year's NCTM Annual Meeting & Exposition. I must admit that having just spent 48 hours listening to research talks, I was a little relieved to be able to just sit back, leave my notetaking devices in my bag, and listen.

Mayim Bialik

I wasn't expecting anything deep and ponderous; after all, this is typically a time for cheerleading and celebrating the teaching of mathematics. (In contrast, consider the tone with which the researchers kicked off the pressession.) Two parts of the talk I especially liked. First, Bialik is outspoken in her support for public schools. Her parents were public school teachers and she was educated in public schools, even though she needed a private tutor at times when she was busy acting as a child. Second, a lot of good, interesting, and fun things came out of the Q&A she did after her talk. For example, she admitted her struggles with calculus and organic chemistry in college, but she was able to perservere to get past that on her way to a PhD in neuroscience.

If there was a part I didn't like, it was that she talked about Texas Instruments for a few minutes too many. Yes, I understand that she's a spokesperson for TI, but there was a moment there when I thought the promotion was starting to take over the talk.

I was interested in hearing Bialik's responses to questions regarding women in science. She emphasized the need to see strong female examples in STEM fields, even when those examples might be fictional, like her character on The Big Bang Theory. That's not a bad answer, but I think at this point my more critical and feminist-thinking colleagues have tuned my ears to mentions of identity development and "disrupting hegemonic power structures" in conversations like this. Perhaps her next talk could be titled, "The Power of Just One Teacher in a Professionalized, Adequately Resourced Education System." I won't hold my breath waiting for that, but that's a conversation that needs to keep happening.

Still, I like the idea that a lot of teachers at this conference are going to return to their classrooms tomorrow and excite kids with their stories of seeing Amy Farrah Fowler. You can watch Bialik's talk for yourself on the NCTM website: http://www.nctm.org/conferences/content.aspx?id=36436. Jump to about 19 minutes in to see Bialik's introduction.

NCTM Denver 2013: Kosko's Piloting Online Professional Development for Facilitating the Common Core

Research Presession - Wednesday, April 17, 3:00 pm

Karl Kosko - Kent State University

As part of an interactive paper session, +Karl Kosko presented on efforts to use online tools in professional development around the Common Core Standards for Mathematical Practice. This work is likely still in its early stages, so the research presented involved the participation of five teachers and the first practice standard, "Make sense of problems and persevere in solving them." Using LessonSketch, teachers observed an animation of a classroom and could "pin" moments related to that practice.

Following the viewing of the animation, teachers' homework was to work the practice into an upcoming lesson. Finally, teachers reflected on the lesson in an online forum. Kosko was able to find interpretations of the standard that ranged from "vague" to "emerging" to "clear," but further details were somewhat limited by the short duration of the professional development.

Kosko suggested that more work needs to be done to transition teachers' emerging interpretations of the standard to clear interpretations. Despite some teachers marking almost everything as an example of the standard, there was an indication that using pins to mark examples of the practice was an effective strategy for developing an understanding of the practice standard, and the homework connecting the PD to teachers' classroom practice was essential.

NCTM Denver 2013: Remillard et al's Using Curriculum Materials to Design and Enact Instruction

Research Presession - Wednesday, April 17, 1:00 pm

Janine T. Remillard - University of Pennsylvania
Ok-Kyeong Kim - Western Michigan University
Luke Reinke - University of Pennsylvania
Napthalin A. Atanga - Western Michigan University
Joshua Taton - University of Pennsylvania
Dustin O. Smith - Western Michigan University
Hendrik Van Steenbrugge - Gent University
Shari Lewis - Aquinas College

This group, composed mostly of scholars from the University of Pennsylvania and Western Michigan University, reported on the findings of their ICUBiT project. The project, "Improving Curriculum Use for Better Teaching," has the goal of better understanding how teachers interact with their curriculum materials to design instruction. Using Brown's (2009) concept of pedagogical design capacity, the study looked at teachers using five different elementary curricula and assessed teachers' knowledge of the mathematics embedded in curriculum materials. The five curriculum were Everyday Math, Investigations, Math in Focus, Math Trailblazers, and Scott-Foreman.

Using interview data, curriculum logs, video recordings, field notes, pre-observation check-ins, and two teacher content knowledge assessments, the group looked for teacher actions and understandings related to transitions from the written to intended and intended to enacted curriculum (Stein, Remillard, & Smith, 2007). This was also related to fidelity of curriculum implementation and in-the-moment decision making.

Ok-Kyeong Kim discussing fidelity

Effort was put into understanding how each curriculum presented curriculum materials and how that affected what teachers read. Many used visual markers, putting answers in a different text color, and tips in special boxes. This led into a discussion of teachers' ability to identify the mathematical point of a lesson and then steer children in that direction. Using the framework of Sleep (2012), the group coded how mathematical points were defined, linked to, explained, decomposed, etc. You can get a small sense for the coding in this picture:

Part of a coding scheme for mathematical points

The group listed some of the factors that influence fidelity of implementation, like the student, policy, mathematical goals, content, situation, teacher experience, resources, and whether teaching was seen to be driven by curriculum or teacher knowledge. For this work they focused on influences related to the student, the mathematical goals, the curriculum, and teacher knowledge. A question in the Q&A asked about the influence of standards, and Remillard admitted that was an area that needed to be investigated.

Near the end of the presentation the team displayed some interesting "lesson maps" that help show the work of teachers and where they do curriculum design work. The graphs show the use of tasks across time in a lesson, the origin of the task, and ties to the mathematical point of the lesson.

A lesson map highlighting design decisions

In closing, Remillard cited a need for digital and "fluid" support for curriculum use. Static help in teachers' guides are limited in their ability to guide instruction, and "It seems sometimes that putting text in its own box is an invitation to not read it."

References

Brown, M. W. (2009). The teacher-tool relationship. In J. T. Remillard, B. A. Herbel-Eisenmann, & G. M. Lloyd (Eds.), Mathematics teachers at work (pp. 17–36). New York, NY: Routledge.

Sleep, L. (2012). The work of steering instruction toward the mathematical point: A decomposition of teaching practice. American Educational Research Journal, 49(5), 935–970. doi:10.3102/0002831212448095

Stein, M. K., Remillard, J. T., & Smith, M. S. (2007). How curriculum influences student learning. In F. K. Lester (Ed.), Second handbook of research on mathematics teaching and learning (pp. 319–369). Charlotte, NC: Information Age.

NCTM Denver 2013: Boaler's Using Research to Make a Difference

Research Presession - Plenary Session - Wednesday, April 17, 10:30 am

Jo Boaler, Stanford University

"I think math is the subject with the greatest gap between what we know works from research and what happens in classrooms."

Jo Boaler used her plenary session as a plea to the research community to not only do good work with their research, but to find ways to make it more shareable. "New knowledge is not enough," says Boaler. As part of a presentation she made at ICME in 2006, she surveyed teachers from countries around the world, asking, "What research study has made the biggest impact on your practice?" She found that in the Netherlands, the Freudenthal Institute has a huge impact in classrooms, and the Centre for Research in Pedagogy & Practice has a similar impact in Singapore. But often in other countries, including the U.S., teachers often cannot tie any of their practice to an influential body of research.

That doesn't mean that U.S. research hasn't had an impact on practice. When Boaler asked math ed researchers, Jeremy Kilpatrick pointed to Cognitively Guided Instruction (CGI) for having a long history of helping elementary teachers and their students. Boaler found other examples, but they were all from elementary. She suspected that elementary teachers are both eager to learn about students' mathematical learning, and they recognize the enormous capacity for learning they see in elementary students.

Boaler pointed out one of the tensions of sharing research: Just as we know students don't learn simply because we tell them something, teachers don't learn simply because we hand them a journal article. "I am really sure that teacher learning, one thing about it, needs to be generative and we need to be giving teachers things they can take back to their classroom and continue to learn from." Boaler also recognized doing research in partnership with teachers is probably the gold standard, both for identifying questions that are really important for teachers and working along side them.

Boaler stressed the need for "records of practice," including student work, curriculum materials, videos of classroom teaching, teacher notes, and assessments. "The most powerful impact I've had comes from showing these records of practice." As an example, Boaler showed a video of two classrooms in the same high school. The first classroom, as a result of the math wars, was a class that reverted from using IMP to a traditional approach. The teacher in the classroom is talking while they work out an example, and students sit quietly in rows. Engagement is passive -- at best, the students are listening. The second video Boaler showed was from a classroom down the hall. They were working a problem of a skateboarder spinning off a merry-go-round towards a wall, and the engagement in that classroom was much different. Before the students had solved the problem, the teacher stopped students to share their strategies. Students were active at the board and at their desks, questioning each other or reaffirming each others' strategies.

Using Research to Make a Difference

In the second part of her talk, Boaler used the metaphor of hedgehogs and foxes to describe a divide in the math education research community. Hedgehogs tend to look through the lens of one idea, while foxes are more interested in variety and looking across ideas. Jeremy Kilpatrick claims that mathematics education has an oversupply of hedgehogs, and it would be good for mathematics classrooms if there were more foxes in our field. (A chapter by Kilpatrick on this topic will be appearing in a book called "Vital Directions for Mathematics Education Research.")

Boaler then showed some of the successes and difficulties she's faced in her own journey as a researcher. Her book, What's Math Got to Do with It?, has brought focus to math education to a widespread audience. Boaler found herself talking on radio shows or in interviews for newspapers with people sharing their traumatic experiences as math students in traditional classrooms. Boaler also decided to reach out to press outlets, writing articles for newspapers, and was invited to speak with politicians about policies regarding math education. In those discussions, Boaler has found many opportunities to share the research of others, including the negative uses of timed tests and tracking.

Boaler shared difficulties related to her long-running scuffle with Jim Milgram and other mathematicians who strongly favor a more traditional form of mathematics. Boaler has told much of this story on her website, so I won't recap all of it here. She took this as an opportunity to thank people for the support she's received from so many people, and believes that while it's good to question and disagree, it's not okay to attack and attempt to discredit. There are, and likely will continue to be those in the math community who don't share visions for reform mathematics, as evidenced by this quote from Milgram: "Those of us who actually know the subject strongly believe that in order to improve outcomes we have to dramatically increase teacher knowledge of the subject, and that teachers need to more or less directly impart that knowledge to students." As a field we should work with people who want to cooperate and don't claim that some fields of research are inferior to their own.

Turning back towards ways we can improve math education, Boaler looked to new technologies for sharing research and records of practice. She's been working with Udacity and will be offering a course this summer called "How To Learn Math." The course will first be offered to learners, but she will be releasing a teacher version of the course. Boaler has recognized a lot of desire in the Silicon Valley community to improve the way we teach and learn mathematics, but they don't know the body of research in mathematics education to help make that happen. For example, Sebastian Thrun of Udacity once told her, "My friend Bill Gates tells me that algebra is the big problem for students in school." Boaler boldly replied, "Oh, Bill Gates, the math educator told you that, did he?" This opened a long conversation that exposed the curiosity of Thrun and others to find ways to teach math better, and the need to put the expertise of math educators in front of those who can drive change.

Photo courtesy of Rita Sanchez

Lastly, Boaler made four points for going forward and using research to make a difference:
  • Work harder and more intentionally to translate and communicate research to teachers and the public
  • Engage with different forms of media
  • Stand up to unethical behavior and academic bullying
  • Work as hedgehogs, foxes, and hedgefoxes to make math classrooms places in which all students thrive


NCTM Denver 2013: Saxe et al's Engagement in Mathematical Discussion: Linking Practices and Outcomes

Research Pressession - Wednesday, April 17, 8:30 am

Geoffrey B. Saxe - University of California, Berkeley
Maryl Gearhart - University of California, Berkeley
Ronli Diakow - University of California, Berkeley
Nicole Leveille Buchanan - University of California, Berkeley
Jennifer Collett - University of California, Berkeley
Bona Kang - University of California, Berkeley
Kenton De Kirby - University of California, Berkeley
Marie Le - University of California, Berkeley
Discussant: Deborah Loewenberg Ball - University of Michigan

This group from Berkeley presented their findings from the use of Learning Mathematics through Representations, or LMR. LMR is a research-based curriculum unit for the teaching of integers and fractions in the elementary grades. Despite only being 19 lessons long (at the time of their study), it carefully attended to students' definitions of number, unit intervals, subintervals, and early fraction sense. When compared to similar coverage by Everyday Math, LMR produced significantly higher learning gains at all stages.

I'd have been hard-pressed to take a worse picture than this.

The group did find variability in LMR results. After checking for curriculum coverage differences and not finding anything significant, they developed measures for both content and participation. The group found that communication was key, especially for the lowest-achieving students.

This research group also did intensive classroom observation and video collection. They looked at interesting teacher moves designed to disrupt student thinking in ways that elicited student protest, where students became motivated to express their understandings of concepts in ways that corrected the teacher's intentional mistakes. The group paid specific attention to the trajectory of student understanding about unit intervals. Over six weeks, with pre-, interim, and post-assessments, they showed how students' understanding of the fraction 8/7 grew (for most students) over time.

Saxe's anthropological approach to studying shifts in understandings over time add some theoretical nuance to this work, examining the semi-durability of ideas as they are reproduced and altered. This kind of detail is often lacking in research, but can provide some key insights about teaching and learning.

The Discussant, Deborah Ball, began her comments with "Wow." I think that says a lot. She commented on the project at a meta-level, about the project itself, and "being greedy," she asked questions about what else we can get out of this body of work. Ball appreciated the connectedness of the project, both to other projects and across the history of Saxe's work. She also appreciated how the work was situated across all aspects of instruction, including teaching, curriculum, and class discussion. Ball called the work "programmatic" in the ways it carefully broke down the issues of the study and carefully applied the right methods, the care taken with definitions, and the depth with which instruction was analyzed. Ball also asked about "correcting the teacher," wondering more specifically what they perceive that move/strategy to be, and how it might not fit into either direct or dialogic instruction. For her "greedy" questions, Ball asked:

  1. What are you learning about teaching?
  2. What are you learning about the challenge of "drop-in" curriculum?
  3. What are you learning about the assessment of student learning?
  4. What did you learn about who was talking in class? What were the relationships with social or identity markers? What are you learning about the use of problems that weren't situated in the real-world of the students?
  5. How can the rest of us learn how to do this kind of programmatic work?

To answer #4, the nature of the work and issues with Human Subjects precluded them from collecting demographic information about students. In the interest of time, the panel decided to take most of the other questions under considerations while allowing time for Q&A from the audience. The most interesting answer in the Q&A was in regards to the availability of the LMR curriculum. Saxe said they tried getting it published commercially, but commercial publishers want to sell K-5 series of textbooks, not a 19-lesson replacement unit. So instead, the group is planning to post all the materials online and make them free for teachers to use in their classrooms.

NCTM Denver 2013: Herbst et al's Methods to Study Decisions in Mathematics Teaching

Research Pressession - Tuesday, April 16, 3:00 pm

Pat Herbst - University of Michigan
Daniel Chazan - University of Maryland
+Karl Kosko - Kent State University
+wendy rose aaron - Oregon State University
Justin Dimmel - University of Michigan
Orly Buchbinder - University of Maryland
Ander W. Erickson - University of Michigan

There are many things about teaching that can be described, but some very important things can be very difficult to measure. This group is working on creating an infrastructure and set of methods to make quantitative claims about phenomena in teaching, such as teacher decisions, recognition of norms and obligations  mathematical knowledge for teaching, and teacher beliefs. Developing these methods should help get the field beyond "what works" to a place of "how things work," ideally resulting in a model where we can better justify certain teaching practices and predict the results of their use. In addition to exploring interventions in classrooms, or working with teachers watching videos of classrooms, the group is using LessonSketch, a cartoon-like technology that allows users to quickly create and/or watch classroom scenarios.

There are several aspects to this research. First is looking at the breaching of norms. Using LessonSketch, teachers watched animations of classroom practice and "pinned" instances of norms being breached. The group found that participants were more likely to pin and remark on actions that breached norms, with more experienced teachers being more adept at identifying the breaches.

Wendy Aaron, Pat Herbst, and Justin Dimmel (right to left)

Using similar tools, the group also looked at teachers' preferences toward different student strategies, evidence for hypothesized instructional norms, and different kinds of professional obligations. The group found that when a norm was breached, teachers' justifications for the breach could be categorized as: (a) disciplinary, (b) institutional, (c) individual, or (d) interpersonal. The group considered these categories as part of the collective knowledge of the teaching profession, even though teachers' obligations may not be indicative of personal acceptance.

+Karl Kosko dug into some of the measurement of decision making. Teachers were provided scenarios of teaching geometry that ended at key points, followed by four potential actions designed to assess the likelihood of norm compliance. Using multinomial regression, the group found that teachers' pedagogical content knowledge for geometry didn't have any measurable effect on decision making in this study, but years of experience and perceived autonomy did have an effect.

In the conclusion, Dan Chazan hoped that work such as this will move teaching practice away from being perceived as an independent variable that determines student achievement. Instead, teaching will be an "achievement verb," something that is dependent on a host of input variables that shape teaching practice.