Colorado Math Pathways Conference

On Monday, I attended the Colorado Math Pathways Conference, a gathering of faculty and department chairs from Colorado's 2- and 4-year colleges and universities, mostly representing mathematics departments, general education programs, and advising. In coordination with Colorado's Department of Higher Education, the Colorado Math Pathways Task Force has been examining the mathematics options required of Colorado's college students, and they've made these four recommendations:

1. Revise the curriculum to provide a calculus pathway, a statistics pathway, and a quantitative thinking pathway. Many students who would have previously taken college algebra would now take one of two courses: social and behavioral science majors would take statistics, and arts and humanities majors would take a course in quantitative reasoning.
2. Improve college advising through the creation of "meta-majors" with similar core and math requirements, and use multiple measures (beyond a sole placement score) to determine a student's readiness for college-level mathematics.
3. Support and develop instructors to teach pathway courses, especially the increased number of introductory statistics courses that will be needed.
4. Use multiple venues to improve communications about the pathways and their recommended use.

With these recommendations in mind, the purpose of the conference was to come together and work on some details of implementation. Colorado has thirteen 4-year institutions and fifteen more 2-year institutions, and a lot of effort over the years has gone into coordinating courses across the institutions to ensure transferability of general education courses.

Lieutenant Governor Joe Garcia led off the conference with comments about Colorado's successes in preparing undergraduates and our continued need to prepare more. Reflecting on a particular reform effort during his time leading Pikes Peak Community College, Garcia remembered a proposal to shift from three to four remedial courses prior to a gateway math course. When he looked at the data, he learned that fewer than 10% of students enrolled in remedial courses were actually making it to the gateway course. At Pikes Peak and elsewhere in Colorado, there has traditionally been more focus on enrollment and revenue generation than actually getting students through their programs. For an unfortunate number of students, they find themselves able to complete everything in their program except their mathematics requirement, and that is why the work of the Math Pathways Task Force is so important.

Dean Allison, the math department head at UNC, discussed the growing need for graduates to interpret and analyze data. For this reason, he and the task force want to join a national trend that moves away from college algebra as the default math course and towards courses in statistics and quantitative reasoning. This will require a lot of effort on the part of departments, advisors, and instructors.

The task force has been working with the Dana Center at the University of Texas at Austin, a national leader in mathematics education who has been working with multiple states on math pathway revisions. Uri Treisman keynoted the conference to describe some of the efforts, trends, and findings he's seen across the country. He said that math was currently under two different spotlights: the first, a harsh spotlight, that shows a lack of completion and high failure rates, and a second, a positive spotlight, where students with skills in mathematics are more successful negotiating rough job markets and becoming upwardly mobile in society.

Treisman continually stressed that the reality of today is different than the reality of 10 or 20 years ago. Right now, three times as many students take calculus in high school (almost 25% of all high schoolers) than in higher education, and those students tend to succeed at high rates in college. Studies of transcripts reveal that those succeeding in college calculus almost certainly took calculus in high school. Only about 1% of students going beyond college calculus to a higher level course (like differential equations) started college in college algebra. We've been placing students in college algebra to start them on a path towards math-intensive majors, but almost no students are actually following this path to get there. It's not that we haven't been trying to make this path work, either. At a meeting of community college leaders in Ohio, Treisman had the group list all the various initiatives for improvement that they've tried. They managed to list a total of 81 initiatives, most of which lasted 2-3 years, and zero of them were sustained long-term. People work hard, but too often people vital to the effort move elsewhere, money dries up, or 4-year institutions don't cooperate.

Uri Treisman and Amy Getz, comparing the success rates of a co-requisite support model vs. a pre-requisite support model.

Treisman recommended four coordinated innovations needed in new pathways:

1. Relevance: Too often college advisors accept students' wishes when it comes to course placement, rather than really trying to align the student's interests with the mathematical skills required of various majors. Placement tests haven't helped this, either. Said Treisman, "No longer should a placement test ensure a student is misplaced into a course, supported by a criminal enterprise of testing companies."
2. Acceleration: We've been trying to help students catch up by slowing them down in remedial courses, but we've learned that about two-thirds of students can succeed in regular courses when they're given proper support. There was a belief that these supports would ensure success for all students — a belief that was "polished by hope and unfettered by data" — but we now recognize that to see success for this other third of students it will take a national, Manhattan-type project to figure it out.
3. Narrow the gap between support and instruction: At Northwest Vista Community College in Texas, an advising "SWAT Team" crashes classes to confront students about why they're enrolled in the class they're in. About a quarter say it's because it fit their schedule, not because it's the class they need. They also aggressively supply support to students in the first three weeks of a course, before students slip behind or think there's a stigma about asking for help.
4. Introduce evidence-based strategies into instruction: Treisman says this may very well be the hardest part, yet it's less important relative to the structural improvements needed and described above.

Treisman then pointed out some key challenges. For one, math departments need to come to the realization that tradition shouldn't be the dominant force in program design. Math departments also need to push back against other programs, like business schools and psychology departments, who often use math prerequisites to weed students out of their programs. Math departments need to be a partner instead of a service discipline that is often placed as a barrier to student success. As Treisman said, "We no longer want math departments to be a burial ground for tens of thousands of people wishing to better their lives."

Amy Getz, also working with the Dana Center, spoke passionately about needed shifts in our thinking. We need to think beyond the classroom and past our old ways of unnecessary prerequisites, predictable outcomes, inconsistent course alignments, and a lack of clear messaging about math requirements.

The rest of the day was structured around breakout sessions and whole-group debriefs of those sessions. In the morning, the breakouts were organized around each of the quantitative reasoning, statistics, and calculus pathways. In the "StatsPath" breakout, I learned that the CU-Boulder math department is preparing to drop college algebra altogether, knowing that the easiest way to get advisors to stop recommending it would be to eliminate it. The discussion moved towards the details of the content of a prerequisite course students might take before statistics, but Uri redirected the conversation back to structural elements, which led to good conversations about transfer credits, the need for leadership, and developing the capacity to offer new courses. People in the quantitative reasoning breakout talked about how to work with partner disciplines, while the calculus path group reported on various ways to provide student support.

Afternoon breakouts centered on issues of advising, degrees with designation, and on-ramps for under-prepared students. With advising, we talked about the need to coordinate higher ed and high school advising, and to see if the meta-major model really helps advisors and students make decisions about the math courses that align with student interests. The degrees with designation group talked more about working with partner disciplines to determine the learning outcomes they want and why, and putting people in positions to advocate and create change in the system. The on-ramps group focused on issues of communication, including things like email lists, websites, blogs, virtual open houses, and coordinating with organizations like the Colorado Mathematics Association of Two Year Colleges (ColoMATYC), the Colorado Association for Developmental Education (CoADE), the Rocky Mountain Section of the MAA, and CCTM.

A Madness to Our Methods

How do I learn to teach people to use this stuff?

When I was an undergraduate majoring in mathematics teaching, I got quite a bit of practice teaching, studying curriculum, writing lessons, and other things I'd be expected to do as a teacher. But now, as a PhD candidate in mathematics education, I'm not getting similar training to become a teacher educator. I've done a little teaching of introductory classes for math and science preservice teachers, but that's about it, and I don't think my experience here at CU-Boulder is an exception. Here and elsewhere, how you teach undergrads is still largely something you're supposed to figure out on your own.

It may not be long before I have a job that involves me teaching "methods" courses. I look forward to that opportunity, but dread the feeling that I'd be creating such courses essentially from scratch. It's happened to a few colleagues of mine, and it seems a bit silly that we folks in curriculum and instruction don't have more organization and purposeful, shared design in our curriculum and instruction for preservice teacher methods courses.

Here at CU-Boulder we have a math and science seminar that meets about every three weeks and the topics of the seminar change year to year. This year, pushed by myself and a few others, some of us (including @jybuell) are studying the design of methods courses in math and science. As a first step, we're looking at what others are doing elsewhere, and here's where I'd like some help. Do you have a syllabus or story to share about methods classes you've taught or taken? If you send those my way (to raymond@mathed.net) or comment about them in the comments, I'll continue to write about what I find and learn as the year progresses, and hope to have methods course lesson plans and scope and sequence documents to share by the end of the year.

Monday Night at NCTM: Poster Session and Kenneth Zeichner's Plenary Address

Braving the snow, Ryan Grover and I slogged our way via RTD from Snowy Boulder to Almost-as-Snowy Denver. We arrived in time for a poster session. If you're unfamiliar with poster sessions, I'd describe them as a sort of academic mingling punctuated by awkward moments of silence as you read people's posters before deciding if the person is worth talking to. Yes, it's as fun as it sounds. I had one extended conversation with Sara Lohrman Hartman from Ohio University. Her recent dissertation work in rural Appalachia studied how a small school utilized a math coach. That's a rare luxury for a small rural school, at least in my limited experience.

As 7:00 neared I made my way to the plenary session and happily found Amanda Jansen standing outside the door. I'd never met Amanda in person, but I've conversed with her regularly on Google+ for more than a year. In that time she's made time for me and put up with my questions and rambling; it may not mean much to her, but as a grad student it means a lot to be welcomed by others in the field. We took some seats midway back in the room and settled in for a talk from Kenneth Zeichner, a well-known expert in the field of teacher education. His talk was not specific to mathematics education, but still of great interest for anyone who cares about how teachers are prepared and the institutions that prepare them.

My horrible picture of Ken Zeichner

As a student in Jennie Whitcomb and Dan Liston's Research on Teaching and Teacher Education class this semester, Ken's talk was very familiar. (If you've ever been told of the importance of reflective teaching, Ken and Dan literally wrote the book on the subject.) Ken began by framing three perspectives in the current debate about teacher education. First are the defenders, those who want to preserve the function and status of college and university-based teacher education (UBTE). Next are the reformers, those who seek to deregulate teaching and let the quality of teachers and teacher preparation be dictated by free market principles. Ken frequently cited Rick Hess as a proponent of this perspective, and shared George Will's belief that closing ed schools was the surest path to improving education in the U.S. Last are the transformers. This group, and where Ken places himself, wants reform of teacher education, but in a way that builds upon the capacity and expertise of UBTE and preserves a democratic, public system not driven by market forces.

From here, Ken shared what he sees as the two visions for preparing quality teachers. One is a system that produces professionals for a teaching profession; the other is a system that is technocratic and highly skill-driven, with teachers focused on actions that raise test scores in a narrow curriculum. Teaching skill is important, admits Zeichner, but professionals go beyond the management of a classroom to understand the social and political context of schools and their communities, and have skills that allow them to better reflect in ways that positively impact their teaching.

There are about 3.6 million teachers in the U.S., and alternative pathways for training those teachers (TFA, teacher residency programs, etc.) have grown since the early 1990s. Still, somewhere between 70-80% of the nation's teachers have earned licensure through UBTE at one of the almost 1400 college- and university-based programs around the country. Even if we were to deregulate teacher education, the market does not (and probably would not) have the capacity to prepare teachers without the help of UBTE.

Zeichner admits that UBTE has gotten a lot of non-critical, negative attention from the media. We've come to believe that too many teachers are not getting good results and efforts are now underway to trace poor student results back to the programs that prepared the teacher in those classrooms. When a recent bridge collapsed, said Zeichner, "The engineer who designed the bridge was exposed in the media but I don't remember anyone asking what engineering school he attended." Using another analogy, Zeichner said judging teacher education this way was similar to the idea that we should judge medical schools by patient outcomes. The result, says Zeichner, would be doctors avoiding treating those who need treatment most, just as teachers will avoid teaching in the tough schools with histories of poor test scores. This will lead to further inequality in how quality teachers are distributed, something the U.S. -- which was recently ranked 26th out of 29 nations in a UN report about child well-being -- cannot afford to have happen. The reformers' cries of "No Excuses!" does not rid us of the ails of poverty, no matter how good the teachers might be.

Zeichner addressed some key contradictions in the teacher education argument. For example, the Obama administration has pushed for higher standards for students while simultaneously pushing for alternative pathways to teaching that have lower standards. Also, as Mike Rose has pointed out, the logic of removing teacher education requirements and regulation, or to disregard teaching experience as a measure of quality, makes little to no sense in any other field. (Nobody would trust a neurosurgeon who hadn't been properly trained and lacked experience.) Further, no high-achieving country has deregulated their system of teacher education. Countries like Finland and South Korea have set their standards high, not lowered them.

Zeichner admits that messages like this, and his affiliation with schools of education, have led many to see him as a defender of UBTE. Zeichner insists that he's not a defender, but a transformer. Instead of deregulating UBTE, he sees a great need to change, yet not replace, our current system. This transformation would shift UBTE closer to the world of practice, providing a clinical experience for preservice teachers in a hybrid approach. This move would require the sharing of responsibility and accountability for teacher preparation with schools and their communities, a shared space with values that would prioritize democratized knowledge. Zeichner never advocated for this as a cure-all, but it is the approach most likely to support a system of effective, professionally prepared teachers.

Q&A

(Note: These quotes from Zeichner are rough approximations. Caveat emptor.)

Q: An attendee from Chicago asked about the dismantling of public education in Chicago and the closing of public schools in lieu of charters.

Zeichner: "I'm not against charter schools. I'm against disempowerment of parents and local stakeholders." Zeichner says he was seeing similar school closings in his home of Philladelphia, where public schools are being closed and replaced by charters run by charter management operators. A new study should be out soon that shows the ineffectiveness of these approaches so far. "There's a lack of democratic debate. I'm not for the alternatives to go away, but I'm for an honest debate."

Q: Why do educational entrepreneurs think they're going to make a lot of money?
A: Zeichner: "Some alternative routes use non-profit status for the tax breaks and they outsource their services. From a distance it's hard to see where they're making money, but they are. ... I'm not implying anything about morality. Greedy, self-serving attitudes are everywhere, including universities. But there is a lack of transparency about what's going on. It astounds me how the NewSchools Venture Fund has so much influence while flying under the radar."

Q: I've done UBTE and "quick fix" teacher education programs. Quick fix doesn't work, and online programs are dangerous. Still, too many teachers don't really their profession like a true profession.
A: Zeichner defends online education to a degree, having been involved with some online programs that offered a lot of quality. "I'd be careful in discounting all online programs. Some universities are using them along with more traditional approaches. They can be improved and we still have a lot to learn. We need to be careful about demonizing anything just based on their sponsorships or affiliations."

Q: What are things a new generation of mathematics educators should fight for in public education? How do we take our conversations outside the education community?
Zeichner: "For me, I've decided that ed schools alone will not accomplish much. The ownership of teacher education needs to be broader and shared with communities. I recently met with a lot of stakeholders and listened to their stories. There's a lot of community activism in places like Chicago and Philadelphia, and we can join those efforts already underway instead of coming in from above with solutions to their problems. I wish I'd been more in the public sphere in my career, writing more op-eds and participating in forums, even if many would have disagreed with me. Now, the polarization we're left with is harmful."

Q: How do we measure low-performing teacher education programs?
Zeichner: "I agree with the National Research Council's report on improving teacher education, and that the profession itself needs to take care of its own accreditation and monitoring. I'm afraid that test scores and value-added models will fill this role at great cost and distraction, as simply ranking as a way to improve quality has never really worked. To anoint these people who know little about the field as saviors is idiocy. Twenty-five years from now people will look back and wonder how we could have been so stupid."

(Mic drop. Not really, but I wish.)

My Thoughts

For me, Zeichner's serious (and some might say somewhat depressing) talk was balanced by a sense of validation. Not only did I recently write a paper that expressed many of these same ideas, it gives me hope that the system I'm a part of is recognizing a need and possibility for significant change. For some people, I'm sure a reformed/deregulated system of teacher education seems like a remote possibility, but right now in Colorado we're anticipating the introduction of legislation proposing to do exactly that, removing the requirement for teacher education as we know it. If that bill appears and makes progress, you'll be sure to hear more from me.

OpenComps: Validity and Causal Inference

With the start of my comprehensive exams beginning in 12 days, my studying has hit the homestretch. Thankfully, my advisor has inspired some confidence by telling me that my understanding of the math education literature is solid and I won't need any more studying in that area. That's good for my studying, and something I take as a huge compliment. So now I can focus for a while on preparing myself for the exam question Derek Briggs is likely to throw my way. Typically, one of the three people on a comps committee is tasked with asking a question related to either the quantitative or qualitative research methodology we learn in our first year of our doctoral program. Derek is a top-notch quantitative researcher, and I enjoyed taking two classes from him last year: Measurement in Survey Research and Advanced Topics in Measurement. Where this gets slightly tricky is that Derek didn't actually teach either of my first-year quantitative methods courses, so there's a potential I could get surprised by something he normally teaches in those classes that I didn't see. It's a risk I was willing to take after working with Derek more recently and more closely in the two measurement courses last year.

It certainly won't be a surprise if Derek asks a question that focuses on issues of validity and causal inference. He mentioned it to me personally and put it in a study guide, so studying it now will be time well spent. I feel like I've had a tendency to read the validity literature a bit too quickly or superficially, so this is a good opportunity for me to revisit some of the papers I've looked at over the past couple of years. Here's the list I've put together for myself:

AERA/APA/NCME. (1999). Standards for educational and psychological testing. Washington, D.C.: American Educational Research Association. [Just the first chapter, "Validity."]

Angoff, W. H. (1988). Validity: An evolving concept. In H. Wainer & H. Braun (Eds.), Test validity (pp. 19–32). Mahwah, NJ: Lawrence Erlbaum Associates.

Borsboom, D., Cramer, A. O. J., Kievit, R. A., Scholten, A. Z., & Franic, S. (2009). The end of construct validity. In R. W. Lissitz (Ed.), The concept of validity: Revisions, new directions, and applications (pp. 135–170). Information Age Publishing.

Brookhart, S. M. (2003). Developing measurement theory for classroom assessment purposes and uses. Educational Measurement: Issues and Practice, 22(4), 5–12. doi:10.1111/j.1745-3992.2003.tb00139.x

Chatterji, M. (2003). Designing and using tools for educational assessment (p. 512). Boston, MA: Allyn & Bacon. [Chapter 3, "Quality of Assessment Results: Validity, Reliability, and Utility"]

Cronbach, L. J. (1988). Five perspectives on validity argument. In H. Wainer & H. I. Braun (Eds.), Test validity (pp. 3–17). Hillsdale, NJ: Lawrence Erlbaum.

Eisenhart, M. A., & Howe, K. R. (1992). Validity in educational research. In M. LeCompte, W. Milroy, & J. Priessle (Eds.), The handbook of qualitative research in education (pp. 642–680). San Diego, CA: Academic Press.

Gorin, J. S. (2007). Test design with cognition in mind. Educational Measurement: Issues and Practice, 25(4), 21–35. doi:10.1111/j.1745-3992.2006.00076.x

Haertel, E. H., & Herman, J. L. (2005). A historical perspective on validity arguments for accountability testing. In J. L. Herman & E. H. Haertel (Eds.), Uses and misuses of data for educational accountability and improvement (NSSE 104th., pp. 1–34). Malden, MA: Wiley-Blackwell.

Holland, P. W. (1986). Statistics and causal inference. Journal of the American Statistical Association, 81(396), 945–960. doi:10.2307/2289069

Kane, M. T. (1992). An argument-based approach to validity. Psychological Bulletin, 112(3), 527–535. doi:10.1037/0033-2909.112.3.527

Leighton, J. P., & Gierl, M. J. (2004). Defining and evaluating models of cognition used in educational measurement to make inferences about examinees’ thinking processes. Educational Measurement: Issues and Practice, 26(2), 3–16. doi:10.1111/j.1745-3992.2007.00090.x

Linn, R. L., & Baker, E. L. (1996). Can performance-based student assessments be psychometrically sound? Performance-based student assessment: Challenges and possibilities (pp. 84–103). Chicago, IL: The University of Chicago Press.

Messick, S. (1988). The once and future issues of validity: Assessing the meaning and consequences of measurement. In H. Wainer & H. I. Braun (Eds.), Test validity (pp. 33–45). Hillsdale, NJ: Lawrence Erlbaum.

Michell, J. (2009). Invalidity in validity. In R. W. Lissitz (Ed.), The concept of validity: Revisions, new directions, and applications (pp. 111–133). Information Age Publishing.

Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference (p. 623). Boston, MA: Houghton Mifflin. [Probably Chapters 1-3 and 11, if not more.]

Shepard, L. A. (1993). Evaluating test validity. Review of Research in Education, 19(1), 405–450.

Shepard, L. A. (1997). The centrality of test use and consequences for test validity. Educational Measurement: Issues and Practice, 16(2), 5–24. doi:10.1111/j.1745-3992.1997.tb00585.x

Zumbo, B. D. (2009). Validity as contextualized and pragmatic explanation, and its implications for validation practice. In R. W. Lissitz (Ed.), The concept of validity: Revisions, new directions, and applications (pp. 65–82). Information Age Publishing.

Thankfully, some of these papers I've read recently for my Advances in Assessment course so the amount of reading I have to do is appreciably less than it might look. In my typical fashion, I'll study these in chronological order with the hopes that I get a sense for how the field has evolved its thinking and practice regarding these ideas over the past several decades.

Although I have little other graduate school experience to compare it to, I feel like this reading list is representative of what sets a PhD apart, particularly one earned at an R1 university. It's not necessarily glamorous, and its relevance to the day-to-day teaching and learning in classrooms might not be immediately obvious. But without attending to issues like validity and causal inference, we have a much more difficult time being sure about what we know and how we're using that knowledge. Issues of validity should be at the heart of any assessment or measurement, and when they're attended to properly we greatly improve our ability to advance educational theories and practice.

Scholarly Reading Strategies

While I welcome greater diversity in higher education, I recognize graduate studies aren't for everybody. More specifically, I'd suggest you think twice about a PhD if you're the kind of person who doesn't like to read. The written word is the stuff on which academia survives and thrives, and as such many more scholarly words are produced than any one person could possibly read. But yet our work depends on reading huge chunks of scholarly literature.

I was only a few weeks into my first semester as a PhD student when I realized that there were going to be times when I couldn't finish all of the assigned readings for class. Thankfully, the ever-kind Elizabeth Dutro addressed this problem in class and told us all that this was okay. Yes, sometimes there were things we'd need to understand in great detail, but other times it was enough to just gain familiarity with an article in case we needed to refer to it later. Some readings (for me, Foucault comes to mind) need multiple readings before they make any coherent sense.

I discussed this with my advisor at the time, Finbarr (Barry) Sloane. Knowing that he was a voracious reader with incredible retention and memory (Vicki Hand once told me she wished her internet connected directly to Barry's brain), I asked if he had any special reading strategies. This is essentially what he told me:

I read things three times. The first time I just read and get a sense for the article. The second time I read for details, take notes, and make connections. On the third reading, I read the article out-of-order. If I can read paragraphs or sections at random and understand them without having to re-read the surrounding context, then I know I understand it.

Now I was understanding why Barry's knowledge of the literature was so strong. Unfortunately, I was also understanding why he routinely only got a few hours of sleep every night -- all that reading and re-reading takes time. He wasn't shy about his love of reading; he said that while in graduate school in the mid-1980s, he read every single article in the Journal for Research in Mathematics Education since its first publication in 1970. That's intense.

Maybe I can't read every JRME article three times between now and my comprehensive exams, but I do need to make the most of my comps readings. So long as the quantity of reading doesn't overwhelm me, my three-part strategy will be (a) read, (b) read for detail and take notes, and (c) blog a summary. That's the approach I took with my last post and I felt very good about it. (It helped that the Clements & Sarama article was less than 10 pages long.) The written part of my comprehensive exam gives me a week to answer three questions with essays/reports of about 8-10 pages each. I figure the more I've written on my blog, the more prepared I'll be to write for comps. There's also the side benefit of giving my advisor a convenient way to keep up with my preparation while he's traveling during his sabbatical this semester. I'd love to blog about at least four or five readings a week, and you'll be the first to know if I can keep up that pace.

Nielsen's Reinventing Discovery (2011) in the Context of Education Research

As a Ph.D. student I've taken my share of methods courses, giving me skills in everything from ethnography to ANOVA. But as important as those things are, I've sensed that there are new research methods emerging thanks to technological advancements and online communities. Our lives are too data-rich and our means of communication are too plentiful to limit ourselves to the same methods for research -- and learning -- that we used just 10 years ago.

Even though I feel like I live in the thick of this revolution, engaging with teachers and researchers on Google+ and Twiter, I wanted a broader perspective on how researchers use networks to make new discoveries. For this I turned to Michael Nielsen's book Reinventing Discovery: The New Era of Networked Science. Although Nielsen is a pioneer in quantum computing, I hoped to find some ideas that I could apply to a social science like education research.

Nielsen uses a variety of examples and concepts to describe what works and what doesn't (or hasn't) in networked science. Instead of listing them here, watch this TEDx talk by Nielsen:


If that talk wasn't long enough for you, Neilsen held a longer talk at Google that is worth checking out.

As much as I like Neilsen's example of Tim Gowers's Polymath Project, I can't imagine a direct translation to education research. One of the beautiful aspects of mathematics is that it usually doesn't require conducting an experiment, interviewing subjects, sampling a population, or agreeing on a conceptual framework -- the kinds of things that make social science untidy and difficult. Frankly, if solving problems in education were structured like proving mathematical theorems, I think we'd be solving more problems and finding better solutions than we are currently.

Neilsen's story about Qwiki hits home for me. For some time now I've imagined creating and maintaining a wiki that essentially translates the contents of the NCTM's Second Handbook of Research on Mathematics Teaching and Learning into knowledge that teachers could access and use. Just like Qwiki, it's easy to get math teachers and educators to agree that this would be a great resource to have. Unfortunately, I'm not sure how a math education wiki like the one I've imagined would avoid Qwiki's fate. Without incentives for experts to contribute and maintain the site, I'd probably spend more time fighting spam than helping teachers.

Neither the Polymath Project or Qwiki offer a blueprint for a new kind of mathematics education research. Thankfully, Nielsen describes some general characteristics for successful networked science. First, in his chapter titled "Restructuring Expert Attention," Nielsen suggests networked science has these attributes:

• Harnessing Latent Microexpertise -- The project must allow even the narrowest of expertise. A 3rd-year algebra teacher might not have the broad expertise of an experienced math education researcher, but that 3rd year teacher might have small elements of expertise that exceed that of the recognized experts.
• Designed Serendipity -- The project needs to be easy to follow and encourage participation from a variety of experts. You want problems to be seen by many in the hopes that just a few will think they have a solution they wish to contribute.
• Conversation Critical Mass -- One person's ideas need to be seen by others so they create more ideas, and the conversation around all the contributions keeps the project going.
• Amplifying Collective Intelligence -- The project should showcase the fact that collectively we are smarter than any one individual.

Those are all great characteristics of any project. But what makes this any different than any traditional, offline project? Nielsen offers several suggestions. Unlike a large group project with clear divisions of labor, technology allows us to divide labor dynamically. Wikipedia certainly would not have grown the way it did if labor had been divided statically between a set of contributors. Also, networked science uses market forces to direct the most attention to the problems of greatest interest. Lastly, contributing to an online project rarely feels like committee work, and participants can more easily ignore poor contributions or disruptive members.

Projects like Wikipedia and Linux exhibit the above attributes, but Nielsen explains that such projects needed something extra in order to scale to thousands of participants. Nielsen describes these in a chapter called "Patterns of Online Collaboration," and they are: (1) being modular, (2) encouraging small contributions, (3) easy reuse of earlier work, and (4) signaling to what needs attention. When I look at this list and think of Wikipedia, I can see how well a wiki or open source software project fosters these patterns. But how do we build such a project in education? Given Nielsen's framework above, a project that would interest me needs three key aspects:

• The content of the project has to be something that both teachers and researchers can contribute, such as a collection of math tasks, curriculum plans, or perhaps pedagogical techniques.
• Teachers need to be able to easily use and modify each other's content.
• (This one's the crux!) When teachers use content, there needs to be a way to collect and submit feedback about the use of that content, and that feedback becomes data that researchers can use not only to improve the content of the site, but to produce new and traditional reports of research.

It's that last bullet that's the hardest but most intriguing. There are so many places to get lesson ideas on the internet, but I don't know of any that collect data about the effectiveness of the lesson in a format suitable for research. Khan Academy claims to do this this kind of data collection internally, but KA is a closed project that lacks nearly all of the attributes Nielsen has described in his book. The project I want needs to be an open one, with all of its moving parts exposed and no more owned or identified with a single participant as Jimmy Wales is identified with Wikipedia. If you have ideas for what such a project could/should look like, leave them in the comments!

Bonnie H. Litwiller, 1937-2012

I got word tonight that my undergraduate advisor, Bonnie Litwiller, passed away a couple days ago at the age of 74.

As a freshman at UNI, I had a temporary advisor until my program became more certain. After declaring as a math education major, Bonnie Litwiller was assigned as my advisor. I knew nothing about her. I remember asking Ed Rathmell, whom I had gotten to know while applying for a scholarship, what to expect from Litwiller as an advisor. I remember his response: "If you listen to her and do what she asks, she's great. She'll have your back when you need something. But don't cross her."

That's an uneasy way to know someone before you even get a chance to meet them. It felt like a description more fitting of mafia boss than a professor. But Rathmell's advice, as usual, was solid. Litwiller proved to be tough, and she made it clear to us that being a good math teacher was hard work. She set a good example: she and her research partner, David Duncan, would set aside a day a week where they'd lock themselves away in the library and write. As UNI isn't a top-level research university, the research activities of professors aren't always visible to the students. But Litwiller's dedication to research was clear, and there was no topic too small or journal too obscure. If she thought she had knowledge that would improve the teaching and learning of mathematics somewhere -- anywhere! -- she would write and submit for publication. She continued to write and publish even after her retirement from UNI in 2000, eventually passing the almost unfathomable mark of 1000 scholarly publications.

I took two classes with Litwiller, Teaching Middle School Mathematics and Teaching High School Mathematics. The classes were tough due to Litwiller being both picky about the quality of our work and her lack of clarity in describing what she wanted us to do. Some of us thought she was just being careless with her assignments, but I always wondered if this wasn't somehow purposeful. Either way, it was clear that she didn't want to do a lot of hand-holding. Some of us, ever so quietly yet respectfully, referred to her as the bulldog. A trusty companion that might just bite if you got out of line. If you didn't have the initiative and sense of responsibility to do quality work, I think she wanted a way for you to sort yourself out of the program. It happened, too; every semester some classmate would go missing and we'd try to find out what happened. Inevitably, someone would say, "They couldn't cut it. Litwiller dropped them from the program." You hear a lot today about colleges adopting GPA or test score requirements to improve the quality of their education majors. We didn't have those -- we had Litwiller. And just like letting a GPA decide who can be a teacher, I'm sure her judgement wasn't perfect and mistakes were made. (An acquaintance of mine, who shall remain nameless but now holds a PhD in math education, told me about a narrow escape from Litwiller's axe after a dispute over access to a local school.) But I think Litwiller had a sense for the quality that people expected from a UNI-prepared teacher, and a sense for giving us some survival skills that would get us through our first few years of teaching. There must have been far more successes than failures, too -- by the time I graduated in 1999, someone had estimated that a quarter of all the math teachers in the State of Iowa had been taught by Bonnie Litwiller.

My appreciation for Litwiller and her work has grown through my years first as a teacher and now as a graduate student in mathematics education. It was she who first introduced me to the NCTM Standards, and her direction of the NCTM Addenda Series was and still is an enormous contribution to the field of math education. What I believe was originally intended to be a six-book series to support the Standards grew into 22 total books, each designed to take the research behind the Standards and turn it into something teachers could use. Litwiller might have been the director and not the author of the Addenda Series, but it carried her trademark: getting as much useful information into the hands of teachers as possible. She gave me two books from the series, the middle school and high school books about statistics and data analysis. It was the first time I really thought about statistics education, and it's since become the area of school mathematics I find most interesting.

The world will miss Bonnie Litwiller, but she didn't leave without making a mark, both on the field of mathematics education and on me. Teacher education is a challenging business, and it's probably best to judge it with a certain amount of hindsight. For all of her toughness, she did have my back when I needed it, just as Ed Rathmell said she would. I may not have learned all that she tried to teach me, but maybe her most important lessons -- a sense of dedication and rejection of "good enough" -- have been most helpful in getting me to where I am today.

The Research Works Act and the White House OSTP

Imagine Company X proposed a law that said if they added value to a public highway -- such as by organizing volunteers to pick up trash on the side of the road -- then that gave Company X ownership of the road and the rights to charge the public tolls to use them. Sounds crazy, right? Well, replace "Company X," "public highway," and "pick up trash" with "Reed Elsevier," "publicly-funded research," and "peer review," respectively, and you've basically got the Research Works Act, a bill currently in the U.S. House of Representatives. If passed, the Research Works Act would prohibit federal funding agencies (such as the National Institutes of Health) from requiring that the research they fund (with your tax dollars) be available to the public. Instead, publishers could restrict access to any research they add value to (such as coordinating volunteers for peer review) for profit. Where does that profit come from? Usually from the high subscription fees paid by research universities, money often obtained from public funds and tuition dollars. The effect is that taxpayers are paying twice for research that, in many cases, they still don't have access to.

Fortunately, people are paying attention. Michael Eisen's op-ed in the New York Times explains the Research Works Act and its potential harm to research and scholarship, and plenty more articles on the subject are easily found. Somewhat coincidentally, we are also at the end of a feedback period for the White House Office of Science and Technology Policy (OSTP), where they have made a formal Request for Information (RFI) about open access to scientific publications. Yes, the Research Works Act should be stopped and seen as little more than a request by for-profit publishers to continue having their work (along with their alarming profits) subsidized by tax dollars. But I don't think stopping one bill is enough. Instead, I hope to see all federal funding agencies adopt policies similar to those of the NIH. I expressed these hopes in an email today to the White House OSTP, the text of which I've copied below.

---

University of Colorado at Boulder
School of Education
249 UCB
Boulder, CO 80309-0249

January 12, 2012

To: Office of Science and Technology Policy
Executive Office of the President
725 17th Street Room 5228
Washington, DC 2050

From: Raymond C. Johnson, Doctoral Student in Mathematics Education
School of Education, University of Colorado at Boulder

Re: Response to the White House RFI on OA publications

I am a researcher, concerned citizen, and a supporter of open, public access to publicly-funded research. I speak for myself and not on behalf of my colleagues or my institution, although I believe I express ideas and opinions shared by many researchers and educators. In response to the White House Office of Science and Technology Policy request for information on “Public Access to Peer-Reviewed Scholarly Publications Resulting From Federally Funded Research,” I urge you to preserve policies that require public access (such as from the National Institutes of Health) and expand similar policies to other federal funding agencies such as the National Science Foundation, a key source of funding for education research in mathematics, science, and technology. Currently it is with great jealousy I see the growth of open access publishing in areas such as health and medicine; as an education researcher I wish I had the ability to share the latest research with teachers and administrators, most of whom cannot afford the high fees charged by publishers of education research. Unfortunately, open access journals in education are relatively rare and undervalued. A change in policy, one that would require public access to federally-funded research, would quickly change the perceived valuation of open access publishing outlets and bring much-needed information to educators everywhere.

Prior to my becoming a researcher I was a high school mathematics teacher for six years in high poverty, rural Colorado school districts. I did not have the benefit of a nearby university or a district support staff with access to recent or prominent research. My main link to information was a powerful one: the internet. However, it seemed that my searches for research about teaching methods, curriculum, education policy implementation, etc., all eventually led me to paywalls put up by publishers to “protect” their work, requesting fees I could not afford to pay. Now, as a researcher, I realize that the authors of education research -- much of it funded with federal dollars -- are asked to give their copyrights to publishers in exchange for so-called “widest possible dissemination” of that research. Researchers neither receive nor expect any pay or rewards for giving away their work, other than some scholarly esteem and the hope their research somehow reaches and benefits students and educators. While that publishing model might have made sense twenty years ago, it does not any more. Any claim of “widest possible dissemination” that currently does not include searchable, full-text publication on the public internet is false, at best, and fraudulent, at worst.

In response to the eight questions in the RFI, I encourage you to consider the arguments and recommendations made by Harvard University in their response (http://osc.hul.harvard.edu/stp-rfi-response-january-2012). Their expertise in these matters far exceeds mine. However, I do wish to make the following amendments to their responses for questions (2) and (7):

(2) What specific steps can be taken to protect the intellectual property interests of publishers, scientists, Federal agencies, and other stakeholders involved with the publication and dissemination of peer-reviewed scholarly publications resulting from federally funded scientific research? Conversely, are there policies that should not be adopted with respect to public access to peer-reviewed scholarly publications so as not to undermine any intellectual property rights of publishers, scientists, Federal agencies, and other stakeholders?

Harvard’s response refers to a need to divide and share rights between researchers and publishers. My recommendation beyond their statement is that any discussion of copyright include Creative Commons (http://creativecommons.org/), an organization dedicated to creating and defending content licenses that allow creators to reserve some, but not all, of their copyrights. Their expertise should be invaluable in any discussion about the sharing of intellectual property rights. The Harvard response includes a recommendation of a Creative Commons license at the end of their response to question 1. I also urge you to consider the expertise of SPARC (http://www.arl.org/sparc/), the Scholarly Publishing & Academic Resources Coalition.

(7) Besides scholarly journal articles, should other types of peer-reviewed publications resulting from federally funded research, such as book chapters and conference proceedings, be covered by these public access policies?

In Harvard’s response, they say they “could support mandatory public access” for non-journal works, but consider these to be “secondary issues” and are “not prepared to list all the types of content to which a federal public-access policy ought to apply.” I worry that this position is short-sighted and leaves too much room to abuse public access policies. Often the events that lead to research becoming a book chapter instead of a journal article are entirely matters of circumstance, and not a basis of quality or public importance. In fact, the entire distinction between article and chapter relates to a paper-based publishing economy, one that is increasingly irrelevant in a digital age. After all, if we were still limited to publishing on paper it is unlikely that this kind of public access policy discussion would even exist. If the spirit of these policies is to give the public access to research they have funded through federal tax dollars, there is no need to worry about “types of content” other than to say the research will consist of bytes and files traveling the internet. Furthermore, if the policy only requires “journal articles” to be published openly, what is to keep publishers from re-branding themselves as something other than a journal? By relabeling their products as books, magazines, or something entirely new, unwanted loopholes around public access are sure to emerge.

A Week for Irony and Contradictions in Education

The recent past has been wrought with irony (or at least contradictions) in all corners of education. Let's look at a list:

Read Bruce Baker's "If money doesn't matter..."  Bruce examines the argument that "we keep throwing money at education and it hasn't made a difference," and points out that (a) schools with lots of money tend to do well, and (b) people who make that argument don't mind throwing money at charter schools.

Just days after Vanderbilt University released their study finding no improvement in test scores in Nashville's merit pay system, schools around the country (including Colorado) received millions of dollars from the federal government to implement merit pay systems.

NBC's "Education Nation" summit will gather "the foremost policymakers, elected officials, thought leaders, educators, members of the business community and engaged citizens" to discuss issues in U.S. education. Unfortunately, the list of invited panelists NBC is promoting doesn't include any teachers, students, principals, or professors. The only university-affiliated participant on the list is the President of the University of Phoenix, who happens to be a major sponsor of the event.

In a post called "Does Education Pay?" the Center for College Affordability and Productivity (CCAP) criticize the College Board for "a confusion of correlation with causation." The day before, the CCAP asked, "Should We Abolish Colleges of Education?" and use this logic:

1. U.S. students "perform in a mediocre fashion" on international tests.
2. Kids need remediation and/or drop out of college because of their mediocre education.
3. Good teaching is better than mediocre teaching.
4. Most teachers studied at a college of education.
5. The teachers who didn't go to a college of education are as good or better than those who did, such as Teach for America teachers. (Sorry, CCAP, that's rarely true.)
6. Colleges of education support anti-knowledge and anti-intellectual biases and make their poor students look good by inflating grades.
7. Colleges of education don't want teachers to be rewarded for student learning because student self-esteem is more important than knowledge.
8. While there might be some good colleges of education, most of the people who really understand education are not in education schools.
9. Courses in education are less helpful for math teachers (for example) than advanced math courses. (This was not the finding by Floden and Meniketti (2005), who say it's not as simple as "more math is better.")
10. THEREFORE, we should close colleges of education, which are a "blight on true 'higher education' [that] should be discouraged at all institutions depending on taxpayer funds."

I admit, that post had no trouble working its way under my skin. If you can keep track of all the assumptions and correlation/causation confusions in their argument, you're doing better than me.

If I missed anything you want to add to this week's list, feel free to add them in the comments.

References
Floden, R. E., & Meniketti, M. (2005). Research on the effects of coursework in the arts and sciences and in the foundations of education. In M. Cochran-Smith & K.M. Zeichner (Eds.), Studying teacher education: The Report of AERA Panel on Research and Teacher Education (pp. 261-308). Mahwah, NJ: Erlbaum.