Come Study With Me

Have you ever thought about pursuing a PhD in mathematics education? It's application time here at CU-Boulder and I'd like to encourage you to apply. For me, I had no idea what to expect of a PhD program, or why I should consider one program over another. I feel fortunate to have landed a spot in a high-quality program in my now-home state, but for many of our students it's worth a move across the country to be in Boulder to work with some of the best faculty you'd find anywhere. If you apply to CU's School of Education, here's some of what you should expect.

If you get accepted you should hear from us in February and be invited to a recruiting weekend in March. I'm guessing we may want to accept 2-4 math ed students this year, but that all depends on the quality of the applicants for both mathematics education and for all the other programs. At the recruiting weekend you'll meet the faculty, tour the campus, and spend time talking to your future advisor. (More on those people later.)

If you choose to attend CU-Boulder and enter the program, you'll be required to take on graduate studies full-time. You'll take most of your first-year courses as part of a cohort, and you'll have an assistantship that will likely include doing research, teaching courses, or supervising student teachers. There are many PhD programs that allow for part-time study over a decade or more to complete a degree. We're not one of them. Instead, you get to make graduate school your full-time focus, and in return the university promises to support you with an assistantship and tuition credits for at least three years. No, you won't get rich in grad school, but for many students it allows them to pay the bills and avoid racking up debt.

There are five program areas in the School of Education: Ed Psych/Learning Sciences, Foundations and Policy, Equity and Cultural Diversity, Research and Evaluation Methodology, and Curriculum & Instruction. Mathematics education fits into C&I along with science education and literacy studies. During your first year you'll be exposed to people and ideas from all the program areas as you learn the research methods and foundations that will prepare you for your future studies.

We have three faculty members in math education. Over the course of your studies you'll be advised by one or more of them, you may have an assistantship with them, and will probably take a course from all of them. Here's a little bit about each:

David Webb (my advisor) has interests in mathematics curriculum, assessment, computer science, and Realistic Mathematics Education (RME). RME is a curriculum design philosophy originating with Dutch mathematician Hans Freudenthal, and when David came to CU from Wisconsin he brought the Freudenthal Institute US with him as its Executive Director. If you study math ed at CU-Boulder, you're also studying at the Freudenthal Institute, and RME gives us a common foundation for how we conceptualize the teaching of mathematics and the design of curriculum.

Vicki Hand's primary interests are in learning theory and equity. She combines the two in powerful ways to describe how math can be taught and learned to all students. It's tough but critical work. I was fortunate to take Vicki's "Theories of Math and Science Learning" course last spring and it was one of the best courses I've had at CU. Even better, Vicki is an absolute joy to be around.

The newest addition to our math ed faculty is Edd Taylor, who will join us this spring. Not only will Edd give us the extra elementary math ed experience and knowledge we've been looking for, but his interests seem to be a natural bridge between David and Vicki's areas of expertise. Some of Edd's recent work involves working with teachers to understand and modify their curriculum to better meet the needs of culturally diverse learners. We're all excited to have him join us and I can't wait to take a class with him next semester.

While those are the main three math ed faculty, you'll encounter other faculty with knowledge or interests in math ed. For example, Kim Bunning is one of our master teachers in our CU Teach program and is herself a product of our math ed PhD program. Bill Penuel is a learning scientist with a science background, but has several projects (including one I'm on) that reach into mathematics education. Margaret Eisenhart has for a long time studied how to interest females and minorities in STEM fields. If you have any interest in statistics and quantitative research methods, we have some of the best faculty anywhere, including Derek Briggs, Greg Camilli, Andrew Maul, and our Dean, Lorrie Shepard. So while we might not have the biggest program around, we make up for it with high quality experiences and high quality people school-wide.

Last but not least are the other math ed graduate students. Of those who aren't busy finishing their dissertations, you'll come to know:

  • Michael Matassa, a former middle school math teacher and instructional coach who teaches elementary methods and is interested in researching mathematics teaching;
  • Bill Campbell, a former elementary teacher with broad educational interests and knowledge whose research interests lie in RME and elementary math education;
  • Louisa Harris, a math-turned-math ed doctoral student interested in the challenges for women studying math in college and graduate school;
  • Ryan Grover, another math-turned-math ed doctoral student interested in RME approaches in undergraduate mathematics;
  • Ian Her Many Horses, a former computer science and math teacher who's pioneering a route in the emerging field of computer science education;
  • Fred Peck, a former high school math teacher who's interested in better understanding how students progress from informal to formal mathematical understanding; and
  • Vinnie Basil, a former science and math teacher who is interested in educational equity and integrated approaches to math and science curriculum.


So How Do I Apply?

The application process isn't horribly difficult, but you'll have to act quickly if you don't already have application materials put together. You'll need things like your GRE scores, reference letters, transcripts, and a personal statement. Although the personal statement isn't very long, it's your opportunity to show that you have the kind of writing skill that you can later apply to your dissertation. That's important! And of course, as a math ed student, we expect your GRE math score to be pretty good. Applications are due January 1, and you should follow these instructions and not be afraid to ask questions!

OpenComps: Candidate Status Unlocked. Loading Next Level...

I'm more than a week tardy in reporting this, but my oral examination went well and I've transitioned from "PhD student" to "PhD candidate." In other words, I passed my comprehensive exams. Apparently the title isn't universal (I've heard some schools progress you from "candidate" to "Candidate," changing only the capitalization), but what it means is that my focus and responsibility shifts away from coursework and onto my own research.

Normally this means I'd be taking few, if any, classes next semester and working on a prospectus. But as luck would have it, the School of Education is chock-full of great course offerings in the spring. So I'll be taking a full slate of courses: Language Issues in Education Research, Research on Teaching and Teacher Education, and Advanced Topics in Mathematics Education. Throw in our departmental seminar and the five dissertation hours we're required to carry each semester, and it looks like I'll be scheduled for 15 credit hours. Which is a lot for a doctoral student, er, candidate.

Realistically, this means my prospectus will probably wait until summer. That shouldn't be an inconvenience. It's going to take me a while to focus in on a research question anyway, and I think a combination of working on Bill Penuel's Inquiry Hub project and taking the Research on Teaching and Teacher Education class with Dan Liston and Jennie Whitcomb will give me plenty to think about. I am very interested in issues of research to practice, which means I need to look more at Paul Cobb's latest work, Cynthia Coburn's work, and keep working with Bill on Design-Based Implementation Research. I also want to learn more about how and why teachers modify their curriculum, which means getting up-to-date with the work of people like Janine Remillard and Corey Drake. The better I understand the current boundaries of work in these areas, the better I'll know what direction my work should go.

OpenComps: Written Exam Down, Oral Exam to Go

About two weeks ago I submitted my written responses to my comprehensive exam questions. I can't go into detail about the questions, but I'll summarize them this way:
  1. Here's a dichotomy from the learning sciences. Deal with it.
  2. Somebody did a quantitative study X and now wants to do Y. Before you think Y is a good idea, what do you have to know about X?
  3. How would you help math teachers learn about X given conditions Y?
I hadn't quite anticipated Question 1 so there was some background work to do before I could address certain details. Thankfully, I was pretty well prepared to structure my argument, and it was on this question that I did my best writing. While I'd had dreams of finishing a couple questions before the end of the weekend, my actual pace was slower than that. A lot slower. By the end of Friday, I'd written about a paragraph, and by the end of Saturday, I'd written about a page. Fortunately, that was the foothold I needed to have the rest of the 9-page paper finished on Sunday.

Next I answered Question 2. In some ways this was the question that worried me the most, but my studying definitely helped. Still, my writing was slow and it wasn't really until late Wednesday when I had this question finished. When you have three questions to answer in seven days, taking six days to answer the first two questions is less than ideal.

That left me to answer Question 3 in a bit of a writing sprint starting in the wee hours of Thursday morning, breaking to attend and teach class Thursday afternoon and evening, and then writing until 7am Friday morning to finish. Question 3 was my advisor's question and the one for which I was most prepared; in fact, a couple pages was largely a rehash of some of some things I'd blogged about in the past. Having that for a strong start certainly helped the rest of the paper take shape rather quickly.

It was a relief to reach the end of comps week, but I couldn't get too much rest because I had put off a number of things (okay, almost everything) during comps and in the weeks leading up to comps. Professors and fellow students are very understanding about it, which is great, but I wasn't entirely comfortable using comps as an excuse to not do much else during that time. In the past two weeks (including some of every day of my fall break), I've been catching up with the class I take, the class I teach, and the research project I'm on. I haven't been blogging and my social media activity has been pretty minimal during this time, but I'm starting to feel caught up.

The last hurdle to clear is the oral examination, scheduled for this Tuesday morning. I'm not too concerned about it, and thankfully, the message from my comps committee has been not to worry. But between now and then I will be going back over my responses, double-checking the literature I cited, and reading a few new things I uncovered during the comps process. My advisor hinted at some things he wants to talk about and I'll be sure to prepare for those things, too.

OpenComps: Final Preparations

By 9 am Friday, November 2nd, my advisor will email me my three comprehensive exam questions. I have exactly a week to answer them. He says I'm prepared, and I appreciate his confidence in me. I think I'm reasonably prepared, too, and I greatly appreciate that among my numerous anxieties, test-taking isn't one of them. Far from it, in fact. See, I'm one of those mystical kids that policymakers have in mind when they come up with laws like No Child Left Behind. I'm the one who actually likes taking tests and fools himself into thinking they're just a harmless yet useful snapshot of broad academic knowledge and skill. Give me a #2 pencil and bubbles to fill in and I'll happily work for hours.

There won't be any bubbles on my comprehensive exam, but there will be hours of work. Over the past week I've been making my final preparations, most of which are designed to make next week go as smoothly as possible. A summary:

Ready My References

I think my personal library will have most of the math and learning science books I might want, but I felt like some extra perspectives and guides concerning experimental design, casual inference, and statistics might come in handy. I know I can't expect to read any of these cover-to-cover in the course of the next week, but if nothing else the examples and explanations they contain could be valuable.

Having books around is a luxury, but for this level of work, it's even more important to have a way of keeping track of the hundreds of journal articles that I might want to use in my comps responses. I've been using Mendeley as my reference manager since the spring of 2010. Regardless of what tool you use -- Zotero, RefWorks, Endnote, Papers, etc. -- it's important during any writing period to have something that allows you to focus on writing, not scrambling for citation information and digging through the APA style book.

One of the best investments I've made as a grad student has been my diligent attention to the annotation, curation, and metadata accuracy of my Mendeley library. I was somewhat lax about it during my master's year and my first year of the PhD program, but then I spent most of two weeks of a summer going back through every PDF, every book, every syllabus, and every paper I wrote to make sure I had everything neatly cataloged. And I haven't relaxed since. Right now I have 890 references in my personal library, with others in group collections, and I can find or cite any of them in just seconds.

If there's one thing I can't let myself do is turn my comps into a massive search for new literature. I admit, I love the thrill of the hunt, and I've spent many hours digging around in Google Scholar tracking down papers that I realistically have no time to read. I need to trust that most of what I need I already have and I've already read, and keep my literature hunting to a minimum.

Minimize Distractions and Get Comfortable

For my last week before comps, I actually spent very little time studying and more time minimizing potential distractions. I've been to the grocery store, I've washed dishes and laundry, and I reformatted and reinstalled my operating system, virtual machine, and software on my computer because a few things had gotten flaky after a year of hard use. I've never liked studying right before a test anyway, as any attempt to "cram" is nullified by thoughts that always begin, "If I don't know it by now...." I passed my 100-hour studying mark a week or so ago and that will have to be good enough.

I'll probably work mostly at my desktop. If your computer had three monitors, 16 GB of RAM, university broadband peaking at nearly 90Mbps up and down, and a pair of Sennheiser HD 595s, you'd probably work at it, too. I might try working some in my office, and my kitchen table is nice for when a lot of open books are involved. I don't want to be stuck in my office chair for 18 hours a day, so I plan to do some heavy thinking while running and in a pinch I can even prop my laptop up on my exercise bike.

Sometimes I work in silence, but not very often. I don't want to get distracted by moving pictures, but there are a few movies I can play for background noise without losing focus, mostly because I've seen them so many times. I can get distracted by podcasts, so I'll try to listen to those selectively over the next week. I'll listen to a lot of music, and my tastes for a task like this tend to be towards the incredibly gifted (Tori Amos, Curtis Mayfield, Norah Jones, Sia) and music that's downtempo/trip-hop or otherwise having a likable female vocal/bass combo (Thievery Corporation, Zero 7, Garbage). Seriously, in the midst of an important exam, who wouldn't want to perform as relaxed and confidently as LouLou?:



OpenComps Get Less Open

Obviously, yet unfortunately, once I get my questions I'm pretty limited in what I can say about them. I'm not to receive outside help, solicited or unsolicited, and even after the exam is over I'm only to talk about the process in general terms. (I'm assuming that's in the event my committee members want to reuse the same or similar questions in the future.) Assuming I'm not exhausted by the process, I'll try to summarize my approach and workflow, lessons learned, and hopefully some epiphanies that come in the process of working through my questions. I'm looking forward to the week of writing and then readying myself for the oral defense, scheduled for November 27th.

OpenComps: Validity and Causal Inference

With the start of my comprehensive exams beginning in 12 days, my studying has hit the homestretch. Thankfully, my advisor has inspired some confidence by telling me that my understanding of the math education literature is solid and I won't need any more studying in that area. That's good for my studying, and something I take as a huge compliment. So now I can focus for a while on preparing myself for the exam question Derek Briggs is likely to throw my way. Typically, one of the three people on a comps committee is tasked with asking a question related to either the quantitative or qualitative research methodology we learn in our first year of our doctoral program. Derek is a top-notch quantitative researcher, and I enjoyed taking two classes from him last year: Measurement in Survey Research and Advanced Topics in Measurement. Where this gets slightly tricky is that Derek didn't actually teach either of my first-year quantitative methods courses, so there's a potential I could get surprised by something he normally teaches in those classes that I didn't see. It's a risk I was willing to take after working with Derek more recently and more closely in the two measurement courses last year.

It certainly won't be a surprise if Derek asks a question that focuses on issues of validity and causal inference. He mentioned it to me personally and put it in a study guide, so studying it now will be time well spent. I feel like I've had a tendency to read the validity literature a bit too quickly or superficially, so this is a good opportunity for me to revisit some of the papers I've looked at over the past couple of years. Here's the list I've put together for myself:

AERA/APA/NCME. (1999). Standards for educational and psychological testing. Washington, D.C.: American Educational Research Association. [Just the first chapter, "Validity."]

Angoff, W. H. (1988). Validity: An evolving concept. In H. Wainer & H. Braun (Eds.), Test validity (pp. 19–32). Mahwah, NJ: Lawrence Erlbaum Associates.

Borsboom, D., Cramer, A. O. J., Kievit, R. A., Scholten, A. Z., & Franic, S. (2009). The end of construct validity. In R. W. Lissitz (Ed.), The concept of validity: Revisions, new directions, and applications (pp. 135–170). Information Age Publishing.

Brookhart, S. M. (2003). Developing measurement theory for classroom assessment purposes and uses. Educational Measurement: Issues and Practice, 22(4), 5–12. doi:10.1111/j.1745-3992.2003.tb00139.x

Chatterji, M. (2003). Designing and using tools for educational assessment (p. 512). Boston, MA: Allyn & Bacon. [Chapter 3, "Quality of Assessment Results: Validity, Reliability, and Utility"]

Cronbach, L. J. (1988). Five perspectives on validity argument. In H. Wainer & H. I. Braun (Eds.), Test validity (pp. 3–17). Hillsdale, NJ: Lawrence Erlbaum.

Eisenhart, M. A., & Howe, K. R. (1992). Validity in educational research. In M. LeCompte, W. Milroy, & J. Priessle (Eds.), The handbook of qualitative research in education (pp. 642–680). San Diego, CA: Academic Press.

Gorin, J. S. (2007). Test design with cognition in mind. Educational Measurement: Issues and Practice, 25(4), 21–35. doi:10.1111/j.1745-3992.2006.00076.x

Haertel, E. H., & Herman, J. L. (2005). A historical perspective on validity arguments for accountability testing. In J. L. Herman & E. H. Haertel (Eds.), Uses and misuses of data for educational accountability and improvement (NSSE 104th., pp. 1–34). Malden, MA: Wiley-Blackwell.

Holland, P. W. (1986). Statistics and causal inference. Journal of the American Statistical Association, 81(396), 945–960. doi:10.2307/2289069

Kane, M. T. (1992). An argument-based approach to validity. Psychological Bulletin, 112(3), 527–535. doi:10.1037/0033-2909.112.3.527

Leighton, J. P., & Gierl, M. J. (2004). Defining and evaluating models of cognition used in educational measurement to make inferences about examinees’ thinking processes. Educational Measurement: Issues and Practice, 26(2), 3–16. doi:10.1111/j.1745-3992.2007.00090.x

Linn, R. L., & Baker, E. L. (1996). Can performance-based student assessments be psychometrically sound? Performance-based student assessment: Challenges and possibilities (pp. 84–103). Chicago, IL: The University of Chicago Press.

Messick, S. (1988). The once and future issues of validity: Assessing the meaning and consequences of measurement. In H. Wainer & H. I. Braun (Eds.), Test validity (pp. 33–45). Hillsdale, NJ: Lawrence Erlbaum.

Michell, J. (2009). Invalidity in validity. In R. W. Lissitz (Ed.), The concept of validity: Revisions, new directions, and applications (pp. 111–133). Information Age Publishing.

Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference (p. 623). Boston, MA: Houghton Mifflin. [Probably Chapters 1-3 and 11, if not more.]

Shepard, L. A. (1993). Evaluating test validity. Review of Research in Education, 19(1), 405–450.

Shepard, L. A. (1997). The centrality of test use and consequences for test validity. Educational Measurement: Issues and Practice, 16(2), 5–24. doi:10.1111/j.1745-3992.1997.tb00585.x

Zumbo, B. D. (2009). Validity as contextualized and pragmatic explanation, and its implications for validation practice. In R. W. Lissitz (Ed.), The concept of validity: Revisions, new directions, and applications (pp. 65–82). Information Age Publishing.

Thankfully, some of these papers I've read recently for my Advances in Assessment course so the amount of reading I have to do is appreciably less than it might look. In my typical fashion, I'll study these in chronological order with the hopes that I get a sense for how the field has evolved its thinking and practice regarding these ideas over the past several decades.

Although I have little other graduate school experience to compare it to, I feel like this reading list is representative of what sets a PhD apart, particularly one earned at an R1 university. It's not necessarily glamorous, and its relevance to the day-to-day teaching and learning in classrooms might not be immediately obvious. But without attending to issues like validity and causal inference, we have a much more difficult time being sure about what we know and how we're using that knowledge. Issues of validity should be at the heart of any assessment or measurement, and when they're attended to properly we greatly improve our ability to advance educational theories and practice.