How the Race (to the Top) Was Won (Part 2 of 2)

Note to self: In the future, don't go on a multi-state road trip and become otherwise distracted for more than a month between Parts 1 and 2 of a two-part blog post. Sorry, readers!

In my previous post, I set out to answer the following questions about Phase 1 of Race to the Top (RTT):

1. Where in the RTT rubric could states score the most points?
2. For the portions of the RTT rubric identified in #1, which states scored highest?
3. For the states identified in #2, what did their application propose and what were the judges' comments?

The answer to #1 was Section D of the RTT application, "Great Teachers and Leaders." If you break down that section, you'll find that the highest subsection scores belonged to Delaware, Tennessee, Georgia, South Carolina, Rhode Island, Kentucky, Louisiana, and Kansas. This post will dig into the actual applications to see what proposed reforms warranted those high scores, along with some of the comments made by the judges during the scoring.

(D)(1) – Providing High-Quality Pathways for Aspiring Teachers and Principals (21 points)

The RTT scoring rubric specifies that this criterion must be judged for both teachers and principals. High points are awarded for alternative certification routes that operate independently of institutions of higher education (IHEs) and include at least four of the following five definitional criteria: (a) programs are operated by a variety of providers, (b) candidates are admitted selectively, (c) candidates have school-based experiences and ongoing support, (d) limited coursework, and (e) certifications are the same as traditional programs (Kentucky Department of Education, 2010, p. 7; U.S. Department of Education, 2010b, p. 10).

Kentucky has a 20-year history of alternative certification programs, and in 2003 the Kentucky Legislature allocated funds for the creation and growth of such programs (Kentucky Department of Education, 2010, p. 118). Kentucky now has seven alternative programs, including specific programs for people with extensive work experience, college faculty, and veterans of the Armed Forces, as well as district-based, university-based, and institute-based options. Most of the programs are selective; the work experience route requires at least ten years of work in the area of certification and several others require Bachelor's degrees in the relevant content area (Kentucky Department of Education, 2010, p. 120). Ten percent of Kentucky's current teachers and 17 percent of new teachers in 2009-2010 completed an alternative program.

There were only two aspects of this portion of Kentucky's application that received criticism from the reviewers, and only two of the five reviewers deducted any points for this section. The first deficiency was in the area of alternative principal licensing. Kentucky only has one program for alternative principal certification, and it does not meet all five of the definitional criteria (U.S. Department of Education, 2010c, p. 32). Only one new principal was alternatively licensed in 2009-2010, and that individual went through a university-based alternative program (Kentucky Department of Education, 2010, p. 122). The other deficiency was in Kentucky's process for identifying and acting upon teacher and principal shortages. Shortages are identified by Kentucky's LEAs, but current efforts to place teachers are limited both in number and geographic reach (U.S. Department of Education, 2010c, p. 32).

My take: Kentucky got perfect scores from 3 of 5 judges, and the top score overall in this area, despite only having one alternative program for administrators that produced only one new administrator the previous year. This was the best proposal in the country.

(D)(2) – Improving teacher and principal effectiveness based on performance (58 total points across four subsections)

This criterion, also applicable to both teachers and principals, is worth a maximum of 58 points. Rhode Island scored 94 percent of those points to lead all applicants. Unlike the previous criterion, this one is divided into four distinct subsections. Rhode Island had the top score or tied for top score in three of those subsections.

(D)(2)(i) - Measuring Student Growth (5 points)

Even though this subsection is only worth five points, Tennessee was the only state to be awarded a perfect score. To earn the points, states must "establish clear approaches to measuring student growth and measure it for each individual student" (U.S. Department of Education, 2010b, p. 11).

Tennessee has been using the Tennessee Value-Added Assessment System (TVAAS) since 1992. Tennessee claimed to track every grade 3-12 student in every subject, and the state claims the TVAAS is the largest student database in history (Tennessee Department of Education, 2010, p. 82). Despite the "every subject" claim, a brief look at the publicly-viewable data on the TVAAS website (https://tvaas.sas.com/evaas/welcome.jsp) indicates only core subjects are tested, and not at every grade. Tennessee has used the TVAAS to determine Adequate Yearly Progress, to support progressive districts, identify strengths and weaknesses in grades or subjects, and inform instruction, but prior to the 2009-2010 school year only 14 percent of the state's teachers had access to the database (Tennessee Department of Education, 2010, p. 82). Now that Tennessee has opened up access to all teachers, they plan to train both current teachers and administrators, as well as pre-service teachers, in the use of the database. Not only will this value-added data be linked to teacher and principal compensation and evaluations, the state "will monitor and report access and usage of the system at the teacher, school, and district levels" (Tennessee Department of Education, 2010, p. 82). No explanation is given in Tennessee's application for this level of monitoring and reporting, but one might assume it is to apply pressure to ensure that the system is universally used.

None of the five reviewers of Tennessee's application made any mention of the criticisms that have been levied against the TVAAS, even though such research is easily found (Amrein-Beardsley, 2008; Kupermintz, 2003). The statistical model employed in the TVAAS, the Education Value-Added Assessment System (EVAAS), may currently be the best value-added model available, but experts have had difficulty resolving its flaws because neither the statistical algorithms nor full value-added data sets have been disclosed for peer review. As one researcher stated, "My own and others' attempts to access the EVAAS value-added data have consistently gone without response or been refused with the justification that the value-added data, if released to external researchers, might be misrepresented" (Amrein-Beardsley, 2008, p. 68).

My take: Tennessee got a perfect score despite not collecting data for all students, at all grade levels, for all subjects, and they use a system that researchers aren't allowed to inspect because the state doesn't think they'll understand it. This was the best proposal in the country.

(D)(2)(ii) – Developing Evaluation Systems (15 points)

This subsection encourages the development of an evaluation system that "differentiate[s] effectiveness using multiple rating categories that take into account data on student growth ... as a significant factor" (U.S. Department of Education, 2010b, p. 11). The evaluation system should be "rigorous, transparent, and fair" and be "designed and developed with teacher and principal involvement" (U.S. Department of Education, 2010b, p. 11).

Rhode Island was the top-scoring state in this subcategory, but unlike Tennessee, Rhode Island does not have a value-added assessment system currently in place. Instead, it is rushing to implement one by the 2011-2012 school year so it can be "fully operational" by 2013-2014 (Rhode Island Department of Education, 2010, p. 95). Rhode Island plans to use this system liberally in educator evaluations:

Every decision made in regard to the professional educators in Rhode Island, whether made by an LEA or the state, will be based on evidence of the respective teacher's or principal's impact on student growth and academic achievement in addition to other measures of content knowledge, instructional quality, and professional responsibility. These new RI Standards ensure that no child in Rhode Island will be taught by a teacher who has received an "ineffective" evaluation for two consecutive years. (Rhode Island Department of Education, 2010, p. 97)

Instead of mandating a single statewide evaluation system, Rhode Island will allow individual LEAs to develop their own, provided they comply with the rigorous standards specified by the state. If LEAs choose to develop their own systems (or fail to), they can/must adopt a state-provided evaluation system.

Reviewers of Rhode Island's application awarded them 96 percent of the possible points for this subsection. In response to Rhode Island's "no child will be taught by an ineffective teacher" clause, one reviewer commented, "This is bold, it shows the seriousness of effort and it is an incredibly important foundation for RTT plans to get traction" (U.S. Department of Education, 2010d, p. 5). Only one reviewer seriously questioned Rhode Island's aggressive timeline for implementing their evaluation system. Even though the state forecasts a "fully operational" value-added system by 2013-2014, value-added data will account for 40 percent of a teacher's evaluation starting in 2011-2012 before rising to 45 percent in 2012-2013 and 51 percent in 2013 (Rhode Island Department of Education, 2010, p. 98; U.S. Department of Education, 2010d, p. 44).

My take: Rhode Island outscored every other state by mandating that districts transparently and fairly evaluate teachers based on data that didn't exist yet and a growth model the state didn't yet have.

(D)(2)(iii) – Conducting Annual Evaluations (10 points)

Two states, Tennessee and Rhode Island, scored the maximum ten points on this subsection. To earn the maximum points, the RTT scoring rubric requires that states have policies requiring "annual evaluations of teachers and principals that include timely and constructive feedback" (U.S. Department of Education, 2010b, p. 11) and that those evaluations include student growth data.

Tennessee gained favor in the scoring by having recently passed their "First to the Top Act," which establishes a 15-member Teacher Evaluation Advisory Committee tasked with developing a new evaluation system. All participating Tennessee LEAs will use the new evaluation system as described:

The evaluation system may be used to publicly report data that includes, but is not limited to, differentiation of teacher and principal performance (percentage in each rating category), the LEA's ability to increase the percentage of effective teachers and principals, and percentage of compensation based on instructional effectiveness. To ensure accountability on improving performance of teachers and principals, the state will encourage LEAs to set annual improvement goals, with a minimum of 15% improvement in terms of the number of educators moving up in each rating category. (Tennessee Department of Education, 2010, p. 86)

Much appears to be hinging on the application's use of "may be used" and "the state will encourage." One reviewer, despite awarding a perfect score, advised that "It would make sense to pilot some of these ideas in several districts and make any needed adjustments before adopting them statewide in July, 2011" (U.S. Department of Education, 2010e, p. 4). Meanwhile, another reviewer questioned, "With such heavy weighting on student achievement data, it is not clear what solutions the State has to evaluate teachers in non-tested subjects or grades" and "It is not clear if this new evaluation system will need to be collectively bargained, and if so, how the State intends to secure teacher buy-in" (U.S. Department of Education, 2010e, p. 12). None of the reviewers explicitly questioned the ability to expect a minimum, annual 15 percent improvement of the number of teachers moving up the evaluation rating categories. Only time will tell if this is a sustainable goal.

Compared to Tennessee, Rhode Island's annual evaluation proposal looks decidedly unremarkable and received few comments from reviewers. Rhode Island called for annual evaluations at a minimum, and the state is responsible for providing teachers and principals the academic growth data that constitutes the bulk of their evaluation. The evaluations must also be based on the “quality of instruction (or, for principals, quality of instructional leadership and management), demonstration of professional responsibilities, and content knowledge” (Rhode Island Department of Education, 2010, pp. 101-102). LEAs are expected to review evaluations to guide their professional development programs.

My take: Tennessee planned to evaluate everyone but only had a system designed to measure teachers in just a few subjects. How they would negotiate an expansion of their system wasn't clear. How they expected endless annual 15% improvements wasn't clear. Still, this and Rhode Island's rather bland proposal were the best in the country.

(D)(2)(iv) – Using Evaluations to Inform Key Decisions (28 points)

By far the largest subsection of criteria (D)(2), constituting nearly half its possible points, this subsection is targeted towards using evaluations to inform "key decisions." The RTT rubric specifies four such "key decisions:"

(a) Developing teachers and principals, including by providing relevant coaching, induction support, and/or professional development;
(b) Compensating, promoting, and retaining teachers and principals, including by providing opportunities for highly effective teachers and principals ... to obtain additional compensation and be given additional responsibilities;
(c) [Granting] tenure and/or full certification (where applicable) to teachers and principals using rigorous standards and streamlined, transparent, and fair procedures; and
(d) Removing ineffective tenured and untenured teachers and principals after they have had ample opportunities to improve, and ensuring that such decisions are made using rigorous standards and streamlined, transparent, and fair procedures. (U.S. Department of Education, 2010b, p. 11)

South Carolina and Rhode Island tied for the top score on this subsection, each earning 93 percent of the possible points. South Carolina currently uses two data systems: the system for Assisting, Developing, and Evaluating Professional Teaching (ADEPT) and the Program for Assisting, Developing, and Evaluating Principal Performance (PADEPP). (Acronym-loving South Carolina's RTT application is named INSPIRE, short for “Innovation, Next Generation Learners, Standards & Assessments, Personalized Instruction, Input and Choice, Redesigned Schools, Effective Teachers & Leaders, and Data Systems.”) South Carolina plans to tie these systems into their state-controlled certification system (which determines contract and due process rights) and statewide salary schedule (U.S. Department of Education, 2010f, p. 29). With the state handling certifications, tenure, and salaries, it will be much easier for South Carolina to implement the reforms specified in the RTT scoring rubric.

One reviewer only awarded 18 of 28 points and had particularly critical comments for this part of South Carolina's proposal:

The state proposes to provide induction support for beginning teachers and principals. There is no mention of coaching services after the induction period. The state application explains various statutory issues related to tenure and insists that tenure will be related to performance. The explanation is inadequate and does not lay out a clear plan. (U.S. Department of Education, 2010f, p. 4)

A different reviewer gave South Carolina the maximum 28 points for this subsection, saying only that "all beginning teachers and principals [will] receive induction support and mentoring" and "Salary incentives are part of South Carolina's plan, teacher effectiveness, retention, full certification, and removal, if necessary" (U.S. Department of Education, 2010f, p. 20). This kind of variability is a problem with the design of the RTT rubric and will be discussed in the conclusion of this post.

South Carolina might have the top-scoring proposal for using evaluations to inform decision-making, but their assessment and data systems have some glaring overall problems. The statewide data systems, ADEPT and PADEPP, do not use a value-added model. Some LEAs are piloting a value-added "approach," and the state plans on developing or selecting a statewide model in the near future. The data used in that eventual model will first be from their current statewide assessment and the Measures of Academic Progress (MAP), but the state plans to abandon those assessments in favor of one aligned with the Common Core K-12 standards, whenever one becomes available (South Carolina Department of Education, 2010, p. 102).

Rhode Island equaled South Carolina's score, but did so while retaining a more traditional measure of local control. In most cases, the LEAs will be setting their policies to meet the proposed goals of Rhode Island's RTT application, and the State Department of Education will assume an enforcement role. For the compensation piece, Rhode Island proposes funding four pilot programs with RTT dollars. By 2015, LEAs will be able to choose one of the four compensation models or develop their own with the state providing guidance and support (Rhode Island Department of Education, 2010, p. 106).

As discussed previously, Rhode Island plans to use their evaluation system for promotion, retention, and certification of teachers. LEAs will have to prove to the state that they are using evaluation data in these decisions and report to the state those teachers who have earned promotions or leadership responsibilities, which will require at least an "effective" or "highly effective" rating on their annual evaluation (Rhode Island Department of Education, 2010, p. 107). LEAs will also have to certify that they have removed all non-tenured ineffective teachers and any teacher marked "ineffective" two years in a row (Rhode Island Department of Education, 2010, pp. 108-109). The state will continue to manage the certification system and current educators will be subject to the new rules as their current certificates renew.

My take: Despite the high point value of this subsection, the U.S. DoE seems unclear if they believe more strongly in local control or state control, in current tests or future tests, or in mentoring or induction support. These were the best proposals in the country.

(D)(3) – Ensuring equitable distribution of effective teachers and principals (25 points across two subsections)

Louisiana led all states by taking 90 percent of the maximum 25 points for this section, but did not have the high score in either of the two subsections. Instead of reviewing Louisiana's application, we will instead focus on the applications from Georgia and Kansas.

(D)(3)(i) – Ensuring equitable distribution in high-poverty or high-minority schools (15 points)

The RTT scoring rubric for this subsection requires policies that "ensure that students in high-poverty and/or high-minority schools ... have equitable access to highly effective teachers and principals ... and are not served by ineffective teachers and principals at higher rates than other students" (U.S. Department of Education, 2010b, p. 11). Georgia earned 93 percent of the 15 available points in this subsection to lead all states. Georgia's strategy clearly delineates into solving problems of supply and demand. On the demand side, Georgia plans to award bonuses to effective teachers and principals in high-need schools "tied to the degree of reduction made in the student achievement gap every year" (Georgia Department of Education, 2010, p. 121). To entice effective teachers to move to high-need rural areas, the state is proposing $50,000 tax-free bonuses that vest over three years and require the teacher to maintain a high rating on the state's Teacher Effectiveness Measure (TEM). Districts wanting to participate in this program must compete for the funds and prove that the teachers eligible for bonuses have an established record of high achievement. Georgia is being bold with this plan, despite their decision not to "[offer] these kinds of bonuses to principals, having experimented with significant bonuses for principals in the past and having found that these incentives were not effective in getting principals to relocate" (Georgia Department of Education, 2010, p. 121). To improve the supply side of equitable teacher distribution, Georgia will work with LEAs to improve professional development and partner with organizations like Teach for America and The New Teacher Project that have experience recruiting teachers for hard-to-fill positions.

Only one reviewer offered the most glaring criticism of Georgia's plan: "There is also detail missing ... on the systems to ensure distribution over time" (U.S. Department of Education, 2010g, p. 40). The RTT money allocated for the bonuses is temporary, and programs like Teach for America and The New Teacher Project are not well-known for placing teachers who remain in high-need areas for more than a few years.

My take-away: Georgia actually had a straightforward approach here -- fill difficult assignments by offering significantly more money to teachers who have shown an ability to raise scores and close achievement gaps. Will it work? No one's sure, but this proposal should be worth following up on. After all, it was the best proposal in the country.

(D)(3)(ii) – Ensuring equitable distribution in hard-to-staff subjects and specialty areas (10 points)

Kansas, whose application ranked 29th overall, makes a surprise appearance at the top of the scoreboard. They introduce this section of their application with some startling statistics:

The Teaching in Kansas Commission found that:

• 42% of Kansas teachers leave the field after seven years,
• 36% of Kansas teachers can retire within the next 5 years,
• 25% fewer students entered the teaching profession over the past six years,
• An 86% decrease in Kansas teacher biology licenses will occur within 6 years,
• A 50% decrease in chemistry licenses will occur within 6 years, and
• A 67% decrease in physics licenses will occur within 6 years. (Kansas Department of Education, 2010, p. 81)

Kansas's plan mostly consists of expanding their UKanTeach program at the University of Kansas, both at KU and to other institutions of higher education around the state. Kansas claims that “UKanTeach is dramatically increasing the number of math and science teachers graduating from KU, resulting in over 100 new, highly qualified science and math teachers each year” (Kansas Department of Education, 2010, p. 81). They claim this “dramatic increase” without citing the number of graduates before the UKanTeach program and fail to address non-STEM hard-to-staff subjects such as special education and language instruction. Neither of these criticisms were mentioned by any of the five reviewers of Kansas's application. Additionally, despite having other plans for teacher preparation and retention in hard-to-serve areas, the reviewers almost universally fail to cite them in their comments (U.S. Department of Education, 2010h).

My take: Kansas's proposal sounds practical but lacks details. Can UKanTeach do anything for non-STEM teachers? Why did the judges find this to be the best proposal in the country without an answer to that question?

(D)(4) – Improving the effectiveness of teacher and principal preparation programs (14 points)

This criterion asks for a quality plan for linking student achievement and growth to the in-state teacher and principal preparation programs and expanding those programs identified as successful.

Tennessee, which earned 90 percent of the possible points, uses brief but strong language to sell this part of their application. They proudly boast "The cornerstones are competition and accountability," and "Our State Board of Education (SBE) has broken the monopoly on teacher preparation held by institutions of higher education" (Tennessee Department of Education, 2010, p. 110). Tennessee claims to publicly report their teacher preparation program quality data, but a search of their Department of Education website (http://tn.gov/education/) when the RTT results were announced revealed nothing. Tennessee planned in 2010 to gather stakeholders from across the state to examine how they link student achievement data to teacher preparation programs and develop a plan to "reward programs that are successful and support or decertify those that fail to produce effective teachers" (Tennessee Department of Education, 2010, p. 111). Most of the reviewers of Tennessee's application cited a lack of focus on principal preparation programs to match those for teachers (U.S. Department of Education, 2010e).

Rhode Island doesn't use Tennessee's tough language, but claims to "[act] aggressively to close programs that do not meet its rigorous current standards and has closed two programs, including a principal preparation program, in the last 5 years" (Rhode Island Department of Education, 2010, p. 125). Every educator preparation program in the state must be re-approved every five years and Rhode Island plans to include data from teacher and principal evaluations in the re-approval process. Specifically, Rhode Island wishes to track how many educators from each preparation program earn full Professional Certification and a disaggregation of preparation program graduates in high vs. low poverty and minority schools (Rhode Island Department of Education, 2010, p. 125).

My take: It's troubling to see Rhode Island acknowledge their closing of two teacher/principal preparation programs, and more troubling to see the judges view that as a positive achievement, without knowing in detail the specific failures of those institutions that led to the failure. How were the programs not meeting Rhode Island's "rigorous standards" and what efforts had been made to improve them? It would have been far more impressive for our country's best proposals to describe a successful rebuilding of those programs than their simple termination.

(D)(5) – Providing effective support to teachers and principals (20 points)

This criterion is based on two goals: provide ongoing, targeted professional development and supports while also monitoring and improving the quality of the professional development and supports. The supports could include “coaching, induction, and common planning and collaboration time to teachers and principals” (U.S. Department of Education, 2010b, p. 12).

Delaware earned 95% of the available points by requiring all participating LEAs to adopt a comprehensive professional development plan that contains all the supports specified in the rubric. Despite being the top-scoring plan, one reviewer commented:

The key weakness of this plan is the lack of specificity about how LEAs will know what is a good PD model and what is not – this section seems vague and not well thought through. Compared to other plans in the Delaware application, this area is not very creative nor clear. (U.S. Department of Education, 2010i, p. 15)

Delaware does specify plans for certifying effective professional development programs and requiring states to adopt such high-quality programs by the 2010-2011 school year, but the eleven pages of description in the Delaware application didn't translate into rich commentary from the reviewers, despite the high scores.

My take: It's as if the reviewers are confident in Delaware's plan despite not being able to accurately describe what the plan contains. Somehow, this was still better than the proposals from all other states.

Discussion

Taken all together, we see a policy preference for: (a) many alternative routes to certification, (b) an extensive value-added assessment system, (c) teacher and principal evaluations based on student performance and growth data, (d) annual evaluations of all teachers and principals, (e) teacher and principal compensation, promotion, and retention policies tied to evaluations, (f) incentives for teachers and principals to serve in high-need areas, (g) programs to increase the supply of teachers for hard-to-fill subjects, (h) quality, accountable teacher preparation programs, and (g) effective professional development.

This should be no surprise, because this is precisely what the RTT rubric asked for. How did this encourage a large pool of innovative and creative reforms? Is Kentucky's 20-year-old alternative licensure program creative? Is Tennessee's value-added assessment system, in use since 1992, innovative? It's very possible the RTT rubric has stifled creativity and innovation as much as it encouraged it. Even worse, states may have abandoned the innovative ideas they developed in Phase 1 and instead chose to copy the above high-scoring states in the hopes of winning funding.

A very troubling aspect of many proposed policies is the dependence of so many important decisions on a value-added student performance model that is not 100 percent transparent. Regardless of opinions concerning the use of value-added models, or beliefs that value-added models could achieve perfect accuracy and reliability, the use of a non-transparent model (such as the EVAAS) in so-called transparent evaluation systems is a significant flaw. Software is patentable and profitable, while the underlying mathematics is not, so the motivations for keeping at least some parts of these growth models secret is understandable, even if undesirable. Still, the RTT process could have been strengthened significantly if the scoring rubric had required 100 percent transparency for any and all statistical operations provided on educational data.

My final criticism of this process lie in the RTT rubric itself. Why have 500 total points? Why is "providing high-quality pathways for aspiring teachers and principals" worth 21 points and "ensuring equitable distribution of teachers" worth 25? Who decided that one category should be worth four points more than the other and why? If those four points had been allocated elsewhere, would the results have changed?

In a paper by Peterson and Rothstein (2010), the authors expose the arbitrary nature in which points were allocated in the RTT rubric and show how changes in the weights of categories could have changed the outcome of the entire RTT competition. For example, adding a mere 15 points to any of the four criteria (improving student outcomes, using data to improve instruction, using evaluations to inform key decisions, and ensuring equitable distribution), then decreasing the other criteria less than a half-point to keep the rubric's total score at 500, Georgia would have won the RTT competition (Peterson & Rothstein, 2010, p. 4). Similarly, the "demonstrating other significant reforms" criterion was only allocated one percent (5 points) of the total rubric. Given the innovation possible in this "other" category, including reforms called for in the DoE Blueprint and other federal education programs, it would have been reasonable to justify giving that category a larger weight. If that weight had been 25 percent of the application, then Pennsylvania would have been the winner (Peterson & Rothstein, 2010, p. 5).

This design of the RTT rubric and its point allocation not only affected the outcome of Phase 1, but likely affected the following phases even more strongly. The elements of the proposals examined in this paper were chosen regardless of the margin of victory. Not only are slim margins statistically insignificant in a 500-point rubric, but the scoring process itself leads to some arbitrary selections. Unfortunately, when trying to play catch-up with the winners, the simplest thing to do is copy, not create. In doing so, RTT reinforces a "don't just stand there, do something" atmosphere for reform, even if the choice and effectiveness of those "somethings" is uncertain and arbitrary.

References

Amrein-Beardsley, A. (2008). Methodological Concerns About the Education Value-Added Assessment System. Educational Researcher, 37(2), 65-75. doi:10.3102/0013189X08316420

Georgia Department of Education. (2010, January 19). Race to the Top: Application for Initial Funding. Retrieved from http://www2.ed.gov/programs/racetothetop/phase1-
applications/georgia.pdf

Kansas Department of Education. (2010, January 14). Race to the Top: Application for Initial Funding. Retrieved from http://www2.ed.gov/programs/racetothetop/phase1-
applications/kansas.pdf

Kentucky Department of Education. (2010, January 14). Race to the Top: Application for Initial Funding. Retrieved from http://www2.ed.gov/programs/racetothetop/phase1-applications/kentucky.pdf

Kupermintz, H. (2003). Teacher Effects and Teacher Effectiveness: A Validity Investigation of the Tennessee Value Added Assessment System. Educational Evaluation and Policy Analysis, 25(3), 287-298. doi:10.3102/01623737025003287

Peterson, W., & Rothstein, R. (2010). Let's do the Numbers: Department of Education's "Race to the Top" Program Offers Only a Muddled Path to the Finish Line (Briefing Paper No. 263). EPI Briefing Papers. Washington, D.C.: Economic Policy Institute. Retrieved from http://www.epi.org/page/-/BriefingPaper263.pdf

Rhode Island Department of Education. (2010, January 14). Race to the Top: Application for Initial Funding. Retrieved from http://www2.ed.gov/programs/racetothetop/phase1-
applications/rhode-island.pdf

Tennessee Department of Education. (2010, January 18). Race to the Top: Application for Initial Funding. Retrieved from http://www2.ed.gov/programs/racetothetop/phase1-applications/tennessee.pdf

U.S. Department of Education. (2010b). Race to the Top Scoring Rubric Corrected. Washington, D.C.: U.S. Department of Education. Retrieved from http://www2.ed.gov/programs/racetothetop/scoringrubric.pdf

U.S. Department of Education. (2010c). Race to the Top: Technical Review Form - Kentucky. Retrieved from http://www2.ed.gov/programs/racetothetop/phase1-applications/comments/kentucky.pdf

U.S. Department of Education. (2010d). Race to the Top: Technical Review Form - Rhode Island. Retrieved from http://www2.ed.gov/programs/racetothetop/phase1-
applications/comments/rhode-island.pdf

U.S. Department of Education. (2010e). Race to the Top: Technical Review Form - Tennessee. Retrieved from http://www2.ed.gov/programs/racetothetop/phase1-applications/comments/tennessee.pdf

U.S. Department of Education. (2010f). Race to the Top: Technical Review Form - South
Carolina. Retrieved from http://www2.ed.gov/programs/racetothetop/phase1-applications/comments/south-carolina.pdf

U.S. Department of Education. (2010g). Race to the Top: Technical Review Form - Georgia. Retrieved from http://www2.ed.gov/programs/racetothetop/phase1-applications/comments/georgia.pdf

U.S. Department of Education. (2010h). Race to the Top: Technical Review Form - Kansas. Retrieved from http://www2.ed.gov/programs/racetothetop/phase1-applications/comments/kansas.pdf

U.S. Department of Education. (2010i). Race to the Top: Technical Review Form - Deleware. Retrieved from http://www2.ed.gov/programs/racetothetop/phase1-applications/comments/delaware.pdf

How the Race (to the Top) Was Won (Part 1 of 2)

Race to the Top (RTT), the foremost education policy instrument under the Obama administration, was introduced in 2009 as part of the American Recovery and Reinvestment Act. The first two winning states, Delaware and Tennessee, were announced in the spring of 2010. At that time I wrote a paper about RTT for a policy class but didn't blog about it. Since RTT has continued with more rounds of state competition and a new RTT program for school districts, I hope there's still some relevance in sharing some of what I learned about RTT and what states did to score well on their applications.

Using policy definitions developed by McDonnell and Elmore (1987), RTT is a near-perfect example of an inducement, where money is exchanged for action. Inducements are a simple model:

RTT adds a layer of policymaking to this simple process. In the model below, the state departments of education have all policy instruments at their disposal. The inducement exists between the federal and state policymaking bodies, and not necessarily between the state and the local education authority (LEA). (The new RTT for districts will obviously change this arrangement.) Regardless of the instrument(s) used by the states, the goal, as defined by the U.S. DoE, is for states to "[lead] the way with ambitious yet achievable plans for implementing coherent, compelling, and comprehensive education reform" (U.S. Department of Education, 2010d). Additionally, the DoE clearly states that "Creativity and innovation are rewarded in this competition" (U.S. Department of Education, 2010c. p. 15).

In the ideal RTT scenario, the competition would look like this:

1. The U.S. DoE rewards the states with the most promising reforms.
2. Winning states would enact and enforce new education policies.
3. The effects of the new policies would be measured.
4. Policies that prove to be successful would be replicated by other states.

There are two significant problems with emphasizing creativity and innovation in RTT. First, creativity and innovation necessitates deviation from proven reforms. You can't be creative by saying, "We're going to do what we know works." Delaware and Tennessee's "innovative" reforms (a label worth questioning) may have helped them win Phase 1 of RTT, but it may some time before we know if the reforms perform as intended. Implicit in the RTT competition is an assumption that established, effective reforms are too few, too expensive, or too difficult to scale, so RTT challenges states to create new reforms that might be cheaper or more easily implemented. This creates a condition where RTT money gets awarded for the potential of a policy, and not its past, proven effectiveness.

The second problem with emphasizing creativity and innovation is that RTT was not structured as an brainstorming, anything goes kind of policymaking process. RTT comes with a detailed scoring rubric, and any state wishing to score well obviously wrote policies to satisfy the rubric. How does that encourage creativity or innovation? In addition, states or districts applying beyond the first round will likely replicate the highest-scoring applications from Phase 1. This means that policies developed as part of RTT Phase 2 and later are likely to be less diverse, less creative, and less innovative, but no more proven.

So how did states manage this balance of creativity versus scoring high on the rubric? Thankfully, the rubric, the applications, and the judges' scorecards are all publicly available, so we can see exactly what each state proposed in their application. The analysis in this post will answer the following questions:

1. Where in the RTT rubric could states score the most points?
2. For the portions of the RTT rubric identified in #1, which states scored highest?
3. For the states identified in #2, what did their application propose and what were the judges' comments?

Race to the Top Scoring

Here is a summary of RTT Phase 1 scoring:

Selection Criteria	Points Possible	Percent of Total	Average Score (Points)	Average Score (Percent)	Standard Deviation (Points)
A. State Success Factors	125	25	90	72	18.03
B. Standards and Assessments	70	14	61	88	9.63
C. Data Systems to Support Instruction	47	9	33	70	7.04
C. Data Systems to Support Instruction	47	9	33	70	7.04
D. Great Teachers and Leaders	138	28	91	66	21.5
E. Turning Around the Lowest-Achieving Schools	50	10	36	72	11.19
F. General	55	11	37	68	12.96
Competitive Preference Priority 2: Emphasis on STEM	15	3	11	73	6.73
TOTAL	500	100	359	72	67.22

(Source: U.S. Department of Education, 2010a, 2010b)

Looking at how the points on the rubric are allocated, it's clear that for any state to do well they'd need to score well on Section D, "Great Teachers and Leaders." It was worth 28 percent of the 500 total points, and we now can see that of all the sections, the fewest points (as a percent) were awarded in this section. That means there was a lot of potential upside here, and we can dig into the details of "Great Teachers and Leaders" to see which states scored best. In the table below you'll find all the subsections of Section D, along with the scores of the eight states (DE, TN, GA, SC, RI, KY, LA, and KS) who had the top score (in bold) in at least one of those subsections.

Selection Criteria	DE	TN	GA	SC	RI	KY	LA	KS
Overall RttT Phase 1 Rank	1	2	3	6	8	9	11	29
(D) "Great Teachers and Leaders" Score	86%	83%	81%	82%	88%	80%	89%	63%
(1) Providing high-quality pathways for aspiring teachers and principals	82%	71%	70%	74%	84%	94%	89%	32%
(2) Improving teacher and principal effectiveness based on performance	87%	91%	86%	91%	94%	76%	90%	56%
(2)(i) Measuring student growth	88%	100%	48%	88%	80%	84%	96%	56%
(2)(ii) Developing evaluation systems	85%	91%	81%	89%	96%	83%	91%	68%
(2)(iii) Conducting annual evaluations	92%	100%	96%	90%	100%	58%	92%	68%
(2)(iv) Using evaluations to inform key decisions	86%	87%	91%	93%	93%	76%	89%	45%
(3) Ensuring equitable distribution of effective teachers and principals	85%	74%	87%	73%	79%	73%	90%	89%
(3)(i) Ensuring equitable distribution in high-poverty or high-minority schools	83%	68%	93%	68%	92%	72%	88%	85%
(3)(ii) Ensuring equitable distribution in hard-to-staff subjects and specialty areas	88%	82%	78%	80%	60%	74%	92%	94%
(4) Improving the effectiveness of teacher and principal preparation programs	81%	90%	73%	81%	90%	77%	86%	66%
(5) Providing effective suport to teachers and principals	95%	75%	75%	79%	84%	92%	84%	80%

(Source: U.S. Department of Education, 2010a)

Just by reading the titles of the subsections above, you can recognize some of the most contentious areas of education policy we've seen the past several years. So not only is this part of the RTT application about high points, it's high-stakes. If the greatest opportunity for improvement in education truly lies in this area, it will be critical to get these policies right. In Part 2 of this post, we'll look at each subsection and the application from the state with the highest score in that area. Some of the policies are sound, but some aren't, and sometimes the judges' comments indicate divergent interpretations of both the application and the rubric.

References

McDonnell, L. M., & Elmore, R. F. (1987). Getting the job done: Alternative policy instruments. Educational Evaluation and Policy Analysis, 9(2), 133-152. Retrieved from http://www.jstor.org/stable/1163726

U.S. Department of Education. (2010a). Detail chart of the Phase 1 scores for each State. Retrieved from http://www2.ed.gov/programs/racetothetop/phase1-applications/phase1-scores-detail.xls

U.S. Department of Education. (2010b). Race to the Top Scoring Rubric Corrected. Washington, D.C.: U.S. Department of Education. Retrieved from http://www2.ed.gov/programs/racetothetop/scoringrubric.pdf

U.S. Department of Education. (2010c, May 27). Race to the Top Program: Guidance and Frequently Asked Questions. Retrieved from http://www2.ed.gov/programs/racetothetop/faq.pdf

U.S. Department of Education. (2010d, April 16). Race to the Top Fund. ED.gov. Retrieved June 6, 2012, from http://www2.ed.gov/programs/racetothetop/index.html

Open Access Publishing in Mathematics Education

As I write this, the White House petition to require free access over the internet to scientific journal articles arising from taxpayer-funded research is within 150 signatures of the 25,000 needed to guarantee a response from the White House. If you're unfamiliar with the petition, this video concisely explains the issue:



Most of the advances in open access publishing seem to be in the natural and medical sciences -- mathematics, biology, medicine, etc. Much is this is due to a policy by the National Institutes of Health (NIH) that requires publications from NIH-funded research be made available to the public, and the hope of the petition is that a similar policy would spread to other government funding agencies. Given that a significant amount of education research is funded by the National Science Foundation (NSF), education researchers are going to have to think about their open access publishing options should policies require open access to publicly-funded research.

I'm a teacher, not a researcher. Should I care about open access to research?

Yes! Few things annoy me more than the assumption that teachers should not read or take any interest in published education research. I strongly believe that the more researchers think about teachers as part of the audience for their research, the more relevant that research is likely to be and the more quickly we can implement the results. So if you're a teacher and you come across a research article, pay attention. If it makes no sense to you or seems totally irrelevant to you as a teacher, there's probably something wrong. You would be doing a great service to bring that article and your problems with it to the attention of the research community, and researchers should welcome your input. Right now a lot of high-quality research hides from you in for-profit, closed journals, which I believe has allowed the goals of teachers and researchers to drift apart. With open access journals and greater communication via social media, I hope the divide between teachers and researchers can come together with greater frequency.

Current Top-Tier Mathematics Education Research Journals

In mathematics education, the following four journals are often seen as the most prestigious. Let's look at their current publishing policies:

Journal of Research in Mathematics Education - JRME is NCTM's research journal and  probably the top journal in the field. Unfortunately, they have a very author-unfriendly publishing policy:

Assignment of copyright for the article to the National Council of Teachers of Mathematics is required as a condition of publication. After acceptance by JRME, a manuscript may not be published elsewhere, including on the internet, without written permission from NCTM. Each author of a paper published in JRME will receive five complimentary copies of the issue in which the paper appears.

Wow, five complimentary copies? With those I can freely distribute my work to 5 people, or approximately 0.0000002% of worldwide internet users, all of whom can read this measly blog page.

Educational Studies in Mathematics - This journal was founded by Hans Freudenthal in 1968 and is currently published by Springer. Although Springer is an enormous publishing company with a vested interest in a traditional publishing model, they are making efforts to find ways to increase access while still making a profit. Authors have a choice: (a) Transfer their copyright to Springer or (b) opt into Springer's "Open Choice" program, which makes articles freely available on SpringerLink and allows the author to retain copyright and publish under a Creative Commons Attribution License. The catch? Springer charges the author a $3000 fee.

International Journal of Science and Mathematics Education - This journal is also published by Springer and has the same "Open Choice" option as ESM.

Mathematical Thinking and Learning - This journal is published by Routledge, part of the Taylor & Francis Group. Their copyright agreement (PDF) includes the classic language about why authors should transfer copyright (and when I say "classic," I mean old, as in pre-internet): "The transfer of copyright from author to publisher must be clearly stated in writing to enable the publisher to assure maximum dissemination of the author's work." The thought that putting written work in an expensive journal distributed to a relatively small number of people and institutions "assures" a wider distribution than the open internet is plainly laughable. The copyright agreement does throw a few bones the author's way with these three exceptions:

1. Authors can copy their own article for their use in classrooms.
2. Authors can reuse the work in a textbook they might author.
3. Authors can copy their work for internal distribution within their institution.

Exception #2 is not to be overlooked -- some publishers will not grant that exception. I know one researcher who wanted to re-use an article he'd written as a dissertation chapter and was denied, forcing him to start the research and writing anew.

Current Open Access Journals

Assuming the journals above don't convert themselves to an open access publishing model (one without $3000 fees), the most immediate option for publishing under an open access mandate would be in an journal that's already open access. No, these journals don't have the history or prestige that the above journals have, but I do get the sense that tools like Google Scholar are making the journal name less relevant than in the past. Many of these journals have emerged in just the past 5-10 years, and I'll limit the list below to those that publish primarily in English and appear to receive submissions from U.S.-based researchers. All the journals found below were indexed in the Directory of Open Access Journals (DOAJ).

I divide open access journals into two main camps -- those where the publisher takes copyright, and those where the author retains copyright. Given the open nature of these journals, I imagine some negotiation about copyright would be very acceptable, particularly if using something like the SPARC Author Addendum.

Author Copyright

International Journal for Mathematics Teaching and Learning - IJMTL is a joint publishing effort between Plymouth University, UK, and the College of Nyiregyháza, Hungary. Their author guidelines say nothing about copyright, but authors are expected to do their own copy editing and formatting of their final article. The articles I looked at made no mention of copyright or licensing, so I assume authors have retained copyright and have the right to assign a Creative Commons license if they wish.

Journal of Statistics Education - JSE has been published by the American Statistical Association since 1993 and clearly indicates the author's copyright on each article.

Journal of Urban Mathematics Education - JUME is edited by David Stinson at Georgia State and appears to be one of the higher-quality open access efforts, publishing articles by William Tate, Rico Gutstein, Megan Staples, Jere Confrey, Michael Battista, Jo Boaler, and others. Authors retain copyright with first publication rights granted to JUME.

Numeracy - This journal specializes in quantitative literacy and is hosted by the University of South Florida. Authors retain copyright under a Creative Commons Attribution 3.0 license. There are no publication charges.

Philosophy of Mathematics Education Journal - Edited by Paul Ernest at the University of Exeter, UK, this journal has existed since 1990 and the copyright notice reads: "All materials published herein remain copyright of the named author(s), or of the editor if unattributed. Permission is given to freely copy the journal contents on a not-for-profit basis, provided full credit is given to the author and the journal." This sounds much like a "legal lite" interpretation of a Creative Commons Attribution - Non Commercial license, although there's no explicit mention of derivative works.

Pythagoras - This is the journal of the Association for Mathematics Education of South Africa, in existence since 1980. The copyright notice on a recent article indicates that the author retains the copyright and the work is licensed under a Creative Commons Attribution license.

Technology Innovations in Statistics Education - TISE is edited by Robert Gould at UCLA and authors retain copyright with publication under a Creative Commons Attribution - Non Commercial - Share Alike license.

The Teaching of Mathematics - This is published by the Mathematical Society of Serbia and makes no mention of copyright on their site or on published articles, so I assume copyright would stay with the author.

Publisher Copyright

Contemporary Issues in Technology and Teacher Education - Part of this journal includes articles about technology and mathematics education, with editorial review organized by AMTE. The publisher retains copyright to published articles.

International Electronic Journal of Mathematics Education - IEJME is published by Gökkuşağı, a Turkish publisher, and dates back to 2006. Their author guidelines don't say anything about who retains copyright of the published articles, but the journal itself indicates the copyright is held by Gökkuşağı.

Journal of Mathematics Education at Teachers College - This is a good looking journal with submissions from some well-known authors. The submission guideline page says nothing about copyright, but the journal itself and each article claims copyright for the publisher.

Journal of STEM Education - This journal requires the author pay a fee ($395 for the first 8 pages and author's bio, then $35 for each additional page) and also requires the transfer of copyright.

Statistics Education Research Journal - This international journal appears to include a wide variety of content but requires authors to transfer copyright.

The Mathematics Educator - TME is a student-produced journal from the University of Georgia and was first published in 1990. Despite the maturity of this journal, copyright is very unclear -- the site says nothing about transferring copyright, and the journal itself claims copyright for the publisher in the front matter, but nothing on articles themselves, and articles are available individually.

The Mathematics Enthusiast - Formerly known as the Montana Mathematics Enthusiast, this journal says nothing on the site about transferring copyright, but the articles themselves indicate a copyright held by the publisher.

Conclusion

It's difficult for me to predict exactly how an open access mandate would affect current journals. Those top four journals, because of their prestige, might not change a thing and hope to get submissions from authors who aren't funded by major federal agencies. Some of the open access journals are obscure now and will likely stay that way, at least to U.S. researchers. What I'd like to see is some of the currently closed "second-tier" journals open themselves. Some, like The Journal of Mathematical Behavior (an Elsevier publication), isn't a likely candidate. For a journal like For the Learning of Mathematics, opening access might be easier. (FLM already has a FAQ including the question, "Can I reprint an FLM article on my web site / anthology / lunch box?" with the simple answer, "If you are interested in reprinting articles that appear in FLM, please contact the managing editor.") Some journals already have policies that would seem to pull them in the direction of open access. Teaching Statistics, for example, is closed but allows authors to retain copyright so long as they give an exclusive license to publish to the journal. I don't know what good it is to have a copyright but no right to publish, but some tweaking of those policies might turn such a journal into something open.

Then again, maybe math education researchers will gravitate towards current large open access repositories. The article Number Concepts without Number Lines in an Indigenous Group of Papua New Guinea caught my eye not just for its content, but the fact it is published in PLoS ONE, a journal that's flourished publishing open access science and medicine content, not necessarily education-related articles. But there's no reason PLoS ONE can't expand its scope, something it's likely to do if new governement open access policies demand more open publications in more content areas.

RYSK: Gutiérrez's (Re)Defining Equity: The Importance of a Critical Perspective (2007)

This is the eighth in a series of posts describing "Research You Should Know" (RYSK).

Do you ever find yourself talking about something, defending something, or promoting something when you suddenly realize you don't have a good definition of that thing?

In one way or another, I've been thinking about equity in math education ever since I was an undergraduate. I remember debating the value of "equality of opportunity" versus "equality of outcomes," and getting a sense for how the NCTM Standards prescribed a type of school mathematics for all students. Here at CU-Boulder, issues of equity and social justice are never far away. But what, exactly, do we mean by equity in math education? And why is it important?

Rochelle Gutiérrez focuses on issues of equity as an associate professor of mathematics education at the University of Illinois at Urbana-Champaign. Even though she's been publishing on issues of equity for well over a decade, in 2007 she wrote a book chapter titled, (Re)Defining Equity: The Importance of a Critical Perspective. In that chapter, she argues why we need a definition of equity that gives teachers and researchers a clear sense of purpose.

When equity is loosely defined, it comes under attack from several directions. First is a belief that not all students can learn, and that mathematical proficiency has more to do with natural ability than with effort. The second threat to equity is a "deficit theory" towards groups of students that haven't had much historical success in mathematics, whether that deficit is seen as biological or cultural. The third threat to equity, says Gutiérrez, comes from within the research community itself: so many issues get covered under the umbrella of "equity" that few of them get the kind of focused attention they need, even while many agree equity is important. As Gutiérrez puts it:

Perhaps the lack of a clear definition is what contributes to a general consensus that equity is worth striving for, everyone having his or her own vision of what it means. However, having a poorly defined target means we are only sure we are moving toward it when, in fact, we are very far away. (p. 38)

Gutiérrez argues that we should leave behind the traditional "excellence versus equity" and "traditional versus reform" debates in favor of a new perspective: dominant versus critical. Instead of teaching mathematics that "reflects the status quo in society, that gets valued in high-stakes testing and credentialing, that privileges a static formalism in mathematics," (p. 39), we should be favoring critical mathematics, that which "squarely acknowledges the positioning of students as members of a society rife with issues of power and domination" (p. 40). This includes using math to examine social and political issues, to highlight perspectives of different cultures, and to challenge the view that mathematics is a static entity. Gutiérrez does not wish to create a dichotomy here -- in fact, she argues the importance of learning dominant mathematics because it can help students better understand and criticize the world.

With that perspective in mind, Gutiérrez defines equity. First, she warns not to confuse it with equality; whereas equity implies "justice" or "fairness," equality implies "sameness." Gutiérrez is *not* arguing that all students should experience the same instruction using the same materials, or that we should expect all students to have the same outcomes. Gutiérrez fully recognizes that, within any group, experiences and outcomes will vary, and that some students will have interests that lead them away from mathematics. That's okay. Instead, she says equity in mathematics should consist of three main parts:

1. "Being unable to predict students' mathematics achievement and participation based solely upon characteristics such as race, class, ethnicity, gender, beliefs, and proficiency in the dominant language" (p. 41, emphasis original). To clarify, Gutiérrez says, "I contend that only when there is sufficient variation within groups and no clear patterns associated with power or status in society between groups can we conclude that this aspect of equity is being addressed" (p. 42, emphasis original). As for measuring achievement, Gutiérrez says we should use standardized tests (but not exclusively), because those are often the tools we use to grant power to individuals.
2. "Being unable to predict students' ability to analyze, reason about, and especially critique knowledge and events in the world as a result of mathematical practice, based solely upon characteristics such as race, class, ethnicity, gender, beliefs, and proficiency in the dominant language" (p. 45, emphasis original). It's this aspect of equity that Gutiérrez uses to stress the critical aspects described above.
3. "An erasure of inequities between people, mathematics, and the globe" (p. 48, emphasis original). Gutiérrez claims "This aspect of equity addresses the fact that having equal access to cultural capital and critical stances to society are necessary but insufficient conditions for change" (p. 48). While this aspect of equity is by far the most difficult to measure, and may not happen in our lifetimes, it should be the key goal of any long-term reform in mathematics education.

Gutiérrez closes her paper with this paragraph:

It might be the case that the first two aspects of equity must be addressed before we would see any changes in the third aspect. That is, students who gain both (1) dominant and (2) critical mathematics identities will lead to different kinds of mathematicians in the academy, thereby changing what counts as mathematics as well as how it is evaluated. The important thing to consider in this (admittedly simplistic) model is that neither the first nor the second aspects of equity are sufficient to redress injustices in the world. Students need to be able to do both -- be able to play the game of mathematics that is currently associated with power and intellectual potential, and be able to change the game of mathematics to serve a better society. (p. 49)

References



Gutiérrez, R. (2007). (Re)defining equity: The importance of a critical perspective. In N. S. Nasir & P. Cobb (Eds.), Improving access to mathematics: Diversity and equity in the classroom (pp. 37-50). New York, NY: Teachers College Press.

RYSK: Staples's Supporting Whole-Class Collaborative Inquiry in a Secondary Mathematics Classroom (2007)

This is the seventh in a series of posts describing "Research You Should Know" (RYSK).

In her book What's Math Got to Do With It?, Jo Boaler recounted her first contact with the math wars. A group of parents at a local school had organized against the adoption of reform textbooks, and were telling students that if they took the classes with the new books, they wouldn't be eligible for college. Apparently the parents had called admissions offices and asked a question like, "Would you accept a student who had not taken any math in high school but had just talked about math?" (Boaler, 2008, p. 33). Of course the colleges said no, and from that question and response parents based a claim about reform texts and college admission.

If we try to put angry politics aside for a moment, where did the parents in Boaler's story get the idea that reform math was all about talking about math? Certainly that thought was not a total fabrication. In fact, the learning theories that influenced much of the reform math movement led teachers and researchers to think about how to structure classrooms in ways that maximized learning, and that led to an attention to classroom discourse -- the teacher-student and student-student speaking and writing that happens in classrooms. By studying classroom discourse, we can gain key insights about what and how math is learned, and how our experiences and our environment affect that learning. (This, I believe, is not unique to reform classrooms -- all good math teachers and students pay careful attention to how we talk about and otherwise communicate mathematics.)

Megan Staples, a former advisee of Jo Boaler, is an assistant professor at the University of Connecticut specializing in mathematics education and classroom discourse. In her article Supporting Whole-Class Collaborative Inquiry in a Secondary Mathematics Classroom (2007), Staples takes on the challenges of being the teacher and supporting students in "collaborative inquiry." Instead of simply viewing collaboration as working in groups, or cooperating to complete a task, Staples says collaboration, "implies a joint production of ideas, where students offer their thoughts, attend and respond to each other's ideas, and generate shared meaning or understanding through their joint efforts" (p. 162). As for inquiry, Staples sees it as a way of "engaging with and making sense of the world" (p. 163), and applies the term to "both inquiry into mathematics and inquiry with mathematics" (p. 163). Finally, and perhaps most importantly, is how Staples defines learning mathematics itself. Using a situative perspective, Staples sees mathematics as a cultural practice, where "learning results from, and is evidenced by, student participation in both standard and disciplinary practices (e.g., justifying, representing algebraically) and an array of other practices of mathematical communities (e.g., questioning, communicating, informal reasoning)" (p. 163). With these conceptions of collaboration, inquiry, and learning mathematics in mind, Staples conducted a year-long observation and analysis of Ms. Nelson, a veteran, award-winning teacher known for her dedication to reform mathematics principles. The class Staples analyzed was called "Math A," a lower-track 9th grade class for students with a history of low performance in traditional mathematics classrooms.

Staples's analysis yielded two models for teaching that support student participation in class discussion. The first model describes the role of the teacher in a whole-class discussion, while the second describes how Ms. Nelson increases the class's ability to collaborate over time. I'll present Staples's findings in outline form, with attention to specific recommendations for teachers wishing to support collaborative inquiry in their own classrooms. (Be patient -- the original article is 57 pages long, after all.)

1. Model 1: The teacher's role in supporting whole-class collaborative inquiry
   
   1. Supporting students in making contributions
      
      • Eliciting student ideas -- Instead of just asking questions like, "Why?" or "How do you know?," Ms. Nelson presses students to share with comments like, "Come on, I'm really interested, come on, you can do it" (p. 175), gave students adequate time to formulate explanations, and offered participation points as a reward for contributing ideas.
      • Scaffolding the production of student ideas -- Ms. Nelson helps direct struggling students to use multiple representations and models, such as number lines, graphs, diagrams, etc. The key, says Staples, is to provide structure for the mathematics without constraining how the students will work out the mathematics (p. 178).
      • Creating contributions - Ms. Nelson treated incomplete and incorrect contributions by students the same as correct ideas, saying things like, "Remember the idea is to go up and give us some good discussion...that helps the class move along regardless of whether it's right or wrong, it enables us to have good discussion" (pp. 178-179).
   2. Establishing and monitoring a common ground
      
      • Creating a shared context -- Ms. Nelson focused on creating shared contexts among students. This was accomplished by repeating of student statements and encouraging students to record and share their representations and ideas on the board.
      • Maintaining continuity over time -- Ms. Nelson emphasized a sense of purpose when asking students to contribute. She directed students with phrases like, "Come up [to the board] please. Ron says that there are more diagonal lines. That Oscar didn't put enough in" (p. 181). Ms. Nelson also gave students time to understand and add clarity to other students' ideas before introducing new ideas.
      • Coordinating the collective -- Ms. Nelson actively positions students to respond to each other. When a student, Jay, had difficulty explaining an idea and Ken raised his hand, Ms. Nelson asked, "OK, do you wanna explain some more Ken? Or do you have a question for Jay?" (p. 185).
   3. Guiding the mathematics
      
      • Guiding high-level task implementation -- Ms. Nelson selected tasks that were difficult enough to invite collaboration, but guided students in ways that avoided unproductive exploration. This sometimes involved recounting the steps students had taken to reach their current thinking or requesting new representations of ideas. Either way, the focus was on how the students were thinking about the problem, and not just giving hints for the next step.
      • Guiding with a map of students' algebra learning -- This is where Ms. Nelson showed her experience with mathematics and the learning of mathematics, knowing the "pressure points" (p. 190) where students needed to pay particular attention to the structure of the mathematics.
      • Guiding by following: "going with the kids" -- Ms. Nelson showed a willingness to let go and follow students' thinking and be flexible with the intended destination of the lesson.
2. Model 2: The development of a community of collaborative learners
   
   1. The development of practices over time -- High school is a difficult time to introduce collaborative inquiry because students have longer histories with traditional mathematics and because classes meet for a limited time each day. Expectations need to be made explicit and modeled for students.
   2. The model -- Developing community is an iterative process involving tasks or strategies that Staples calls "cycle starters" (p. 196) that spur student participation, which elicits negotiation of meanings, which leads to student interpretations and understandings. From there the cycle can repeat and improve.
   3. Negotiation of meanings -- For example, early in the year students, when asked to explain, automatically assume they've given a wrong answer. Once this practice is established, students improve in the ways they respond to questions about their thinking.
      
      • Helping students make sense of practices -- Early in the year, Ms. Nelson would fill in commentary when students did work silently in front of the class, saying things like, "He is noticing a pattern over here" (p. 198). This modeled the practice of thinking aloud for the class and emphasized the value of sharing one's thinking.
      • Providing evidence for the value for learning -- When students struggled, Ms. Nelson encouraged them to stop and express what it was they were struggling with. Making mistakes became an acceptable part of doing mathematics so long as they became opportunities to learn and correct misunderstandings.
      • Negotiation of the joint enterprise -- Ms. Nelson explained early in the year that these students were doing to do mathematics differently than in the past, and that they didn't need the math dumbed-down just because they hadn't been successful before.
   4. Cycle starters -- Ms. Nelson included not only engaging tasks, but helped create a vision of how that task could be accomplished, sometimes by describing the expected collaboration but also by showing a video of older students collaborating and discussing how they worked together.
   5. Students' interpretations and understandings of practices -- From student interviews and surveys, Staples found that students interacted with each other during class either for social reasons or to make the class less boring. By the end of the year, students reported that working together helped them learn because of the opportunities to listen and respond to each others' ideas.
   6. Transforming a community's repertoire -- Ms. Nelson's effort to transform the way it does mathematics was an ongoing effort that lasted the entire year. The effort is a negotiation, where as the class built new experiences together they could reflect on what was working and adjust their practices in future lessons.

In her discussion, Staples focuses on how teachers support collaborative inquiry while maintaining their role. It is certainly possible for a teacher to ask students to share ideas or report strategies, but it takes extra effort to build that common ground where students analyze and evaluate each other's ideas. Defining this common ground is difficult and it will be different in every classroom, but it is up to the teacher to develop and maintain it throughout the school year. Teachers also must be mindful of the mathematics, even when "going with the kids." It takes a skillful teacher to subtly push the mathematics while keeping the class collaborative. Lastly, the teacher must maintain a sense for a "long-term trajectory of student learning" (p. 210), although having such a sense does not necessarily support collaboration by itself. Staples's research is thorough and well-grounded in qualitative methodology, but it is not without its criticisms. I see critiques coming from two directions: from a more cognitive perspective, Staples doesn't attend much to individual student thinking, preferring to focus on social practices and classroom norms for participation. From a more purely sociocultural perspective, Staples doesn't account for how the influence of other, beyond-the-classroom cultures and community norms affect how students approach and understand mathematics. This is not to fault Staples, however -- she prefaced her findings with defining a situative perspective, and she maintained that perspective throughout. This just means that there are multiple ways of describing classroom communities and learning, and more work can be done to describe and build bridges across multiple perspectives.

References

Boaler, J. (2008). What’s math got to do with it? How parents and teachers can help children learn to love their least favorite subject (p. 273). New York, NY: Penguin Group.

Staples, M. (2007). Supporting whole-class collaborative inquiry in a secondary mathematics classroom. Cognition and Instruction, 25(2), 161-217. doi:10.1080/07370000701301125