RYSK: Shulman's Those Who Understand: Knowledge Growth in Teaching (1986)

This is the 15th in a series describing "Research You Should Know" (RYSK) and part of my OpenComps. I also Storified this article as I read.

Lee Shulman. (CC BY-NC) Penn State

George Bernard Shaw once said, "He who can, does. He who cannot, teaches." For that, you could say that Lee Shulman takes offense. Shulman, a long-time faculty member at both Michigan State (1963-1982) then Stanford, explained his position and a new way of thinking about teacher knowledge in his AERA Presidential Address and the paper, Those Who Understand: Knowledge Growth in Teaching. Shulman is now an emertius professor but stays active traveling, speaking, and occasionally blogging.

Wondering why the public often has a low opinion of teachers' knowledge and skill, Shulman first looks at the history of teacher examinations. In the latter half of the 1800s, examinations for people wishing to teach were almost entirely content-based. In 1875, for example, the California State Board examination for elementary teachers gave a day-long, 1000-point exam that covered everything from mental arithmetic to geography to vocal music. Its section on the theory and practice of teaching, however, was only worth 50 of the 1000 points and included questions like, "How do you interest lazy and careless pupils?" (p. 5)

By the 1980s, when Shulman wrote this article, teacher examinations painted almost the opposite picture. Instead of focusing on content, they focused on topics such as lesson planning, cultural awareness, and other aspects of teacher behavior. While the topics usually had roots in research, they clearly did not represent the wide spectrum of skills and knowledge a teacher would need to be a successful teacher. More specifically, by the 1980s our teacher examinations seemed to care as little about content as the examinations a century prior seemed to care about pedagogy.

Looking back even further in history, Shulman recognized that we haven't always made this distinction between content and teaching knowledge. The origins of the names of our highest degrees, "master" and "doctor," both essentially mean "teacher" and reflected the belief the highest form of knowing was teaching, an idea going back to at least Aristotle:

We regard master-craftsmen as superior not merely because they have a grasp of theory and know the reasons for acting as they do. Broadly speaking, what distinguishes the man who knows from the ignorant man is an ability to teach, and this is why we hold that art and not experience has the character of genuine knowledge (episteme) -- namely, that artists can teach and others (i.e., those who have not acquired an art by study but have merely picked up some skill empirically) cannot. (Wheelwright, 1951, as cited in Shulman, 1986, p. 7)

Shulman saw a blind spot in this dichotomy between content and teaching knowledge. What he saw was a special kind of knowledge that allows teachers to teach effectively. After studying secondary teachers across subject areas, Shulman and his fellow researchers looked to better understand the source of teachers' comprehension of their subject areas, how that knowledge grows, and how teachers understand and react to curriculum, reshaping it into something their students will understand.

Pedagogical Content Knowledge

To better understand this special knowledge of teaching, Shulman suggested we distinguish three different kinds of content knowledge: (a) subject matter knowledge, (b) pedagogical content knowledge, and (c) curricular knowledge. It was the second of these, pedagogical content knowledge (PCK), that Shulman is best remembered for. Shulman describes the essence of PCK:

Within the category of pedagogical content knowledge I include, for the most reguarly taught topics in one's subject area, the most useful forms of representation of those ideas, the most powerful analogies, illustrations, examples, explanations, and demonstrations -- in a word, the ways of representing and formulating the subject that make it comprehensible to others. Since there are no single most powerful forms of representation, the teacher must have at hand a veritable armamentarium of alternative forms of representation, some of which derive from research whereas others originate in the wisdom of practice. (p. 9)

In addition to these three kinds of teacher knowledge, Shulman also proposed we consider three forms of teacher knowledge: (a) propositional knowledge, (b) case knowledge, and (c) strategic knowledge. These are not separate from the three kinds of knowledge named above, but rather describe different forms of each kind of teacher knowledge. Propositional knowledge consists of those things we propose teachers do, from "planning five-step lesson plans, never smiling until Christmas, and organizing three reading groups" (p. 10). Shulman organized propositional knowledge into principles, maxims, and norms, with the first usually emerging from research, the second coming from a practical experience (and generally untestable, like the suggestion to not smile before Christmas), and the third concerning things like equity and fairness. Propositions can be helpful but difficult to remember to implement as research intended.

Learning propositions out of context is difficult, so Shulman proposed case knowledge as the second form of teacher knowledge. By case, he means learning about teaching in a similar way a lawyer learns about the law by studying prior legal cases. In order to truly understand a case, a learner starts with the factual information and works towards the theoretical aspects that explain why things happened. By studying well-documented cases of teaching and learning, teachers consider prototype cases (that exemplify the theoretical), precedents (that communicate maxims), and parables (that communicate norms and values). (If you're scoring at home, Shulman has now said there are three types of cases, which itself is one of three forms of knowledge, each of which capable of describing three different kinds of content knowledge.)

The last form of knowledge, strategic knowledge, describes how a teacher reacts when faced with contradictions of other knowledge or wisdom. Knowing when to bend the rules or go against conventional wisdom takes more than luck -- it requires a teacher to be "not only a master of procedure but also of content and rationale, and capable of explaining why something is done" (p. 13).

The value of this article by Shulman goes beyond the theoretical description of pedagogical content knowledge. Additionally, this article serves as a strong reminder that when we judge a teacher, we must consider a broad spectrum of skills and abilities, and not limit ourselves to only those things we think can be easily measured. As Shulman explains:

Reinforcement and conditioning guarantee behavior, and training produces predictable outcomes; knowledge guarantees only freedom, only the flexibility to judge, to weigh alternatives, to reason about both ends and means, and then to act while reflecting upon one's actions. Knowledge guarantees only grounded unpredictability, the exercise of reasoned judgment rather than the display of correct behavior. If this vision constitutes a serious challenge to those who would evaluate teaching using fixed behavioral criteria (e.g., the five-step lesson plan), so much the worse for those evaluators. The vision I hold of teaching and teacher education is a vision of professionals who are capable not only of acting, but of enacting -- of acting in a manner that is self-conscious with repect to what their act is a case of, or to what their act entails. (p. 13)

In our current era of teacher evaluation and accountability, with all its observational protocols and test score-driven value added models, this larger view of teaching presented to us by Shulman is a gift. His recommendation that teacher evaluation and examination "be defined and controlled by members of the profession, not by legislators or laypersons" (p. 13) is a wise one, no matter how politically difficult. Shulman hoped for tests of pedagogical content knowledge that truly measured those speical skills that teachers have, skills that non-teaching content experts would not pass. I don't think those measurement challenges have been overcome, but continuing towards that goal should strengthen teacher education programs while also improving the perception of teaching as a profession. As Shulman concludes (p. 14):

We reject Mr. Shaw and his calumny. With Aristotle we declare that the ultimate test of understanding rests on the ability to transform one's knowledge into teaching.

Those who can, do. Those who understand, teach.

References

Shulman, L. S. (1986). Those who understand: Knowledge growth in teaching. Educational Researcher, 15(2), 4–14. Retrieved from http://www.jstor.org/stable/3202180

A First Day Statistics Activity

I have the honor of again teaching our undergraduate statistics course in the School of Education, better known here as EDUC 4716 Basic Statistical Methods. Perhaps the most interesting thing about the course is that it's not required for any education programs, minors, or certificates. Instead, the course attracts students largely from the Department of Speech, Language, and Hearing Sciences (who don't need it to graduate, but do need it to apply to grad school or, more recently, to get certified) and sociology majors. So how does this course end up in the School of Ed? Probably due to the legacy we have in quantitative methods, thanks to people like Robert Linn, Gene Glass, Lorrie Shepard, and now faculty like Derek Briggs and Greg Camilli. Somehow all of their hard work and success filters down and gives a relative stats-hack like me a chance to teach undergrads.

Many of my students are upperclassmen and have spent much of their college experience avoiding math courses. In fact, on last year's FCQ (Faculty Course Questionnaire) my students' average rating for the item "Personal interest in this subject prior to enrollment" was a 1.8 out of 6 -- a response the university tells me is at the 0th percentile across campus. I like to think of this as a great opportunity in a "nowhere to go but up" kind of way, a chance for me to change the way students think of mathematics and see themselves as mathematical beings. Then again, it's hard to make big changes in only 15 class meetings of 2.5 hours each. If I'm going to make a difference, class has to get off to a solid start.

My opening activity this year started with the preparation of four simple index cards with different distribution shapes:

Four common distributions, clockwise from top left: normal, left skewed, right skewed, and normal.

I have the benefit of a small class of 14 students. So I cut my graphs into a total of 14 pieces:

14 pieces for 14 students. Note on the bottom I've provided the hints A, B, C, and D.

When class started, I mixed up the graph pieces and handed one to each student. Then I told the class to find the other people in class who had the graph pieces that aligned with theirs. Once they had a completed graph, form a group at one of the tables and discuss which of the following they thought their group's graph might describe:

• People born each month of the year
• Student GPAs at this university
• Student heights at this university
• Starting salaries of new graduates from this university

It took my class less than 3-4 minutes to find their groups and then I gave them another 3-4 minutes to discuss what their graph shape might describe. As a class, I had each group share their ideas and then we discussed them. Not everybody agreed initially about which shape matched which description, which led into important comments about how we might think about unbiased sampling of students and imagining different scales and labels along the horizontal axes.

So in less than 15 minutes I combined group-making, statistics, and active, student-centered problem solving into one activity. This activity also gets students thinking about distribution shapes, which I sometimes worry we ignore in the rush to calculate centers and spreads. If you're wondering how to adapt this for your classroom, I offer these suggestions:

• If you have a few more students, cut more slices.
• If you have twice as many students, consider making two of each distribution shape and scaling the x-axis to match one of 8 potential descriptions. (i.e., a normal distribution scaled for heights in inches could be distinguished from one scaled for SAT scores.)
• If you want to use this for Algebra 1, you can make graphs that describe things like, "Toni walked to the bus stop at 2 mph, rode the bus at 30 mph to the bike shop, then rode a bike back home at 12 mph." Such an activity begins CPM's Algebra Connections and was the inspiration for my activity.
• If you want to use this for Algebra 2 or higher, you can use graphs of functions that students will become familiar with (parabolas, cubics, hyperbolas, etc.). I don't think it's worth fretting over vocabulary at this point -- just give students an opportunity to think about how the functions behave and what phenomena they could possibly model.

RYSK: Staples's Supporting Whole-Class Collaborative Inquiry in a Secondary Mathematics Classroom (2007)

This is the seventh in a series of posts describing "Research You Should Know" (RYSK).

In her book What's Math Got to Do With It?, Jo Boaler recounted her first contact with the math wars. A group of parents at a local school had organized against the adoption of reform textbooks, and were telling students that if they took the classes with the new books, they wouldn't be eligible for college. Apparently the parents had called admissions offices and asked a question like, "Would you accept a student who had not taken any math in high school but had just talked about math?" (Boaler, 2008, p. 33). Of course the colleges said no, and from that question and response parents based a claim about reform texts and college admission.

If we try to put angry politics aside for a moment, where did the parents in Boaler's story get the idea that reform math was all about talking about math? Certainly that thought was not a total fabrication. In fact, the learning theories that influenced much of the reform math movement led teachers and researchers to think about how to structure classrooms in ways that maximized learning, and that led to an attention to classroom discourse -- the teacher-student and student-student speaking and writing that happens in classrooms. By studying classroom discourse, we can gain key insights about what and how math is learned, and how our experiences and our environment affect that learning. (This, I believe, is not unique to reform classrooms -- all good math teachers and students pay careful attention to how we talk about and otherwise communicate mathematics.)

Megan Staples, a former advisee of Jo Boaler, is an assistant professor at the University of Connecticut specializing in mathematics education and classroom discourse. In her article Supporting Whole-Class Collaborative Inquiry in a Secondary Mathematics Classroom (2007), Staples takes on the challenges of being the teacher and supporting students in "collaborative inquiry." Instead of simply viewing collaboration as working in groups, or cooperating to complete a task, Staples says collaboration, "implies a joint production of ideas, where students offer their thoughts, attend and respond to each other's ideas, and generate shared meaning or understanding through their joint efforts" (p. 162). As for inquiry, Staples sees it as a way of "engaging with and making sense of the world" (p. 163), and applies the term to "both inquiry into mathematics and inquiry with mathematics" (p. 163). Finally, and perhaps most importantly, is how Staples defines learning mathematics itself. Using a situative perspective, Staples sees mathematics as a cultural practice, where "learning results from, and is evidenced by, student participation in both standard and disciplinary practices (e.g., justifying, representing algebraically) and an array of other practices of mathematical communities (e.g., questioning, communicating, informal reasoning)" (p. 163). With these conceptions of collaboration, inquiry, and learning mathematics in mind, Staples conducted a year-long observation and analysis of Ms. Nelson, a veteran, award-winning teacher known for her dedication to reform mathematics principles. The class Staples analyzed was called "Math A," a lower-track 9th grade class for students with a history of low performance in traditional mathematics classrooms.

Staples's analysis yielded two models for teaching that support student participation in class discussion. The first model describes the role of the teacher in a whole-class discussion, while the second describes how Ms. Nelson increases the class's ability to collaborate over time. I'll present Staples's findings in outline form, with attention to specific recommendations for teachers wishing to support collaborative inquiry in their own classrooms. (Be patient -- the original article is 57 pages long, after all.)

1. Model 1: The teacher's role in supporting whole-class collaborative inquiry
   
   1. Supporting students in making contributions
      
      • Eliciting student ideas -- Instead of just asking questions like, "Why?" or "How do you know?," Ms. Nelson presses students to share with comments like, "Come on, I'm really interested, come on, you can do it" (p. 175), gave students adequate time to formulate explanations, and offered participation points as a reward for contributing ideas.
      • Scaffolding the production of student ideas -- Ms. Nelson helps direct struggling students to use multiple representations and models, such as number lines, graphs, diagrams, etc. The key, says Staples, is to provide structure for the mathematics without constraining how the students will work out the mathematics (p. 178).
      • Creating contributions - Ms. Nelson treated incomplete and incorrect contributions by students the same as correct ideas, saying things like, "Remember the idea is to go up and give us some good discussion...that helps the class move along regardless of whether it's right or wrong, it enables us to have good discussion" (pp. 178-179).
   2. Establishing and monitoring a common ground
      
      • Creating a shared context -- Ms. Nelson focused on creating shared contexts among students. This was accomplished by repeating of student statements and encouraging students to record and share their representations and ideas on the board.
      • Maintaining continuity over time -- Ms. Nelson emphasized a sense of purpose when asking students to contribute. She directed students with phrases like, "Come up [to the board] please. Ron says that there are more diagonal lines. That Oscar didn't put enough in" (p. 181). Ms. Nelson also gave students time to understand and add clarity to other students' ideas before introducing new ideas.
      • Coordinating the collective -- Ms. Nelson actively positions students to respond to each other. When a student, Jay, had difficulty explaining an idea and Ken raised his hand, Ms. Nelson asked, "OK, do you wanna explain some more Ken? Or do you have a question for Jay?" (p. 185).
   3. Guiding the mathematics
      
      • Guiding high-level task implementation -- Ms. Nelson selected tasks that were difficult enough to invite collaboration, but guided students in ways that avoided unproductive exploration. This sometimes involved recounting the steps students had taken to reach their current thinking or requesting new representations of ideas. Either way, the focus was on how the students were thinking about the problem, and not just giving hints for the next step.
      • Guiding with a map of students' algebra learning -- This is where Ms. Nelson showed her experience with mathematics and the learning of mathematics, knowing the "pressure points" (p. 190) where students needed to pay particular attention to the structure of the mathematics.
      • Guiding by following: "going with the kids" -- Ms. Nelson showed a willingness to let go and follow students' thinking and be flexible with the intended destination of the lesson.
2. Model 2: The development of a community of collaborative learners
   
   1. The development of practices over time -- High school is a difficult time to introduce collaborative inquiry because students have longer histories with traditional mathematics and because classes meet for a limited time each day. Expectations need to be made explicit and modeled for students.
   2. The model -- Developing community is an iterative process involving tasks or strategies that Staples calls "cycle starters" (p. 196) that spur student participation, which elicits negotiation of meanings, which leads to student interpretations and understandings. From there the cycle can repeat and improve.
   3. Negotiation of meanings -- For example, early in the year students, when asked to explain, automatically assume they've given a wrong answer. Once this practice is established, students improve in the ways they respond to questions about their thinking.
      
      • Helping students make sense of practices -- Early in the year, Ms. Nelson would fill in commentary when students did work silently in front of the class, saying things like, "He is noticing a pattern over here" (p. 198). This modeled the practice of thinking aloud for the class and emphasized the value of sharing one's thinking.
      • Providing evidence for the value for learning -- When students struggled, Ms. Nelson encouraged them to stop and express what it was they were struggling with. Making mistakes became an acceptable part of doing mathematics so long as they became opportunities to learn and correct misunderstandings.
      • Negotiation of the joint enterprise -- Ms. Nelson explained early in the year that these students were doing to do mathematics differently than in the past, and that they didn't need the math dumbed-down just because they hadn't been successful before.
   4. Cycle starters -- Ms. Nelson included not only engaging tasks, but helped create a vision of how that task could be accomplished, sometimes by describing the expected collaboration but also by showing a video of older students collaborating and discussing how they worked together.
   5. Students' interpretations and understandings of practices -- From student interviews and surveys, Staples found that students interacted with each other during class either for social reasons or to make the class less boring. By the end of the year, students reported that working together helped them learn because of the opportunities to listen and respond to each others' ideas.
   6. Transforming a community's repertoire -- Ms. Nelson's effort to transform the way it does mathematics was an ongoing effort that lasted the entire year. The effort is a negotiation, where as the class built new experiences together they could reflect on what was working and adjust their practices in future lessons.

In her discussion, Staples focuses on how teachers support collaborative inquiry while maintaining their role. It is certainly possible for a teacher to ask students to share ideas or report strategies, but it takes extra effort to build that common ground where students analyze and evaluate each other's ideas. Defining this common ground is difficult and it will be different in every classroom, but it is up to the teacher to develop and maintain it throughout the school year. Teachers also must be mindful of the mathematics, even when "going with the kids." It takes a skillful teacher to subtly push the mathematics while keeping the class collaborative. Lastly, the teacher must maintain a sense for a "long-term trajectory of student learning" (p. 210), although having such a sense does not necessarily support collaboration by itself. Staples's research is thorough and well-grounded in qualitative methodology, but it is not without its criticisms. I see critiques coming from two directions: from a more cognitive perspective, Staples doesn't attend much to individual student thinking, preferring to focus on social practices and classroom norms for participation. From a more purely sociocultural perspective, Staples doesn't account for how the influence of other, beyond-the-classroom cultures and community norms affect how students approach and understand mathematics. This is not to fault Staples, however -- she prefaced her findings with defining a situative perspective, and she maintained that perspective throughout. This just means that there are multiple ways of describing classroom communities and learning, and more work can be done to describe and build bridges across multiple perspectives.

References

Boaler, J. (2008). What’s math got to do with it? How parents and teachers can help children learn to love their least favorite subject (p. 273). New York, NY: Penguin Group.

Staples, M. (2007). Supporting whole-class collaborative inquiry in a secondary mathematics classroom. Cognition and Instruction, 25(2), 161-217. doi:10.1080/07370000701301125

A 2-for-1 "Soft Skills" Special: The Sit-Stand Paradox and Defective Girls

Written for The Virtual Conference on Soft Skills, July 3 - July 31, 2010

Of all my courses as a pre-service math education major, I think I enjoyed educational psychology the least. When you spend much of every day deciphering the infallibility of mathematics, the "theories" of social science don't hold up well against the scrutiny of a brain hardened by the concept of rigorous proof. I now realize I should have adjusted my perspective in whatever way necessary to ensure I got more out of the class, but even if I had I don't think anything would have fully prepared me for a classroom full of independently-minded students. You just have to jump in there, year after year, class after class.

In my six years of teaching high school math I developed some wonderful relationships with my students. Without overstepping the bounds of a teacher-student relationship, my students became my friends, something I seem to remember being told I should never let happen. But I would look forward to seeing my students each day; I would try to make the most of my time with them, and I would miss them when they were gone. If that doesn't describe "friends," then I apparently don't know what a friend is. I might be in the "ivory towers" of academia now, but I honestly think of my former students from those six years every single day.

That's not to say that there weren't MANY bumps and hiccups along the way, and I regret the lack of effort and deficits in my own character that prevented me from forming stronger relationships with ALL my students. But as I reflect back, two lessons learned (one a realization, the other a piece of advice) helped strengthen that special student-teacher bond.

The Sit-Stand Paradox

Ask any teacher what period of the day is likely to be their least favorite and most will answer, "last period." It doesn't matter if your school has four, six, seven, or eight periods -- there is always a last period. By far my toughest group of kids I ever attempted to teach was a last-period business math class. It's a bad sign when, on the very first day of class, a student who you've just met pulls you aside and tells you, "I don't know who decided to put this mix of students together in the same class, but it's a really, really bad idea." I'd like to think my chances with them would have been much better if I'd seen them before lunch.

So what makes last period so tough? I think the explanation is simple: after a long day at school, students are tired of sitting and teachers are tired of standing. Should you be trying to hide this fact from your students? No! They're people, not circus animals that might attack if they sense fear. If you want a class that works with you, not against you, share your motivations and frustrations. Establish common goals and understandings so you can move forward together. Maybe it's time for an out-of-seat activity, a lesson outside, or a trip to a less familiar room in the school. I know it sounds easier than it really is, but even something as simple as sending your students to the board while you sit at their desk can be just the change in perspective everybody needs. If you're worried that instruction might suffer with a little chaos, think of how much it's already suffering when all the students are watching the clock hoping to be somewhere else.

Don't Treat Boys Like They're Defective Girls

During my third year of teaching I had the privilege to share a classroom with Miss Sandra Miley, a 30+ year educator who took a distinct pleasure in teaching freshman boys' seminar and P.E. If you didn't already know, freshman boys are at that awkward age (which lasts from about 11 to 25, as far as I can tell) that can make them awfully hard to teach. Not so for Miss Miley, who passed on this hard-earned wisdom:

"Do you want to know the secret to teaching boys? It's simple. Don't treat them like they're defective girls."

Ever since I was given that Yoda-like advice, I've been trying to unravel the mysteries contained within. Certainly Miss Miley had a perspective from 30+ years in the classroom that I may never match, but I think I got the point. As teachers, our jobs are made easier (not necessarily more enjoyable or effective) when students sit at attention, take notes, raise their hands, follow rules and instructions, and hang on our every word. If you have students who fit that description, I'd bet dollars to doughnuts that the majority of them are girls. Should you want or expect every student to belong in that category? I sure hope not. So don't punish boys who don't happen to behave like those girls. Such behavior is probably not in their DNA.

If you're not convinced, here's a little anecdote to consider. A highly-respected education researcher shared this hypothesis at a conference last fall (identities have been hidden to protect the unpublished):

"I've never been brave enough to try to publish this, but I've long wondered if boys develop better problem-solving skills because they aren't paying attention in class. Girls who listen carefully to instructions and take notes always know exactly where to start because the teacher told them. Boys who goof off during instructions spend a lot more time and effort sorting out the aspects of a problem for themselves, and that practice pays off in the long run."

First Day of School

It seemed fitting to me that the first post on MathEd.net should be like the first day of school.  So please sit in your seats quietly and patiently as I go on for the entire class period telling you what you aren't allowed to do here.

Just kidding.

But isn't that a traditional expectation for the first day?  I tried it that way (several times, I'll admit) and it had to rank as some of the worst teaching I've ever subjected my students to.  I read somewhere once that if you spend the entire first day of school addressing the rules, then you'd better be prepared to deal with them the rest of the year.  I agree.  Instead, I made sure students were involved in a group problem-solving activity the moment they walked in the door.  Rules weren't discussed, but the expectations for my students were clear.

The only rules I had posted in my classroom were my "Math Rules."  You'll have to pardon some of the sarcasm, but I've had success using this kind of humor with students.  (Besides, it's just who I am.)

Math Rules

1. Don't disrespect zero and one. They're your friends. You wouldn't "cancel" a friend, would you?
2. Exact answers are usually better than approximate ones.  Why would you want to do more work to change an exact fraction into an approximate decimal?  Yeah, thought so. If you must "decimalize," two decimal places is usually enough.
3. Of course you have to show your work. Duh! I can't believe you'd even ask such a silly question. The same goes for reducing fractions.
4. Generally, if a complete sentence was used to ask for an answer, respond with a complete sentence. It’s good for you, like vegetables.
5. Don't "plus" things together. Don't "times" things together. Know when to say "add" and "multiply" so you won't sound like a dork.

This list helped accomplish three objectives. First, build rapport with students through humor. Second, establish that how we communicate about math is important. Third, help students become better mathematicians by setting expectations for their work. I think the rules worked well, especially with #5 - students started to look for opportunities to call each other "dorks" and incorrect use of phrases like "I timesed the numbers together" decreased substantially.