Holding Teachers Accountable Ain't That Easy. . .

My boy Bob---who reads the Radical regularly and pushes me with provocative comments---has me thinking about teacher accountability this week.  In response to a recent post about the characteristics of inspirational teachers, he wrote:

I, too, hold similar thoughts, but find mixed results to the following Q: How do you see these ideas translating into student learning rates, the teacher's central duty?

And then in response to my thoughts on preservice educators, he wrote:

As for teacher prep, make it clear that individuals learn, not aggregates. Require student teachers to demonstrate that they can increase measured individual student learning rates before signing off on teaching certificates. Yes? :)

Now, few would mistake me as an apologist for teachers who push back at efforts to hold educators accountable for results.  In fact, I tend to rub colleagues the wrong way more often than not simply because I've grown tired of our resistance to accountability.  It's almost embarrassing to me to hear teachers complain about not earning "professional respect," yet refuse to take responsibility for producing gains in student learning. 

But Bob's comments leave me worried because they illustrate one of the most harmful over-simplifications in education.  The idea that teachers should be held accountable for increasing "measured individual student learning rates" seems obvious, right? 

But it ain't ever that easy.  Here's why:

1.  The term "student learning rates" is poorly defined:  Ask ten parents what kinds of outcomes they most value from schools and you're likely to get ten different answers.

Some will care most about academic achievement measured by standardized tests.  Others will care more about seeing their students learn life skills like how to collaborate.  Society expects schools to prepare students to participate in our democracy.  Public health organizations want us to address challenges ranging from childhood obesity to teenage pregnancy.   

In the end, we need a clear definition of the "student learning rates" that people expect us to increase before we can make any guarantees to anyone----and because of the range of interests that people have in public schools, I'm afraid we'll never get clarity on the outcomes that should be valued the most. 

2.  Defined curricula border on the ridiculous:  Most would agree that teachers should be held accountable for student mastery of their state-defined curriculum, right?  How can we have confusion about what to hold teachers accountable for when there are sets of standards that students are supposed to master in every grade level and content area?

The answer is simple:  State-defined curricula contain far more content than it's actually possible to teach.  Researcher Bob Marzano---in his 2003 book What Works in Schools---determined that the average curriculum for a K-12 student contained nearly 200 standards, which would take nearly 15,000 hours over the course of a child's school career to adequately address. 

The problem:  During the typical 180-day school year, there is an average of about 1,008 hours available for instruction---of which, 69% is actually used for instruction (fire drills and discipline takes time too, you know)----leaving about 9,000 hours of time for student learning between kindergarten and college.  Is it really fair to hold teachers accountable for student mastery of a curriculum that simply won't fit into the teaching hours that we have available to us?   

3.  Testing results in a narrowing of the curriculum and score inflation:  For many, "increasing measured student learning rates" translates to higher scores on standardized tests because standardized test results carry great currency in conversations about education right now. 

They just seem so trustworthy and scientific!

The hitch is this:  Standardized tests only measure student mastery of a small range of skills within a curriculum.  With a little bit of mathematical magic, extrapolations are made and conclusions are drawn about a student's mastery of the entire curriculum based on his or her answers on end of grade exams.

In theory, this should work just fine.  Demonstrated mastery of the subset of skills tested on standardized exams should allow for reliable conclusions about student understanding of the entire curriculum.

In reality, though, it doesn't.  As Daniel Koretz of Harvard University recently showed in his new book Measuring Up, teachers, schools and districts----withering under the pressure to produce results on exams----focus their instruction solely on the skills that they expect to see on standardized tests.

This narrowing of the curriculum produces score inflation.  While results suggest that students have mastered the content for their grade level or content area, they have really only mastered the small set of skills that end-of-grade exams actually test.  Parents and policymakers end up with a false sense of what their kids actually know.

I'm ready for accountability to be introduced to education, primarily because I want the opportunity to be rewarded differently based on my abilities as an educator.  The single-salary schedule has frustrated me since day one because it ensures that my compensation will remain the same regardless of how hard I work to improve my practice. 

But real barriers must be addressed before teachers can be fairly held accountable for "student learning."  We deserve clear definitions of the kinds of learning that communities value the most.  We also deserve to have a manageable curriculum that can be adequately taught in the time that we have available.  Finally, we deserve more complex measures of "achievement" than we currently have. 

Does this make sense to anyone besides me?

(Image credit:  Why Not by Kevin Collins, licensed Creative Commons:  Attribution)

Comments

[Paul Cancellieri]

You respond more eloquently than I ever could have. I have been trying to crystallize my own feelings about the expectations put on teachers, and you have done much to help me along. I try to see every issue from both sides and I am equally frustrated by those in education and outside of it who love to talk about how hard teachers work. Many of my colleagues could improve their practice by working smarter and focusing on improving student learning. I am horrified when teachers throw their hands in the air and claim that they've done their best for a child, but he just can't learn.

And, I honestly believe that performance pay (linked to more than test scores) is the best way to reach the goal.

[Dan Callahan]

Sadly, this is the number one problem for different compensation models. There are so many different variables involved in demonstrating accountability for student learning, and the only real, current measures we have, standardized tests, miss a lot of them. I'm a Special Education teacher, and my students are extremely low-level, so much so that they take an alternate assessment instead of the regular state testing. The assessment has multiple levels of difficulty, but the only reason it seems to be given is to make sure there is No Child Left Untested.

This also leaves the matter of how many teachers we have, especially once you get to the secondary level, who teach subjects that are not tested at all. In my state, grades 3-8 take Reading and Math tests. 10th graders take the same, with (I believe) now 5th, 8th, and 10th graders taking science (as of last year) and writing tests. Assuming the state standardized tests would be used to determine merit pay, that leaves out, among others, teachers of grades K-2, 9, 11, and 12. All Social Studies teachers produce nothing of value. All of the related arts fields are meaningless.

Let's face it, even if we could produce flawless standardized tests that could prove student learning beyond all doubt, do we want to test students in every single one of these subject areas? How much instructional time does your school already lose to state and local testing (to prepare for the state tests, of course)? We have one week of state tests, and 10 other days during the year we have an hour of testing. Language Arts classes have gone on hold for a couple weeks before state tests to go over test-taking strategies. The state tests happen in March, so the Math curriculum is slammed into an even shorter period, where things that are more prominent on the test are made sure to go before the testing date, even if it doesn't make as much sense in the sequence of learning.

Wow, that turned a bit ranty.

[Mike]

Well put, Bill. You've outlined many of the problems relating to "accountability," and to merit pay schemes. There is, however, an over arching concept that is commonly left out of such discussions, when it is acknowledged at all, that is. I refer to the nature of education.

Schools do not educate you; you educate you. When this concept is ignored, or worse, when the opposite is practiced--and much of contemporary educational theory, including high stakes, mandatory tests are based upon the opposite--we find ourselves spending billions of dollars trying to quantify the unquantifiable and chasing the best and brightest out of teaching.

Schools (read "teachers") cannot be all things to all people and to society. They can do one thing only, and for the most part, they do this well: provide educational opportunity. Teachers cannot force students to learn. Students do not have USB or Firewire ports that allow teachers to directly download data into their brains. Teachers cannot be held "accountable" for student failure any more than a jockey can be held accountable for losing a horse race while trying to ride a horse that refuses to move.

No, I'm not saying that teachers shouldn't try to motivate students. I'm not saying that teachers should not work with parents and should not encourage kids, and should not cajole, and should not use every trick in the book (within reason) to help kids recognize and take advantage of the educational opportunity they present. What I'm saying is clearly and simply that if kids won't learn, it's not the teacher's fault, nor can any reasonable, realistic definition of "accountability" attach.

Whose fault is it then (and here I assume that the theoretical teacher is providing the proper educational opportunity--if they're not, that's a supervision problem, not a learning problem)? The fault belongs to the student who is taking no responsibility for their education and the fault belongs to the parent who is taking no responsibility for their role as parent.

Only when you think that teachers are solely responsible for student "outcomes," or whatever term is popular this week, do high stakes tests make sense. I kill an average of two months a year (likely more) specifically drilling my high school kids for state testing. My entire English department does the same, and as a result, our kids pass in the high 90's every year. They learn nothing, however, during those months than how to pass that specific test.

Well. In my state, the shortcomings of the test have been recognized and the legislature is going to fix things--by going to an "end of course" exam. I suspect what this means is that instead of two months, we'll be forced to spend an entire year drilling for a test given at the end of the year, a test whose results won't be known until the following year, when an entirely different end of year test will be required, and so on, ad naseum. Brilliant. But by God, we'll be accountable, even more accountable than now.

Accountability lies in local districts, with involved parents, and with teachers. It does not lie in the generation of data for state and federal educrats. But what about districts that don't perform as they should/could? What about bad teachers? What about corrupt school boards? The answer to those problems is high stakes testing and the single piece of data relative to each child that they generate at the cost of billions of dollars a year? Brilliant. Accountability.

[Sam Rosaldo]

I feel like a first-time caller to a long-standing call-in radio show. "Hi Bill. First time caller--love your show."

I'd first like to respond to Dan. I may be interpreting his comments in a way that he does not intend, but if he's reading this, I'd like him to know how others might receive them. It seems like you're ignoring the fact that students in some teachers' classes learn more and better than students in other classes. That might be a supervision problem, but it might be that the teacher with higher achieving students has learned superior teaching strategies or has a better approach than other teachers. And I am not shying away from the word "better." The teachers with lower achieving students are not bad people or wrong for trying, but they are in a position to accept responsibility for the differences between the two classes and make an attempt to improve. And it seems to me that you are suggesting that either the teachers should not be held accountable for their need to improve, or the only group responsible for the students in some classes not learning as well as others are the students themselves.

Bill, I would add something to the list of three reasons of why this ain't easy. You quoted Bob's "student learning rates," but you left out the adjective before that phrase that also presents problems: "increased." Increased by how much?

A few years ago I had a conversation with some fellows in DC who had played a role in writing NCLB and were proponents of keeping it in place. I made the argument that high stakes accountability might put in place pressures that would help classrooms where no learning was taking place, but it was creating a problems in the curriculum for many teachers and students (you put it better in point #3). They essentially responded, "Look, the bar has been set too low for too long. We're trying to raise the bar, and if teachers exceed that bar, then great. But we're not trying to make all classrooms great. We're trying to make all classrooms good."

I've thought about that for a long time. It seems to me that everyone involved in this argument wants to see learning improve for poor students and students of color, but we can't agree on what IMPROVE, or INCREASE, or GOOD mean. For some people, it might mean that at least all students graduate with basic math, literacy and science skills. And for others, that's not a good enough end game. And for people in the latter camp, like myself, there is a concern that if we agree upon the wrong goal for improvement we are going to end up hurting a huge portion of the student population in the process (similar to point #3).

Finally, part of the solution to this debate about merit pay has been put into practice in many schools. Most people recognize that merit pay based on a single test would be totally problematic, and so it is often based on a variety of assessments, including observations, teacher-produced assessments, as well as standardized tests. I think this goes a long way towards addressing your concerns. At the same time, I am sure that in some places an inordinate amount of weight is placed on the standardized tests, and that would definitely be my concern if an entire school district or state was considering adopting the practice.

[Matt Johnston]

Bill,

As usual, very provacative. A brief analogy. In my role as a lawyer, when examing a piece of legislation or a contract, often the most important section is the definitions section, the part that almost everyone bypasses, but often enforcement of a law or a contract will turn on the definition of a specific term.

In the "education accountability" world, those definitions must include what will be measured, how it will be measured, when it will be measured, what the standard of success shall be and most importantly, what are the sanctions for failing to meet the standard. Of course, this is nothing new and as you point out, it is often left out when people talk about "improving school outscomes" or "raising achievement" or even "closing the achievement gap."

But my defintions diatribe aside, I would like to address one of your points. You said:
"Ask ten parents what kinds of outcomes they most value from schools and you're likely to get ten different answers.

Some will care most about academic achievement measured by standardized tests. Others will care more about seeing their students learn life skills like how to collaborate. Society expects schools to prepare students to participate in our democracy. Public health organizations want us to address challenges ranging from childhood obesity to teenage pregnancy. "

This is really student learning rates, but it does illustrate a slightly different defintional problem. We as a society have allowed our schools to become not institutions of learning, but institutions of "social change." That is how we get concern for obesity, teeenage pregnancy, global warming, poverty, violence and other social ills taking place in our schools. This is not to say that these issues are not important and need to be addressed to one extent or another, but are these really "educational" problems? In short our policymakers, by intent, deed or omission of their duties have allowed the schools to become a poaching ground for whatever societal ill they or some interest group feels a need to be addressed.

Thus the definitioal problem we have first and foremost is not one of student learning rates or even accountability, but simply the definitional problem at education's core, namely what is the MISSION of our schools?

You mentioned that the modern state curricula has some 200+ standards that have to be taught. I think that may actually be an underestimation in some states, but it suffices for illustrative purposes. If you were to look at those standards, how many of them relate to these "social" matters that really ought not to be part of a core curriculum? If the curriculum is too crowded, then the most logical step to take is to reduce the number of standards. But really, how can that happen?

While I understand the need to hold teachers accountable for their successes with students actual learning, and I too could never be mistaken for one who doesn't believe in accountability or differential pay for performance. However, until we as society and the educational policymakers in our midst can get a grip on what education SHOULD be, we will never find a real good answer to any question regarding student learning rates or teacher accountibility.

[Bob Heiny]

Thanks, Bill, for holding the comments section open. Great post and thoughtful comments by your readers.

All teachers know that educators have addressed learning rates in various formal ways since the Simon-Binet test developed over 100 years ago in France. We know more sophisticated ways to measure these rates systematically, reliably and with measured degrees of confidence, including standardized achievement tests. The key to external validity here is the standardization process, which has a relatively high correspondence to classroom learning. We also know how to increase individual learning rates measurably.

I'd like to offer three points about increasing learning rates promptly (probably) in any classroom. I've watched many good teachers follow these steps to help students increase their learning rates:

1. Focus learning activities (in whatever content you offer) by basing them on known principles of how people learn, such as going from known to unknown, easy to hard, simple to complex, etc. These principles address part of Matt's concern about adjusting curricula to a managable size.

2. Count something related to that learning and write it down (better yet, ask each student to write their count down) promptly, anything to start until counting becomes a habit, as with professional musicians and dancers. If the teacher doesn't count it, it doesn't count to the learner either! I count in clock-seconds, sometimes minutes and trial blocks. These yield three sets of rates during the ABCs of learning: (i.e., Ante-Behavior, Behavior, and Consequences to that Behavior).

3. Compare counts at least daily as you judge appropriate across time intervals, class periods, content intensity, etc. Use these counts to adjust lessons on the fly as well as for the future.

Interestingly, such a simple task as counting can provide a teacher with targeted information to use to increase learning rates, irrespective of what policy, number of standards, etc. exist in any school.

And a rock-on back to you, Bill!