The Tempered Radical

Value-Added Teacher Evaluation Models Fail Kids and Communities

Anyone that has spent any length of time following the Radical knows that the test-happy culture that we're swimming in rubs me the wrong way.  It's simultaneously stripping the joy out of learning and driving good teachers out of the classroom all in a thinly-veiled attempt to declare war on teachers.

What frustrates me the most about today's accountability culture has always been the suggestion that measuring student performance on content-driven multiple choice tests given once per year is the best way to measure the "value that teachers add" to their kids, their schools and their communities. 

That's ALWAYS sat wrong with me -- and it should sit wrong with YOU too.  

Teachers do SO much more for our kids than deliver content. We coach and we encourage and we inspire.  We are cheerleaders.  Kids take risks while they are in our classes that they may never have even considered had it not been for the fact that they knew we cared.

But in a world where tests are created with the sole intention of ranking and sorting our teachers, those intangibles -- those small acts of kindness and care that are often the things that keep kids moving forward -- are dismissed, pushed aside by politicians who argue that the only value that matters rests in the results that we can measure with multiple choice exams.

My rants against testing have always felt half-baked to me, though, because I could never back up my assertions with a healthy dose of research -- the Turkish Delight of the #edpolicy kingdom.

Until now, that is. 

You see, a new study from Kirabo Jackson of Northwestern University FINALLY supports my hunch that our single-minded determination to assess and evaluate the cognitive impacts that teachers have on students is flawed policy at best -- and failing our children at worst. 

What Jackson -- who was interested in determining the impact that cognitive and non-cognitive behaviors have on the future success of students -- discovered should force every parent, politician and policymaker who cares about schools to rethink the role that testing plays in our buildings. 

First, he discovered that non-cognitive skills and abilities like motivation, determination, and self-restraint are a better predictor of future success -- particularly for struggling students -- than the cognitive skills measured by standardized tests.

Perhaps more importantly, he discovered that SOME teachers are good at increasing the cognitive skills of their students and OTHER teachers are good at increasing the non-cognitive skills of their students, but MOST teachers AREN'T good at increasing BOTH cognitive and non-cognitive skills.

This has HUGE implications, doesn't it? 

If Jackson's research and methodology is sound, current educational policies -- which prioritize test scores when measuring a teacher's value -- are incentivizing the WRONG behaviors. 

Our students are the ultimate losers in environments that encourage teachers to push non-cognitive skills and abilities -- skills and abilities that Jackson argues are actually BETTER indicators of future success -- to the sidelines in response to the cognitive-first realities they are working in.

What politicians who are pushing the testing culture in our schools should REALLY worry about, though, is that educational policies designed to reward teachers for producing results on standardized tests are actually wasting taxpayer cash. 

As Jackson explains:

"Teacher effects on test scores and teacher effects on non-cognitive ability are weakly correlated such that many teachers in the top of test score value-added distribution will also be among the bottom of teachers at improving non-cognitive skills.

This means that a large share of teachers thought to be highly effective based on test score performance will be no better than the average teacher at improving college-going or wages."

And:

"Because variability in outcomes associated with individual teachers that is unexplained by test scores is not just noise, but is systematically associated with their ability to improve typically unmeasured non-cognitive skills, classifying teachers based on their test score value added will likely lead to large shares of excellent teachers being deemed poor and vice versa."

Just stew in that for a minute, will you?

We are literally spending MILLIONS of dollars  -- not to mention MILLIONS of instructional hours -- every year measuring the wrong skills and rewarding the wrong teachers!  Sure, mastering the content covered in the K12 curricula matters, but according to Jackson, mastering non-cognitive skills -- skills that we currently do nothing to assess -- matters more.

No joke, y'all:  It's time that you start asking your policymakers some difficult questions about their positions on value-added measures of teacher performance.

If Jackson is right, those policies -- which have rapidly become the norm instead of the exception in most states in America -- are wasting our time AND our money.

____________________________

Related Radical Reads:

Nate Silver on Using Test Scores to Evaluate Teachers

Three Things Parents and Policymakers Need to Know about Merit Pay in Education

What Economists Don't Understand about Educators

 

Nate Silver on Using Test Scores to Evaluate Teachers [SLIDE]

One of the hottest statistical minds in the world right now is Nate Silver, the mind behind the FiveThirtyEight blog which is dedicated to helping readers to "cut through the clutter of this data-rich world." 

Silver rose to prominence initially by using algorithms to correctly predict the winner of 49 out of 50 states in the 2008 presidential election.  He's gone on to become one of the most respected voices on the use of data to drive decisions.

Yesterday, Silver participated in an online IAmA Blogger Q+A session over at Reddit -- and the first question that he was asked was whether or not he believed that using standardized test scores to determine a teacher's value was a responsible decision.

Silver's response was worth sharing with every parent and educational policymaker you know:

 

Download Slide_SilveronTesting

 

Now to be fair, Silver DOES say that he'd need to give the topic a much longer look before he'd be able to make any real decisions on using test scores to evaluate teachers. 

But his initial hunch that using test scores as the most important measure of a teacher's effectiveness is an example of using data badly aligns nicely with the thinking of other really bright minds working BEYOND education -- including Bill Gates (see here and here), Dan Pink (see here) and Chip and Dan Heath (see here).

And as Liz Dwyer points out on the Good Education blog, Silver's hunch on testing as a tool for holding teachers accountable also aligns nicely with REAMS of studies already completed by everyone from The National Research Council and the president of Math for America to respected senior scholars like Richard Rothstein.

Finally, Silver's initial hunch ALSO aligns nicely with the opinions of the very companies that make the standardized tests that are strangling education in our country, who have noted time-and-again that their products were never designed to be used to draw conclusions about individual teachers. 

So if there is a broad consensus starting to form that using test scores to evaluate teachers is a bad idea, then why do policymakers -- including President Obama and his Education Secretary, Arne Duncan -- keep pushing for these failed policies?

That's a question that every person you vote for should be forced to answer.

_______________________________

 

Original Image Credit: American statistician Nate Silver at SXSW in Austin, Texas USA on March 15, 2009 by Randy Stewart -- Licensed Creative Commons Attribution Share-Alike on January 10, 2012

Quote from: Data King Nate Silver Isn't Sold on Evaluating Teachers With Test Scores

 

 

 

Are Current #edpolicies Turning Schools into "Brainpower Wastelands"?

Quick question:  How many times have you heard a politician claim that "schools just need to learn to do more with less" in the past few years?

Bajillions, right? 

The simple truth is that most communities are flat broke, and EVERYONE -- including people working in private sector -- are being asked to do more with less.  Productivity just plain matters in an era when nobody has two nickels to rub together.

And while it might surprise you, I don't have ANY trouble with the suggestion that schools find ways to get more out of the resources that they currently have.

I'm just as likely to think conservatively as the next guy, and the way I see it, if I have to find ways to stick to a budget and to stretch a buck, governments should do the exact same thing.

More importantly, I'm CONVINCED that ANY organization -- particularly those that rely on human capacity to drive growth -- CAN get more out of the resources that they currently have at their disposal as long as they take the right steps.

Need proof? 

Then pick up Multipliers: How the Best Leaders Make Everyone Smarter by Liz Wiseman and Greg McKeown.

After spending years studying leadership in some of the world's most profitable companies, Wiseman and McKeown come to a startling conclusion:  Multipliers -- the term that Wiseman and McKeown give to successful leaders -- can get TWICE as much out of their employees as their peers.

Think about the implications of that fact, y'all.  Imagine being able to DOUBLE the productivity of schools without adding ANY resources at all. 

#winning

Simple Truth: Collective Strength Matters MORE than Individual Talent [SLIDE]

Oneof the guys that changes my thinking more than most is Paul Cancellieri -- seventh grade science teacher and the mind behind Scripted Spontaneity. 

Paul and I have talked a TON over the years about the mistaken belief that we can improve schools by doing nothing more than filling every classroom with a talented individual.  While talent certainly matters, it's just not enough to drive change in complex human organizations like schools.

Instead, driving change in complex human organizations like schools depends on building high-functioning teams that can support one another -- bringing complementary skills to bear against shared challenges:

(click to enlarge)

Download PPT Version

 

So the question -- whether you are a principal, practitioner, parent or policymaker -- is simple: 

What are YOU doing to ensure that the teachers who serve the students that you care about are working on teams that are collectively strong?

___________________________

Related Radical Reads:

Three Things Every Parent and Politician Needs to Know about Merit Pay

These are OUR Kids [SLIDE]

What Can Educational Policymakers Learn from Amazonian Explorers?

Are YOUR Learning Teams Playing Together?

 

 

Original Image Credit: Band of Brothers by The US Army

Licensed Creative Commons Attribution on October 21, 2012

Three Things Every Parent and Politician Needs to Know about Merit Pay in Education

On the drive home today, I tuned in to the local talk radio station because the host was interviewing a TON of local and state political candidates and I wanted to learn more about their individual positions. 

The interviews were incredibly interesting --- especially when they turned to education simply because the host seemed to ask every candidate how they felt about merit pay for teachers. 

Considering how important the issue has become in conversations about the future of education, I decided to whip up a list of three things that I wish every parent and politician knew about merit pay.

Here they are:

Experts in fields BEYOND education are convinced that merit pay plans WON'T improve schools.

The simple truth about merit-based compensation plans is that they are ONLY effective in fields that require "rudimentary cognitive skills." 

In fact, research has shown that compensation plans that tie pay to performance may actually have a NEGATIVE impact in knowledge-driven fields like education.

Need proof? 

Then check out this RSA Animate video sharing the ideas of Daniel Pink -- a one-time supporter of merit pay plans in education whose personal opinion changed after studying nearly every independent research report on the impact of incentives on human motivation. 

Or this bit detailing the thinking of Chip and Dan Heath, who argue that our constant attempts to design a better carrot result in a tendency to focus on ONE variable -- which might successfully drive change in simple fields, but are destined to fail in work that is complex and dependent on multiple variables.

 

Most merit pay plans are generally built around high-stakes standardized tests that almost NEVER actually measure anything we REALLY want our students to know and be able to do.

In almost every conversation that I have with stakeholders about the skills that students need in order to succeed in tomorrow's workplace, people -- including employers in almost every field -- argue that kids need to be able to communicate, to collaborate, and to solve problems creatively. 

Sounds about right, doesn't it? 

If YOUR child left school having mastered those three skills, you'd be happy and they'd be successful no matter what profession -- or personal passion -- they decided to pursue.

Here's the thing:  Today's high stakes standardized end-of-grade exams -- which are almost ALWAYS used as the basis for deciding which teachers deserve merit pay bonuses -- don't measure ANY of those skills.  Instead, they measure nothing more than a student's ability to remember and regurgitate facts.

Which means that the merit pay plans that wonks like to tout as silver bullets for motivating lazy teachers end up simultaneously stripping classrooms of the kinds of higher-order skills that we claim to care about. 

Need proof?  Then look at what years of teaching in a tested classroom did to my instruction.

 

If you want to incentivize something, incentivize collaboration.

Something incredibly powerful is happening in my professional life this year:  I'm working on a learning team with two young teachers who are literally changing the way that I teach.  Both are skilled at developing hands-on, inquiry-based science experiences for kids.

That's an area of REAL weakness for me even if I AM certified to teach middle school science!

What's cool, though, is I'm having JUST as a significant impact on my colleagues. 

I bring a bunch of nonfiction reading experience to our collective planning table -- I have spent the better part of my career in language arts classrooms, after all -- and that's helping my peers, who AREN'T trained reading teachers.

That means collaboration is paying off for ALL of the kids on our hallway. 

My students are benefiting from the inquiry-based strengths of my peers and their students are benefiting from my experience as a reading teacher. 

That's beautiful, right?

But here's the thing:  That collaboration will DIE in a merit-based workplace.

Think about it:  Would YOU share your best practices with struggling colleagues if you knew that there was a limited pot of money that was going to be given to a small handful of teachers that produced the best scores? 

HECK NO.  You'd let THEM -- and more importantly, the KIDS in their classrooms -- fail miserably because it would give YOU a better chance of cashing in.

Which is why the report on redesigning professional compensation for educators that I wrote with 18 accomplished teachers a few years ago recommends incentivizing collaboration INSTEAD of individual performance.

If we believe that SOME teachers have discovered the best strategies for teaching essential knowledge and skills to our kids, why wouldn't we design incentive systems that encourage SHARING rather than COMPETITION between peers?

 

I get it, y'all.  There ARE real problems in our educational system that need to be addressed.  Change IS necessary. The uncomfortable truth that we've been all-too-willing to overlook is that our current approach to schooling just ain't working for a TON of our students.

But as a teacher AND a taxpayer, I cringe whenever I hear someone argue on behalf of merit pay programs simply because they WON'T improve schools. 

Instead, they're just another #edpolicy disaster that the kids in our classrooms will end up paying for.

 

What Economists Don't Understand About Educators

In yet another example of economists churning out crappy research that will have a negative impact on #edupolicy, Harvard professor Roland Fryer and Freakonomics co-author and University of Chicago professor Steven Levitt recently released a study touting the #edupower of "loss aversion" merit pay programs.

So what exactly do "loss aversion" merit pay programs look like in action?

Essentially, participating teachers are given a bonus at the BEGINNING of a school year -- in Fryer and Levitt's study, $4,000 -- and then told that they'll have to GIVE BACK monies if their students don't meet and/or exceed expectations on standardized tests given at the end of the school year.

Now THAT is nothing short of pure #edubrilliance, isn't it? 

Hit 'em with a little carrot and a little stick and maybe those lazy teachers will pick up the slack and finally start performing.  The same approach was VERY successful, Fryer and Levitt point out, at improving the productivity of CHINESE FACTORY WORKERS in the ONLY other setting where the benefits of loss-aversion merit pay programs were studied.

#sheesh

#teachingAINTfactorywork

Outside of the obvious methodological flaws in this heaping pile of #edutrash -- the authors themselves admit that unravelling the impact different members of teaching teams have on the performance of the individual students that they share was impossible -- Fryer and Levitt fail to understand a simple #edutruth about teachers:

External incentives -- no matter WHEN they are awarded -- are ineffective in education because we're ALREADY working as hard as we can to do right by our kids.

As my good friend Rick DuFour likes to say, NO ONE in this profession wakes up in the morning thinking, "I'm going to do a half-assed job this morning because I'm just not being paid enough to work any harder than that." 

Teachers are driven by the desire to see our students succeed -- and while our practices may need polishing, to assume that a few thousand bucks might FINALLY force us to give our all is nothing short of #eduignorance. 

Seriously.  Let's think about this for a minute:  Are we REALLY convinced that folks who have willingly chosen a career in the classroom are holding something back out of spite over their salaries knowing full-well that holding back has life-altering consequences for kids? 

The problem in education isn't long lines of pathetic teachers who need a good kick in the pants, y'all. The problem in education is long lines of teachers who are working in dysfunctional, underfunded systems that incentivize irresponsible practices.

And to put it bluntly, we're just plain #eduscrewed as long as we are willing to allow the narrow, pessimistic view of human behavior held by economists to drive the most important choices that we make about what happens in our classrooms.

#whew

#gotTHAToffmychest

_______________________________________

Really Long List of Related Radical Reads Which I Invite Economists to Peruse Before Wasting Anymore Time Studying Merit Pay Programs:

Note to Arne: Cash Incentives Never Work

Need MORE Proof that Incentive Programs are Dangerous?

The Monster You've Created

Bulldozing the Forests

The Truth About Teacher Merit Pay Plans

Merit Pay Plans for Teachers are a Poor Idea

The Unintended Consequences of Rewards Programs in Schools

Tired of Being the Nation's Punching Bag

What CAN Schools Learn from Hero Ball?

Why I NEVER Recommend Teaching as a Profession Anymore

Why I NEVER Recommend Teaching as a Profession

I came to pretty startling realization this week, y'all:  If I had it to do all over again, I'm pretty darn sure that I wouldn't choose teaching as a profession.

Don't get me wrong: I still love working with students. 

#alwayshave

#alwayswill

Knowing that each day brings opportunities to have an impact on lives really IS incredibly rewarding -- and every time one of the kids that I taught risks detention by sneaking out of the cafeteria to come and say hello, I remember that the work I'm doing matters.

But the sad truth is that #edpolicy decisions -- the decisions that govern every aspect of my work -- are increasingly made by politically driven nitwits who almost never have a clue about what really works in education. 

Take the recent proposal by North Carolina Senate leader Phil Berger to end teacher tenure and to institute a merit pay program based on standardized test scores as an example.

Sounds great in theory, right? 

How could anyone argue with a law that establishes formal rewards and punishments for teachers?  Accountability matters, the thinking goes, and there's been too little accountability in education for far too long.  Berger says it this way:

"Until we get the policy right, I don’t think the taxpayers of this state are prepared or should be asked to put more money into the public schools than we are presently."

Read more here: http://www.newsobserver.com/2012/04/23/2020962/senate-leader-berger-proposes.html#storylink=cpy

Here's the hitch in Ol' Phil's argument:  He's not even CLOSE to getting the policy right.

Research has proven that merit pay schemes like the one that Phil is proposing actually lead to WORSE performance in any field that requires anything more than "rudimentary cognitive skill," a point that Daniel Pink makes time and again in Drive -- his book on the economics of motivation -- and in this RSA Animate talk.

But research doesn't matter to legislators like Berger, y'all.  Getting reelected does.

And getting people to the polls with sexy legislation that feeds into the myths that voters like to believe about what works and what doesn't -- something that Pink calls "American folklore" -- is WAY easier and WAY cheaper than building support for effective policies built on a thorough understanding of the characteristics of quality learning spaces and experiences.

Stew in THAT for a minute, would you?

We are literally living in a world where the primary motivation behind the proposals that determine what schools look like ISN'T designing the kinds of learning environments that our kids deserve.  Instead, the primary motivation behind the proposals that determine what schools look like is designing the kinds of learning environments that might get a legislator reelected. 

#perverse

But we are also living in a world that's unlikely to change anytime soon. 

The kinds of policies that would really improve our schools will cost far more -- particularly in the form of providing teachers with more time and/or fewer students to work with -- than most Americans are willing to pay.  What's more, the kinds of policies that would really improve our schools aren't quick fixes -- and with the next election right around the corner, legislators ain't got time for patience.

Which all makes teaching a pretty discouraging profession. 

Those of us who stay will struggle to make crappy policies work -- and STILL end up on the bottom of the political dog-pile when they don't.  Worse yet, we'll be called lazy by the very people whose crappy policies created all of the problems in education to begin with.

Does that sound like the kind of work environment that YOU'D recommend to anyone?

___________________________________

Related Radical Reads:

Note to Arne: Cash Incentives NEVER Work

Need MORE Proof that Incentives are Dangerous?

The Monster You've Created

Merit Pay for Teachers Are a Bad Idea

 

 

Need MORE Proof that Incentive Programs are Dangerous?

If you've read the Radical for any length of time, you'll know that I pretty much despise the carrot-and-stick approach to managing teachers and students that seems to be all the rage in education right now (see here, here and here).

What's completely wild is that many of the top organizational thinkers who are working largely in fields beyond education -- including Authorspeak Keynote presenter Daniel Pink -- agree that incentive programs almost NEVER work.

Download Slide_ThePerfectCarrot

 

Chip and Dan Heath -- authors of a TON of great reads including The Myth of the Garage, a completely free Corwin eBook -- are no less critical of our fascination with incentive programs than Pink.

In Myth of the Garage, they compare incentive programs to the popular urban legend of a Darwin Award winner who strapped a rocket engine to his Chevy Impala looking for the ultimate joy ride only to end up "as a human flapjack" on the side of a lonely Arizona mountain.

The Heaths write:

Incentives are like that jet engine.  There's no question the engine will take you somewhere, fast, but it's not always clear where.  Or who you're going to mow down on the way.

Yet incentives are still the first resort of most managers.  We all think we're smart enough to create the perfect carrot.

(Kindle locations 357-363)

So what makes incentive programs so dangerous -- particularly in fields like education?

According to Chip and Dan Heath, incentive programs inevitably cause well-intentioned people to fall victim to a focusing illusion.  Instead of taking multiple measures into account when judging performance, we overemphasize the single variable that we are attempting to measure.

In other words, when all you are worried about measuring is how fast your car can fly down a dry desert lakebed, you tend to forget about the mountain that you're sprinting towards at 300 miles per hour. 

The Heaths write:

"To be fair, there are some contexts where one variable dominates.  If you're employing a field sales rep who is selling a simple, self-contained product, then it probably makes sense to tie incentives to the sale.

If you're traveling a long, straight road, the jet engine will get you there faster.

But chances are you don't live in a one-variable world.  In your complicated, squishy, matrixed world, if you're dreaming up an incentive plan, you're almost certainly in the grips of a focusing illusion."

(Kindle locations 386-391)

Now, I don't know about you, but I can't think of a MORE "complicated, squishy, matrixed world" than education. 

The honest truth is that we -- teachers, principals, parents, policymakers, community leaders, students -- DO care about multiple variables when looking at the outcomes of education.

Sure we want students to become better readers and writers.  Sure, mathematical competency is an essential outcome for every student. 

But in an era when incentives -- and pretty darn serious consequences -- for both teachers and students have been tied to just two testable variables, I've GOT to believe that we're in the grips of one seriously wicked focusing illusion too. 

My only hope is that we'll come to our senses and switch off the rocket before we run out of road.

_______________________________________

Related Radical Reads:

Bulldozing the Forests

The Monster You've Created

Statistically Snookered

The Unintended Consequence of Incentive Programs in Schools

 

 

 

Original Image Credit: Minty Python’s Frying Circuits 5/52 by Neal

Licensed Creative Commons Attribution on December 23, 2011

 

 

 

Note to Arne: Cash Incentives NEVER Work

So I don't have a ton of time to write today -- I'm sitting in an airport on the way home from Solution Tree's Authorspeak conference and am about to board a flight home to my family -- but I had to get something off of my chest:

I'm sick of simplistic #edpolicies built around the idea that teachers need cash incentives in order to produce.

The fact of the matter -- as Daniel Pink argued convincingly in his Tuesday morning keynote presentation -- is that the notion that cash incentives work in creative professions like education is nothing short of "American folklore."

We like to believe it, right?

I mean, #jeez, we were literally RAISED on cash incentives.

My mom and dad called it an allowance!

But just because we like to believe something doesn't make it true -- and the truth as reported in study after study is that for anything that requires more than "rudimentary cognitive ability" cash incentives are ineffective and may actually lead to a poorer performance than in situations where incentives were never offered.

Check out this RSA Animate video to learn more:

 

So if this is all true -- and I believe it is -- then why in the WORLD is our federal education policy built around introducing cash rewards and merit pay into education?  Isn't that just ANOTHER example of our tax dollars going to waste?

More importantly, if this is all true, why in the WORLD are we tolerating school-based behavior incentive programs like PBIS?  Isn't it time that we start to rethink the kinds of "incentives" that we're offering to teachers and students?

#justsayin

___________________________________

Related Radical Reads:

Performance Pay Will KILL Our Schools

The TRUTH About Teacher Merit Pay Plans

The Unintended Consequences of Reward Programs in Schools

 

 

The Truth About Teacher Merit Pay Plans

As I explained more than once, I’m a firm believer that the way we pay teachers has got to change.  Our “give ‘em the same paycheck no matter who they are or where they are working” approach is simply ridiculous. 

But please understand that my support for changing the way we pay teachers doesn’t mean that I’m in favor of the ridiculously simple “pay ‘em for test scores” plans that Bam, Arne, Bill and Michelle believe in.

In fact, I’d go as far as to say that teacher merit pay plans built around standardized test results are the WORST #edpolicies that we’ve seen in my 18 years of teaching—and that includes anything that Curious George dreamed up between starting wars and clinking scotch glasses at Haliburton shindigs.

Here’s why:

Less than 50% of teachers are working in tested subjects. 

I’m constantly amazed by merit pay plans built around standardized test scores simply because so few educators are working in tested subjects.

In my North Carolina middle school, for example, kids are tested in language arts, math and eighth grade science. 

There’s no tests for ANYONE else.