Gates' Teacher Effectiveness Study: Surprised?

I was invited by National Journal.com, Education Experts blog to share my reaction to the final report of the MET study. Here's what editor Fawn Johnson asked: What is most surprising about the Gates’ findings? What are the easiest ways teacher evaluations can be tweaked to more accurately reflect effectiveness? How important are student perception surveys? What lies ahead for videotaping teachers’ lessons? Do we need to learn anything more about measuring student achievement? Is the task laid out by Gates too daunting for schools to handle?

I deliberately avoided looking at any of the social media spin on the final report of the Gates Foundation funded Measurements of Effective Teaching (MET) study until after I had done my own reading. I took the same approach to the release of the first report back in December 2010.

Then, as now, there are several things about this study that I admire. Like Fawn Johnson (National Journal.com Education Experts editor), I am impressed with the seriousness and sincerity of the researchers in tackling the complex issue of teacher evaluation, especially since there are too many people who want to oversimplify it. I’m also glad to know the data from this study (unlike some of the earlier studies involving value-added measures) is being made available to the wider research community for independent investigation of results.

Most delightful of all is the MET researchers’ recognition of the importance of student voice in determining the quality of teachers’ work. If we are at all serious about preparing our youth to be critical thinkers and contributing citizens, we must start by listening to what only they can tell us about what is and is not working in our classrooms and schools.

Also, unlike some critics of the study, I reject the complaints about the MET’s inclusion of classroom observations by multiple evaluators as an important way to measure teacher effectiveness.  The research team recommended that those observations should not be over or under represented in the blend of measures used in a teacher evaluation system. Here I’m using my parent lens (my husband and I have raised 11 children and shepherded them all through public school).  There is essential information about a teacher’s effectiveness that no test data can reveal: How does that teacher treat my child? I have known teachers who could boast impressive student test numbers, but disrespected and demeaned their students in the process.

The purpose of teacher evaluation is to answer two questions (not one): How good a job is this teacher doing, AND how can this teacher do better? Candid, objective feedback from outside evaluators and thoughtful reflection by teachers on our work is essential for continuing professional growth.

Teachers submitting video of ourselves teaching for evaluation purposes is not new.  Part of National Board Certification, a voluntary process for advanced teaching credential, requires teachers to not only include video examples, but extensive written analysis by the teacher candidate of his/her work using the video as evidence.

As a National Board Certified Teacher myself, and now as a member of the Board of Directors of the National Board of Professional Teaching Standards, I am gratified that the study confirms what the National Board has known and proven for 25 years: There are significant differences in the quality of instruction provided by teachers, and those differences have critical impact on student achievement and on student learning.  

It was not the purpose of the MET study to distinguish between student achievement and student learning, but their interchangeable use of those terms in the report further confuses the concepts in the public conversation. In 2011, a task force commissioned by NBPTS (which included Robert Linn, Rick Hess, Lloyd Bond, and Lee Shulman) released a report that supplied much-needed clarification:

Student achievement is the status of subject-matter knowledge, understandings, and skills at one point in time.

Student learning is growth in subject-matter knowledge, understandings, and skills over time…It is student learning—not student achievement—that is most relevant to defining and assessing accomplished teaching.

Standardized tests are the instruments we use (for now) to measure student achievement, but there is much, much more that we need to know about measuring student achievement and student learning. As my higher education colleagues and many employers will testify, students meeting an arbitrary state cut score may (or may not) indicate factual recall of certain immediate learning objectives, but the method falls grievously short as a measure of what students actually know and can do after the test. How this scenario will change if, when, and after the “next-generation” assessments promised under the Common Core Standards are implemented remains to be seen. But if all we want from teacher evaluation is a way to identify which teachers are the best bets for raising student test scores, we would be setting a disgustingly low bar indeed.

Implementing teacher evaluation systems with a balance of multiple measures as recommended by the MET study will present significant hurdles to states and school districts, cost being only one of them. However, there are already some promising starts. Consider what these teachers from Garfield High School in Seattle, Washington have to say about the challenges of implementing such a teacher evaluation system. Notably, these teachers have also decided not to give the state-required tests to their students this Spring.

Surprise! Effective teacher evaluation not only distinguishes teachers; it empowers them.

Cross-posted at education.nationaljournal.com

Comments

[anthony]

Renee,
Why do you suppose the authors of this report confuse, or perhaps conflate, student achievement and student learning?

The former is, as you suggest, much easier to quantify. The latter is what we really care about -- but which is much harder to fit into mathematical formulae now used to calculate the value teachers are adding.

So long as this confusion persists, we will be trapped in a circular pursuit, where good teachers are identified by their ability to raise test scores, and we validate other indicators by how well they coincide with this first order definition.

Unfortunately, this has the effect of systematically devaluing all other forms of student learning not captured on the tests.

When we went through National Board certification, we had to provide evidence of student learning in quite broad ways. There were standards related to building classroom community, and our videos were expected to show how we were engaging students in active discussions, and how we were provoking critical thinking and inquiry.

We are not going to improve life outcomes for students through the narrowing of our focus to that which can be measured on tests. The VAM approach that the MET project remains loyal to has been demonstrated to be unstable and unfair, resulting in stigmatization of teachers -- especially those working with Special Education students and English learners.

[TeachMoore]

Anthony,
I think you are exactly right about the continued emphasis on VAM and overuse of standardized testing. In the study, the researchers stressed how they attempted to correct for weaknesses in previous VAM studies by randomizing student assignment. But even they had to admit that they could only accomplish a very limited form of randomization (principals made the rosters, and MET used a computer program to assign them to teachers.) Knowing schools as I do, there was probably much less randomization than the researchers think or acknowledge.

I'm not opposed to standardized testing in general; in fact, I think it's good for students to take them--every once in while (like maybe once or twice in their entire time in school). But we have done incredible and irreparable harm to children and teachers with the abuse of this type of assessment.

[TeachMoore]

CTQ founder and my co-author, Barnett Berry, offers his take on the MET study and quotes from some of our joint work in response to it. http://teachingquality.typepad.com/building_the_profession/2013/01/weighing-and-weighting-the-evidence-the-measures-of-effective-teaching-project.html