Assessment's Either/Or Conundrum. . .

I'm enjoying the give and take of the conversation that has developed in the comment section of my recent blog post on the Danish system of assessing students because it is forcing me to think!  As always, diversity of opinion provides the kind of external challenge necessary for new learning. 

While listening, I've noticed two distinct viewpoints developing.  The first argues that an overemphasis on standardized testing as a form of assessment leaves parents and teachers with a superficial understanding of what it is that a student knows and can do. 

As Simon argues:

The point of education has to be to teach kids to do something that they couldn't do before. That's what learning is. Frankly, it's uninteresting how well a bunch of 14 year-olds can cough up the periodic table or list off the members of the Triple Entente in 1914. That kind of information is easily available when you need it and listing it is proof of memorization, not learning.

What is interesting is what you can do with such information. Analysis, critical thinking, using technical terminology - these are skills, and skills that are very hard to measure in a multiple-choice quiz. They are also fairly easy to judge in a conversation. If it is unclear whether the pupil is parroting or not, just ask another question.

The second argues that teacher observations alone are not always enough to paint a complete picture of a child's strengths and weaknesses.  Standardized tests can, therefore, be a valuable tool in the assessment arsenal, filling in gaps between a teacher's observations and a child's performance. 

As K. Borden writes:

Please remember I come at this from the perspective of what happens when the teachers’ observations fail the student, but the tests open the doors of opportunities.

Thanks to tests (Pre-EOGs, Cog-AT, ITBS, EOGs) a different picture than the one being reported by teacher observation emerged. Those results led to seeking more answers, largely via more tests (WISC, Standford Binet, Woodcock Johnson). And those tests revealed a remarkably able and creative student, hindered by a previously unrecognized handwriting disability. In one year via tests we learned far more about one young learner than teachers’ observations ever yielded.

So bash away at those ovals and that data. Meanwhile, one child and her parents thank them.

As both K and Simon note later in the comment conversation, teacher observations and standardized tests should each play a role in a well-rounded system for assessing our students.  Both recognize that assessment should never be an either/or proposition.  Instead, we should get our hands on as much information as possible when trying to diagnose a course of action for our kids.

The problem---and correct me if I'm wrong----is that American schools HAVE made student assessment an either/or proposition.  Teacher observations have become increasingly irrelevant as districts and states try to meet the testing targets set by the No Child Left Behind legislation. 

Let me give you an example of how this shift has played out in my classroom:  Not long ago, I had a parent ask for a conference to learn more about her son's abilities in my language arts classroom.  Having had a great relationship with her boy over the course of the year, I knew his strengths and weaknesses better than I knew the strengths and weaknesses of most of the kids who roll through my classroom, so I was looking forward to our meeting.

During our time together, I went into great detail with this mother, providing writing samples that highlighted strengths and weaknesses, reviewing classroom assessments that had caused struggle, and sharing observations about verbal ability and vocabulary based on countless interactions over the entire year.  I actually felt pretty darn good about the "assessment" that I'd made of my student. 

The first words her mother said when I was finished:  "That's all great, but what does the test say?"

With nine simple words, she'd completely dismissed my professional opinion, opting instead for the cold, hard, seemingly-more-accurate facts.  And when you look at the kind of weight we place on standardized tests as a tool for measuring everything from student performance to school quality, it seems like pushing the professional opinions of classroom teachers to the sideline has become common practice.

So I guess that's why I'm drawn to a system of assessment that places teacher observations first, K.  While I'm all for standardized tests that are used to diagnose student strengths and weaknesses, I think we've moved beyond using standardized tests for diagnosis in our country.  Instead, standardized tests have become our primary tool for evaluation and accountability----of students, of teachers and of schools.

And as Dan Koretz---author of Measuring Up: What Educational Testing Really Tells Us---wrote in this recent interview, using standardized tests for accountability rather than diagnosis is a narrow-minded practice:

Let’s start with test-based accountability, which is perhaps the most pressing issue today.  As both a former schoolteacher and a parent of two children who went through public schools, I am convinced that we need more effective ways to hold educators accountable, and I believe that testing has to be a part of an effective accountability program.

Doing this the way we do in many places now, however — treating one test as a comprehensive indicator of student achievement, pretending that scores taken by themselves are a trustworthy indicator of school quality, and rewarding and punishing teachers and students for scores — is just too simple.

It ignores not only what we know about testing, but also what we know from many other fields, such as healthcare, about the effects of incentive systems. We face an enormous challenge in designing better educational accountability systems, and the first step in doing that is recognizing the limitations of what has been tried to date.

Does any of this resonate with anyone?  Has the proverbial pendulum swung too far in the direction of oval filling as a form of assessment?

Will we ever get back a comfortable---and professional---middle ground where teacher observations are paired with diagnostic tests to paint a clear picture of what our students know and can do?

Comments

[Tom]

Makes sense to me.

Seems like we keep trying to over simplify things. Some things are hard and require a lot of work, time, effort and money to do right. We too often opt not to do the hard work and choose another easier route which creates an illusion of the information what we really want.

I see that on a lot of levels in our society. We create overly simplified models which we pretend are guiding our actions. Basically, if you look at Taleb's The Black Swan you'll see his far better explanation of how these conceptual errors create some really stupid actions and gross errors in over confidence. I think education is headed in the same direction as our banks have because of the reliance on data that is iffy at best.

Even if you believe in standardized testing as an accurate measure you've got to look at how states, counties and schools are doing everything possible to twist and manipulate both the data and the tests. It's a mess.

[K. Borden]

Mr. Ferriter said:
“I think we've moved beyond using standardized tests for diagnosis in our country. Instead, standardized tests have become our primary tool for evaluation and accountability----of students, of teachers and of schools.”

The word diagnosis troubles me when applied to the standardized tests given to students for NCLB compliance. As stated on NC’s End of Grade (EOG) website, “The North Carolina End-of-Grade Tests are designed to measure student performance on the goals, objectives, and grade-level competencies specified in the North Carolina Standard Course of Study.”

A diagnosis seeks to determine causes, or the nature, of a condition via signs and symptoms. These tests measure some of the signs and symptoms, but for individual students, they cannot take the next step and engage in comparing otherwise available information/data or seeking additional data/information to determine causes of existing conditions.

For groups of students these tests may reveal trends, but again they can’t alone answer the cause. For example, is the reason all students in a given class answered question x incorrectly because they never covered the goal, objective or competence as a class or some other reason entirely.

This is where the fault in using them as an exclusive measure to determine teacher performance, school performance and individual student performance rests. Alone they highlight signs or symptoms, but not causes or conditions.

Mr. Ferriter, what was the rest of the story? What was the answer to the parent’s question? "That's all great, but what does the test say?" How far apart were the test results from your assessment of the student?

Does the Danish system use the results of the oral exams administered in 10th grade to determine the performance of the school, the teachers and the student? If so, how do they begin to ferret out which school and which teacher possibly contributed to what result?

When we are talking about what is best for individual student assessment, are we required to link that to what is best to hold schools and teachers accountable for their performance in the exchange that is education? Is it wise to do so? How would a alternative method for measuring teacher and school performance operate?

[Dave]

In consumer studies, there's an idea that a person will be reluctant to buy an item at one store if they think they can get it cheaper somewhere else.

I think this is happening with trust in professionals because of the smaller, more connected world the Internet has enabled. A parent may be reluctant to trust a teacher's professional opinion if they believe it might be possible for them to find and contact a Teacher of the Year and get their opinion.

There's also an element of going with the known evil: parents know that tests don't measure everything, so they feel they can accurately weigh the importance of the test results. It's can be hard for a parent to spot unique quirks a teacher has, which is an obstacle to the parent's trust, so it's harder to weigh the data provided in a teacher opinion.

Trusting the teacher's opinion can be scary: what if the teacher says your student isn't doing well? When it's a test, you can just say it's not measuring the areas the student excels at...but you can't really dismiss or deflect an opinion like that from a good teacher, and that can be a tough pill to swallow.

To get past all of these, teachers really need to be expert communicators who are very in-tune with parents and mindful of whether they present the appearance of a trust-worthy person who has sensible solutions to helping students.