"Now that the Data Quality Campaign has put data quality on the map, we need to work together to leverage this work and push it to the next level by using data to drive reform. The path to real reform begins with the truth - and we must keep facing the truth and finding the answers until every classroom has a great teacher, and every child has an education that prepares him for college, for work, and for life." Secretary of Education Arne Duncan, Data Quality Campaign Forum.
What is Secretary Duncan using as the basis for this truth he says we must face? What does he think constitutes data that is evidence of the truth about education to prepare children for college, work and life? We have been immersed in data-driven instruction and decision-making. Some schools are now wholly devoted to producing good data, defined by standardized test scores. Students’ success or failure depends on test performance, with perhaps a nod to graduation or attendance rates. Secretary Duncan calls for "data driven school reform"--while President Obama acknowledges that using standardized test scores as the measure of good or bad teaching creates other problems.
These kinds of public statements demand a definition of specific outcomes aligned with instruments that gather valid, reliable and relevant data--all kinds of evidence of student progress toward agreed-upon outcomes. Those of us working in the field now have an opportunity to offer advice and ideas based on our real work experience--because as of this moment, we are still committed to standardized tests.
Here's a story about a school I worked in many years ago. Their test scores were very high. One year, all 3rd graders got a perfect score in reading, but the numbers for writing were weaker. In the following years, the district used this information to launch a required school-wide focus on writing. We gave the kids the message that they were weak in writing, and needed to improve. We also paid for a few released test items ($7 for one student’s response to one item), to analyze these weaknesses. We realized that the students were all proficient and advanced on the actual writing samples. It was the multiple-choice questions, where they had to choose an answer with no context, where they lost points. What did this tell us about the data we had used to “drive” our instruction for several years? They knew how to apply the rules to actual writing, but their actual “weakness” was choosing between tricky multiple-choice answers on isolated rules for writing. Should we spend less time teaching them to be writers, in order to work on testing strategies for multiple choice? Should we drill them on memorizing the rules? What would this do for them, in their lives as writers? What effect would this kind of teaching have on their skills and motivation for real writing?
I have now been working for two years in a school that is based on a big vision for every students' active participation and positive contribution in the 21st century world. Measures on standardized tests are inadequate to assess the internalized habits, dispositions, skills and knowledge that we see our students developing: capacity for leadership, creativity in overcoming constraints, active willingness to help others, issue analysis using multiple perspectives, belief in their own responsibility and agency, ability to build a case or analyze another's argument. It is not easy or simple, nor could it be standardized to hold teachers and students who were not committed to this particular vision "accountable."
But I have seen students challenged and inspired to work hard, investigate, create, strive for greatness in all subjects--and still not consistently score well on a test. I feel a very deep responsibility for these children’s growth and well-being. After ten years of accepting accountability for test scores, with increasingly narrowed results for kids, I now feel passionate about communicating the truth of what it means for our students’ future prospects if we continue to rely on standardized tests as the measure and definition of success. We need to learn from our experiences with the limits of standardized tests. We must develop strong alternatives for gathering evidence of progress, aligned with a clear vision of who our students will become, what they need to know and be able to do as a result of their years in school.
As an educator with a background in
qualitative assessment, it breaks my heart to see children’s
potential reduced to test scores, their days in schools reduced to test
preparation, teaching reduced to scripted lessons and canned curricula. The
truth about data? I'm frustrated, and it's not because I do not believe in
accountability, rigor or quality teaching-- but precisely because I do.
Image: TEST/Sidelong/Creative Commons