You can’t argue against being data-driven. Or the word accountability. They’re inarguable, above reproach, Teflon. So I’m trying to be data-driven. I have major problems with high-stakes testing, but when my school’s disaggregated score reports for last springs state exams were released this month, I studied them with interest. The whole faculty, actually, spent a meeting period poring over the data, and coming up with observations and action plans. Which standards did the kids really get? Which ones are they missing?
10.IT-DP.7. Analyze the presentation of information.
Example: Students attempt to follow the directions to use a database program on the computer. Afterwards, they evaluate the directions for clarity and ease of use.
Okay, so we’ve got that down. They really know “presentations of information.” But then I looked closer at the score report and saw that there was one other question on the exam that tested standard 10.IT-DP.7. On that question— on the same standard— only twenty-point-five percent of my school’s kids got it correct. It was their lowest-scoring question on the whole exam.
Madness!
I thought they had nailed analyzing presentations of information. My group of teachers requested to see the test questions, since clearly this standard is taught at varying levels.
No such luck. The tests are “not released.” Nobody can see a DC CAS test to find out which questions actually tripped up the students and why. This makes no educational sense. The tests are taken in April, the results are viewed in September, and the data is contradictory.
These high-stakes exams determine “adequate yearly progress” and hold great influence over schools and students. But they’re not the way to be data-driven.
In the second week of school, I gave my 11th and 12th grade students practice SAT II exams. Juniors took a writing test and seniors took a literature test. I wrangled my entire family (my wife, my parents, my sister, her boyfriend) into helping me tabulate the results. I saw a bunch of gaps in my students’ abilities to answer many of these kinds of questions.
For example, question #1 on the writing test was on subject-verb agreement. They bombed it. Subject-verb agreement was my first “grammar mini-lesson” that I’ve incorporated into their literature unit. I’m going to hammer on subject-verb agreement all year.
Lots of teachers at my school use these kinds of mini-assessments to much better effect than I do. The most useful data comes from in-class activities and small assessments with quick turnaround— not endlessly hyped three-hour extravaganzas with months and months of lag time between Test Week and learning the results.
Media outlets and politicians who thrive on climactic press releases and mass score reports will never understand this, because it defuses the big bombshells on which they thrive. I don’t expect the multi-million dollar test-making and test-grading industry to agree with me either. (For a wild inside glimpse at that dysfunctional world, check out Todd Farley’s highly enjoyable new book, Making the Grades: My Misadventures in the Standardized Testing Industry.)
I don’t like feeling that my classroom is starting to make sense while the larger school system spirals into data madness. I’m not sure how to stop it.
Thanks for a very thoughtful explanation of how "data-driven improvement" falls flat in the absence of good formative assessments and policies to support teachers' use of data.
To their credit, people in the Department seem to understand the need for much better formative assessment to support data-driven instruction. It will, of course, be no easy task to shift assessment regimes to support a better vision.
Posted by: Claus | 09/30/2009 at 06:29 AM