Assessing Learning the Danish Way. . .

As most Radical readers know, I’ve spent the past eight days touring Denmark as a part of a program run by the Center for International Understanding—one of North Carolina’s most important professional development organizations for teachers. 

While I completely enjoyed the opportunities that I had to learn more about the European Union and Denmark’s social welfare system—topics that are a part of the curriculum that I teach—my favorite experience of the entire week was observing two different students go through the only official examination that generations of Danish children have ever taken, which comes at the end of 10th grade. 

  by  ccarlstead 

Kind of geeky, huh?  I mean who spends the better part of 8 days in a 1,000-year old land full of history, cultural geography and really good beer observing tenth grade final exams?!

The thing that drew me to the testing room—besides the fact that exams are public events that can be observed by anyone (including parents and community leaders) who asks for permission ahead of time—was that Denmark’s final exams are probably the most responsible system of student assessment that I’ve ever seen in action.  

Here’s how they go:

1.)  Classroom teachers put together a collection of readings and audio recordings related to content studied during the course of the school year.  In the sessions that I observed—both English exams—there were packets of materials on 20 different topics ranging from racism and role models to terrorism and global warming.

2.)  Also included was a list of related lessons that students had completed on the topic in their regular classes.  These lists included things like articles read, videos watched, and seminars completed.  While the supporting materials for each lesson were not included, these lists allowed students to dig into their background knowledge while attacking new texts.

3.)  Each student arrived at a pre-scheduled time, entered the examination room and randomly selected a number corresponding to one of the predetermined topics.  Then, he/she spent twenty minutes studying the new materials: taking notes, filling out reading guides, listening to recordings, and planning a personal response to the topic that they’d selected. 

4.)  After 20 minutes—a time period that was closely monitored—the student returned to the assessment room where his/her classroom teacher was waiting with a teacher of the same grade level from a school in a nearby town.  The rest of the assessment consisted of an ongoing conversation between the student and the assessors about the topic selected. 

5.)  The assessors carefully listened to each student, looking for evidence of reflective thinking and for the ability to connect new texts to previous materials or experiences.  While students did the majority of the talking during the 20-minute assessment period, both assessors asked prodding questions to challenge students and to test the depth of their knowledge about the topic selected.

6.)  When the students finished working through their thoughts, they were asked to leave the examination room.  Then, the assessors—guided by a predetermined rubric—engaged in a 10-minute conversation with one another to determine the student’s level of mastery. 

7.)  To ensure that final scores weren’t biased, the outside assessor—who had no relationship with the student being tested—took the lead in the conversation and in determining the score given, but both teachers interacted with each other and came to general consensus around each bullet point on the rubric.  If there had been disagreement, the school principal would have been called in to determine a final score.

8.)  Finally, the student returned to the room to receive feedback from both teachers.  Suggestions for future work were offered, compliments were given, and the final score was awarded.

Kind of amazing, huh?  And remember that until recently—Denmark has instituted a much smaller system of standardized testing within the last two years that is designed to give quick feedback to teachers, principals and parents—this was the only “end of grade exam” that Danish kids had to suffer through!

Quite honestly, my Danish friends who teach are completely shocked that we begin giving exams to students in third grade here in the States.  They can’t understand what we think we can learn from tests that we can’t already learn from a teacher’s year-long observation of a student’s performance in the classroom.

Now, I could ramble on for hours about the pros and cons of the Danish system of assessment, but I’m interested in what you think.  Let’s wrestle with this together in the comment section, huh?

Here are some questions to get the conversation started:

1. What can we admire about Denmark’s system of assessing students? 
2. Are there parts of Denmark’s system of assessing students that you’re not sure are as responsible as they seem to be?  Why?
3. What can we admire about our own system of assessing student learning?  (And there has to be something admirable—after all, Denmark has moved a bit in our direction!)
4. What barriers might prevent American teachers from using the kinds of assessment practices that I saw in Denmark?  Are these barriers worth trying to tackle?

I’m really looking forward to reading your thoughts on this one.  I’m afraid I may have swallowed a bunch of Danish Kool-Aid in the last few days and I know that a good conversation will push my thinking deeper. 

I need someone to help me remember that different isn’t always better—or to reinforce for me that in this case, it just might be!

Comments

[TeachMoore]

Welcome back, Bill.
Amazing is right! This deserves closer inspection. For one, is the rubric used for scoring available to the public (and the students) prior to the testing? Is it used in the school system itself? (Can we see it?)

How interesting that the students respond orally rather than in writing or on an "objective" test. I'm particularly impressed that students are given feedback immediately on their performance.

I noted the Danish educators' query about what we hoped to learn from our extensive system of standardized testing that we could not learn from the on-going classroom assessments of teachers over the course of a school year. They apparently place much greater value on the ability of their teachers to assess students given the system they've chosen to use. I'd really like to hear more about this process.

[Cary KIrby]

I have a clarification question. You clearly watched the more "Social Studies" geared exams as that is what you teach. Is the process the same for maths and sciences? Is reading a separate exam or do they feel that in order to pass the other tests, you'd have to pass reading?

I agree with TeachMoore that they clearly have a stronger faith in their teachers' abilities to assess students. I would also imagine, but this could be off-base, that they pay their teachers enough to truly make sure they are all professionals. Most of us reading this blog would fit into that category, but I'm sure all of us have taught next door or down the hall from someone who we'd probably feel iffy about determining kids' futures with no standardized markers.

I don't remember if I read it on this blog or another that what we tend to do in the US is try to treat the symptom. We force everyone down to the lowest common denominator to ensure that the lowest one can succeed rather than figuring out a way to treat the disease, which is poor teaching. I would imagine that in most of the Danish classrooms you visited you probably didn't see tons of scripted curriculum with teachers just "passing out" the material. I see and hear about that a lot around me.

[Simon Oldaker]

I live in Norway, where I certainly find some positive aspects with the Scandinavian way of doing things. We see a tendancy for pupils to score below their test/homework grade on written exams and over on oral exams. To counteract this, my county (Akershus) is moving to make the oral exams more like the Danish ones, with the specific topic given only a short time before the exam.

The point of education has to be to teach kids to do something that they couldn't do before. That's what learning is. Frankly, it's uninteresting how well a bunch of 14 year-olds can cough up the periodic table or list off the members of the Triple Entente in 1914. That kind of information is easily available when you need it and listing it is proof of memorization, not learning. What is interesting is what you can do with such information. Analysis, critical thinking, using technical terminology - these are skills, and skills that are very hard to measure in a multiple-choice quiz. They are also fairly easy to judge in a conversation. If it is unclear whether the pupil is parroting or not, just ask another question.

Foreign languages are an good example. Hyperactive dictionary use, along with technical aids like Google translate can buoy poor pupils through written exams, but a quick conversation gives an excellent and realistic measure of level of acheivement.

@Cary - here in Norway, at least, oral or oral/practical exams exist for all subjects. I believe the same is true for Denmark. For upper secondary education (and gr. 10 in Norway) written exams exist for some subjects in addition.

[Cary KIrby]

I totally agree with Simon's points about the point of an education and the reason to give an exam. Unfortunately, I feel that many of us stateside live in places where we are not trusted to determine whether or not a student is fluent in a subject or not. And as I said in my first comment, I've taught with some people who I certainly don't think were qualified to make such judgements. Of course, it all comes down to money - it always does. These kinds of exams would be quite expensive to administer (hence they only do it once for each student rather than every year). I think it is a better way, but I'm not sure how we move toward that in a society that puts so much value on old paradigms that have proven themselves insufficient.

Simon, out of curiosity, how do they do oral exams for math? I have no doubt it is possible, I'm just curious.

[Russ Goerend]

I think we can admire Denmark's treating of students like learners, instead of oval-filler-outers. The idea of having a conversation with a student to find out what they know is so foreign (excuse the pun) to me as a former K-12 student. It was also never touched upon in my pre-service training. Involving teachers from other districts is crazy-good as well. Those are the two aspects of their assessment that made me think "I don't think any teacher I've ever had would have considered doing that."

I somewhat wish I could find something I didn't find responsible about it, but I really can't. The cynic (American?) in me says, "Where's the data coming from, then?" but I think assessment is more important than data.

I think it's admirable that we still have students who have come from our K-12 schools and want to be teachers. I really mean that. It says so much about our profession that one great teacher can counteract everything that needs fixing in our system.

Barriers? Data-driven systems. High-stakes mindsets. Standardized thinking.

Thanks for sharing your experience, Bill. This was an eye-opening post.

[K. Borden]

Mr. Ferriter:

Please remember I come at this from the perspective of what happens when the teachers’ observations fail the student, but the tests open the doors of opportunities.

Thanks to tests (Pre-EOG’s, Cog-AT, ITBS, EOG’s) a different picture than the one being reported by teacher observation emerged. Those results led to seeking more answers, largely via more tests (WISC, Standford Binet, Woodcock Johnson). And those tests revealed a
remarkably able and creative student, hindered by a previously unrecognized handwriting disability. In one year via tests we learned far more about one young learner than teachers’ observations ever yielded.

So bash away at those ovals and that data. Meanwhile, one child and her parents thank them.

If that rising sixth grader were entering your class, would you want only her former teachers’ observations or would you want those test results as well? Which would serve you best in providing her an opportunity to be an enthusiastic, confident and competent learner?

Would you want for her to wait until 10th grade to be assessed via a process that includes a trained detached observer or would you want to know earlier in her life as a student?

Would you insist she handwrite her five paragraph essays or would you provide her with access to a computer with the spelling and grammar check functions disabled? Which would give you a better idea of what she knows, how she thinks, what she is capable of doing and where she needs improvement? See, I think you would allow her to yield that memory stick with a smile on your face and likely help her discover a world of opportunities at her fingertips, but would your colleagues?

If the question is “whether oral exams consistently conducted, observed by at least one trained detached observer, might yield more information about a student than responses selected from multiple options and indicated by coloring an oval?” The answer from my perspective is that it sure will. However, the occasional standardized tests may just open doors for a less able oral communicator.

At its best, assessment is holistic, comprehensive and varied in its perspective.

You said, “I need someone to help me remember that different isn’t always better”. I don’t know if I accomplished that, but hopefully you see that 10th grade may be too late to know and standardized testing needn’t always be evil incarnate.

[Simon Oldaker]

@ K. Borden. I think you over-simplify. The situation is not standardised tests vs. subjective teacher observation. I am a teacher and my grades are not set on the basis of mere 'observation'. While my pupils do write the odd multiple-choice test, I do not use those results to set final grades, either. Grades are based on what pupils can accomplish. Write a text, engage in a debate, solve a complex problem, etc. These are the skills that they are supposed to learn, and they have products that are gradeable.

You mustn't think that Danish teachers wait 10 years to find out if their pupils know anything, they are merely sceptical about those ovals and trust the competancy of trained professional teachers to evaluate the acheivements of their pupils. Do you think your child's competance is better measured by filling in a bunch of ovals than it would be by, say, writing an essay? Yes, an essay using keyboard, thesaurus, spell-check, etc. Preferably submitted for peer feedback and then submitted in a final version when she was ready?

I don't doubt that standardized tests are useful for diagnostic purposes, but that is something different.

@ Cary I'm not a math teacher and we've just broken up for the summer, so I can't ask my neighbor, but I think it goes something like this:
Two days before the exam, the pupil is informed. They may be given the whole course to prepare, or just certain elements. Shortly before the exam (two hours to thiry minutes) they are given a specific, complex problem (or problems) to solve. They work under supervision, so cheating can be prevented and what aids they use can be controlled. They then present their work. In the 10th grade, they can also be asked to solve simple problems then and there. I think some places have a tradition for testing mental calculation in this way, too.

You can never test full competancy this way, so pupils in such systems always get two grades, one from their teacher and one from their exam. Grades can be formally challenged in most places, and administrators almost everywhere have access to full data so that systematic deviance can be uncovered.

[K. Borden]

Simon said: “@ K. Borden. I think you over-simplify. The situation is not standardised tests vs. subjective teacher observation.”

The insertion of the word “subjective” as a qualifier for “teacher observations” is yours, not mine. Furthermore, I don’t advocate a choice between objective observation by trained professional teachers and standardized tests. I believe there is a place in public education for both as tools and sources of data.

Simon stated: “I don't doubt that standardized tests are useful for diagnostic purposes, but that is something different.”

Standardized tests as administered in public schools do not claim to be diagnostic and are ill suited to the task. The tests didn’t tell us what was producing the disparities between teachers’ observation and the student’s performance on them. They simply provided an alternative assessment of a student’s performance on measured objectives.

Simon asked: “Do you think your child's competance is better measured by filling in a bunch of ovals than it would be by, say, writing an essay? Yes, an essay using keyboard, thesaurus, spell-check, etc. Preferably submitted for peer feedback and then submitted in a final version when she was ready?”

I think the standardized tests and the ovals shed light on a discrepancy between objective trained professional teacher observation and a student’s ability to answer questions in the format provided by the standardized test. As for my daughters’ competence, it took far more effort and further assessment to yield the answers that ultimately have led to understanding why the differences were so great and what to do.

The teachers failed when they predicted she would fail to demonstrate competence and the standardized tests failed to observe what the teachers observed in classroom performance. Neither form of assessment was accurate or complete. As I asked Mr. Ferriter, “If that rising sixth grader were entering your class, would you want only her former teachers’ observations or would you want those test results as well?”

Simon, frankly without knowing what we later learned because we were prompted by the discrepancies in assessments, I do not think my daughter’s competence would be better demonstrated by an essay, peer reviewed or not. She would have been asked to write (something she couldn’t do effectively) or type (something she lacked training to do at the time). It took doing something the school did not do, asking why there was a discrepancy? Stating it “simply”, why was a student who tests so well performing so poorly?

In order to achieve “The point of education…” which as you stated “…has to be to teach kids to do something that they couldn't do before.”, my daughter needed someone to see a discrepancy existed and engage in determining why. Had teacher observation alone been available to us, we would still be insisting to great frustration that she must simply apply herself more. Instead, the availability of alternate assessment via standardized tests opened a window that prompted more questions and she now enjoys learning many things she could not do before.

[Mike H]

I'm a little late in this, but this seems like a great way to have a final exam. In Virginia, students take their end of course exam, and then many exempt out of the Final Exam, which is a shame b/c mostly, they don't deserve it. That's another point.

So, because the kids have already had a high stakes multiple choice test, why not have every student complete this form of final exam?

But then again, most teachers have 100+ students. One class, with 25 kids would take more than 250 minutes (that's 4 hours). And with at least 800 students taking 11th Grade US History for example, I'm not sure how that can be accomplished when there are dozens of other content areas to be tested.

Do the Danish schools have a small number of students?

[Simon Oldaker]

@K. Borden: We seem to have fallen into the trap of a discussion poised between general principles and a specific case, which I know nothing about (I assumed that your daughter could type, for instance, and that being forced to use a pencil was part of the problem). I stuck in 'subjective' because I was fishing for the nature of the contrast. The word 'observation' seemed an odd one - is all teachers do watch?

So - the general principles? I'm sceptical to standardized multiple-choice questionaires as a measure of competancy, because they are notoriously poor at uncovering higher-order skills (creativity, the ability to organise complex thoughts, etc.) These are the skills I am interested in as an educator and as a parent. An excessive use of standardized testing can lead to an unhealthy emphasis on memorization and conformist thinking.

So I'm curious - maybe this is in yur posts, but I just haven't grasped it - what did your daughter get to do on these tests that she did not get to do in class? Or - what was it the teachers missed and why?

[Val Pientka]

It all boils down to trust. Teachers are not trusted as professionals regarding students, content, or assessment. Until we are trusted as professionals, students will be subjected to standardized testing that in turn will yield less than meaningful data that will most likely be used against teachers.

[Clix]

I've been away for awhile. I'm with Mike H. on this - how do they staff it? you've got a group of teachers x ~20 minutes per student (at least). 10 minutes for the interview, 5 for the rubric, 5 for feedback.

It just seems awfully time-intensive. In fairness, though, they do this ONCE, so once you add everything up, maybe it's not as intrusive as allll the standardized testing we do.