Examining Promethean’s Recent Report on IWBs.

Anyone who has spent any time poking around the Radical knows that I’m a pretty outspoken critic of Interactive Whiteboards (see here, here, here, and here).

My beef with IWBs—a tool that many district technology directors and school principals seem to love—is a simple one:  Interactive Whiteboards do little to change instruction in today’s classroom.

They facilitate a presentation culture in a time when our students are craving opportunities to participate.  They encourage and enable teachers to hold on to traditional practices that need to be buried if our schools are going to stay relevant to kids.

Oh yeah—and they cost a heaping cheeseload of cash, consuming already limited technology budgets that could be better spent elsewhere.

And in case you aren’t familiar with my background, I’ve got plenty of experience with IWBs.  In fact, for the better part of an entire year, my classroom was outfitted with a full set of whiz-bang gizmos from Promethean—student responders, slates, and a real-live Interactive Whiteboard. 

I spent that year writing professional development courses for Pearson Achievement Solutions, trying to detail ways that teachers could use Promethean’s tools to create student centered classrooms. 

What I found was disturbing to me:  While it was entirely possible to write student-centered lessons for the IWB, it was a pain in the keister.  Truly great lessons took hours and hours to plan and develop, making the tool pretty useless to me simply because I didn’t have hours and hours to plan. 

What’s more, the IWB became passé pretty darn quickly to my kids.  The “your students will learn more because they’re motivated by the technology” argument that the IWB salesmen push on y’all at conferences proved to be patently false.  Like any tool, the IWB neither created or discouraged motivation in my kids. 

Translate my experience across entire schools outfitted with thousands of dollars of Promethean kits and you’ll understand why I’m convinced that we’re wasting our cabbage on IWBs.

That’s why a Tweet from Sonny Mangana, the Director of Global Research for Promethean—pointing to a new study released by Promethean conducted by research superstar Bob Marzano on the impact that IWBs have on student achievement—caught me by surprise last week.  

You see, I really don’t need any research to convince me that IWBs aren’t worth buying!  I’ve got first-hand experience to fall back on.

Sonny was a pretty persistent fella, though, and several hundred Tweets later—no joke, literally hundreds of Tweets later—I decided to read the report. 

And honestly, I’m glad that I did simply because it’s a report that is likely to be used by school leaders to justify purchasing even more IWBs for their schools because the main finding—that IWBs can produce a 15.9 percentile gain in student achievement—is so darn juicy.

Before you whip out your school checkbook and run off to the Promethean store, though, please note:

That Year 1 of this Promethean study took some flak in the educational research world.

Bob Marzano is an amazing researcher who has literally changed education by putting numbers behind practices and helping educators to identify the changes that hold the most potential for reforming schools.  I respect him greatly.

But he’s only human.  And the methodology that he used in the first year of this study for Promethean met with criticism—including this five part critique from VCU Educational Research Professor, Dr. Jon Becker. 

Spend some time reading through Becker’s critique and you’ll find Promethean’s claims on the positive impact that IWBs have on student achievement a bit harder to swallow. 

That the current iteration of the Promethean research only looked at the work of 46 classroom teachers.

Seriously.  46 teachers working with 1,575 students. 

While I’m not an expert in things like the impact that sample sizes have on educational research results, I do know that I wouldn’t feel comfortable spending thousands of dollars of a limited technology budget based on the findings of a study that looked at the work of only 46 teachers.

That the teachers involved in the current study intentionally chose to participate. 

Think about that for a minute.  If you asked the teachers in your building whether they wanted to be a part of a research project studying the impact that IWBs have on student achievement, who would volunteer?

If your building is anything like the ones I’ve worked in, the chances are that you’d get a bunch of top fliers, right?  Super motivated types that are into the idea of researching their practice?

And while that’s great—I love top fliers who are into researching their practice—wouldn’t that influence the results of Promethean’s research? 

I think I’d trust the study more if it included a random sample of teachers simply because it would be more reflective of the instructional reality of the average building. 

That the teachers involved in the current study already had a high comfort level with Promethean tools.

Of the 46 teachers studied in Year 2 of Promethean’s research on the impact IWBs have on student achievement, 32 expressed a high degree of confidence in their ability to utilize their Promethean tools effectively.

Again, that’s great—I’m jazzed that there are teachers finding ways to effectively integrate the tools that their districts are providing them with—but is that distribution an accurate reflection of the confidence levels that you’d find across an entire building? 

Think about your own school.  Are YOU confident that 70% of your teachers would know how to make the most of IWBs? 

If not, then don’t expect to see Promethean’s results replicated in your own building. 

That the teachers involved in the current study were told that one objective of the study was to “differentiate those teachers who obtain positive effects from Promethean ActivClassroom from those who do not” (p. 56 of the PDF).

While this quote may seem innocuous at first, I want you to think about the potential impact that it might have on participating teachers. 

Essentially Promethean is saying, “We’re looking for teachers who use these tools well.  Show us what you’ve got.” 

Wouldn’t that change the amount of time and energy that the teachers spent developing their IWB lessons?  And if so, doesn’t that skew the results of the study? 

From the looks of it to me, Promethean’s sample of teachers were self-selected high fliers who were very comfortable with IWBs and who had extra motivation to craft quality IWB lessons.

While that’s all fine-and-dandy, you’ve got to decide whether or not that’s an accurate description of your teaching staff before you can be confident that you’ll get the same results from IWBs installed in your own building, don’t you?

There’s one more thing I want you to keep in mind about Promethean’s current study before you start buying IWBs:

That the students involved in this project knew whether they were a part of the “in” or the “out” group.

In an honest—and honorable—attempt to keep this research project from disrupting instruction, participating teachers were asked to teach the same unit to two different classes of kids.  One group was taught using Promethean technologies and the other wasn’t.

Now think about your students for a minute.  If they knew they were a part of an important research project studying the impact of IWBs and they were in the class learning with the IWB, how would they respond?

More importantly, if they knew they were a part of an important research project studying the impact of IWBs and they were in the class learning without the IWB, how would they respond?

Isn’t it possible that at least some of the students in the treatment class—the class using the IWB—might work harder simply because they were excited to be a part of an important research project?

And isn’t it possible that at least some of the students in the control class—the class who sees the IWB in the front of the room but doesn’t get to use it because they’re in “the other” class—might not work as hard because they’re hacked off about being left out?

If your kids are anything like mine, that’s EXACTLY what they’d do.

And considering that the sample sizes of students in each group were somewhat small—700-800—wouldn’t that have an impact on Promethean’s results?

So what does this all mean? 

It’s actually pretty simple:  Promethean’s study—like any study on instructional practices—provides some valuable insights into the effect that IWBs have on classroom instruction. 

As Sonny said in one of his dozens of Tweets, “Some information is always better than no information.”

But it would be irresponsible to believe that every student in your school is going to make 15 percentile point gains in student achievement as long as you spend a bit more cash on Promethean’s tools.

There’s too many factors that go into successful teaching—and into the results of this research report—to think that the right tool is the silver bullet to helping your kids learn. 

 

_______________________________________

 

PS:  I’ve intentionally chosen not to address most of Promethean’s statistical, researchy findings in my critique simply because I struggle to count!  Far be it for a guy like me to wade into those waters.

I do want to mention one finding, though, simply because it sticks in my craw the wrong way.  You’ll see it on page 24 of the PDF:

“Again, 15 out of the 46 studies (or 32.6%) were considered statistically significant (p < .05). Conversely, 67.4% were not considered statistically significant.”

Now, I know nothing about educational research, but this seems to say that of the 46 teachers who participated in the study, 15 saw statistically significant gains in student achievement. 

That’s 1 out of every 3, right?

Sonny tried to explain this to me in a Tweet—something about “correcting for attenuation” and “incredibly complex statistical models.”  Marzano even addresses it in his report, saying:

“Using meta-analytic techniques, aggregate findings might be statistically significant even though a number of the individual studies are not significant.”

But that all feels like statistical hocus-pocus to a guy like me. 

I’ll have to rely on someone with more experience in educational research to make that decision, though.  I’m just plain not qualified to determine whether or not this is something to be concerned about or not.   

Hint, hint, Jon Becker!

Comments

[Book23]

I am currently a student teacher and know of other teachers with similar opinions to yours. They feel that whether you're using a chalkboard, whiteboard, or IWB, you are still standing up front and lecturing to the students. If you were trying to convince a school board not to spend the money on IWBs, where would you argue for that money to go to?

[Katie]

Hi Bill,

I agree that IWBs perpetuate a teacher centered classroom. Is there a type of technology that you have found helps student learning more than others?

[Clix]

Also, if you design a lesson based on using ANY tool (even, say, a novel), and then you teach that same lesson WITHOUT the use of that tool... well... what do YOU think the result would be?!

[gasstationwithoutpumps]

"thousands of dollars"? One of our smallest local school districts recently spent $1.5 million on the boards: http://www.santacruzsentinel.com/ci_16603007

This in an area where librarians and teachers are being laid off, school districts have one nurse for every 3000-4000 students, and so on.

The Promethean sales people must be really good (or someone is getting kickbacks).

[Brian Linger]

Mr. Bill Ferriter, I think you hit the nail on the head!

"There’s too many factors that go into successful teaching...to think that the right tool is the silver bullet to helping your kids learn."

I am a student teacher currently taking a technology class where we are exploring the usefulness of apple products (specifically, the iTouch) in a classroom. I’m a fan of using an iTouch or iPhone mostly because they are so small that each student would need their own. The only opportunity I’ve had to work with IWBs has been in the classrooms where I am doing my student teaching. I think IWBs are pretty cool and could be very helpful…however, I see their usefulness based LARGELY on the teacher’s comfort level with using new technology.

So I find myself thinking, “Would an iPad make more sense?”. Especially with the number of apps out there (that is constantly growing), is there anything that IWB could do that an iPad couldn’t? Being as against IWBs as you are, have you thought about the use of other forms of technology to aid instruction? I would be curious to hear your thoughts on the issue.

http://teacherlinger.blogspot.com/
-Here’s a link to my blog if you (or anyone else reading this post) would like to enlighten a future teacher in the making.

[Bill Ferriter]

Hey Katie, Book 23 and Brian,

Glad y'all stopped by the Radical, and I'd say that Brian's thinking on technology in the classroom aligns nicely with mine.

I want to see as many portable, internet connected devices in my classroom as possible. While I'd love iPads, they're still pretty pricey, which would limit the number of devices that any school could afford.

Instead, I'd probably rather have more iTouches. You can run many of the same apps and you get portable digital video and still cameras in the same purchase.

If I had 10 per class---something entirely possible if someone is already planning on buying me an IWB---I could do a ton.

I'd use free online polling services and turn my iTouches into student responders. I'd have my kids backchanneling in chat rooms during classroom Paidea seminars. I'd have my kids creating videos on topics we're studying in class.

If we bought the newest iTouches, I'd have my kids videoconferencing with students in other classes. I'd have my kids fact-checking on the Internet during student presentations.

I'd even have my kids learning to write their own apps--a programming skill that will come in handy some day.

Can you do any of that with an IWB?

Nope.

They're a waste of cash. Period.

Anyway...thanks again for stopping by....Hope to see you again,

Bill

[Bill Ferriter]

Mr. Station,

Great article---thanks for sharing it!

What I found most interesting was the description of the IWB lessons that teachers were delivering with their million-dollar tools.

Nothing there was revolutionary at all. Nothing changed the teaching/learning transaction in a meaningful way.

That's a pattern I always see in articles describing instructional practices supported by IWBs.

And that's still more proof that they're a waste of cash.

Bill

[LY]

Thanks for pushing our thinking about technology. After observing many classrooms using expensive IWBs mainly as projector screens, I just shake my head. When money is so tight can't our dollars be used more effectively? Also, what about the alternatives? TCI sells a product for $150 that does essentially the same thing.If you are curious:
http://www.teachtci.com/whiteboard/

I think there is still great potential for technology to be used in a meaningful way, but we need to continue the conversation about what those ways are. Thanks for your insights!

[Adam]

Bill-
You also might want to look at who made the assessments that the students showed gains on for the study. I am pretty sure that those assessment are not valid.

Also, before you respond with iPods you might want to go back to this statement in your post above, "They facilitate a presentation culture in a time when our students are craving opportunities to participate."
I know I am preaching to the choir, but our students want to produce just as much as they consume and given that notion we need to provide them with tools that will allow them to produce and consume. The iPad and iPod will allow for consumption and not a lot of production. I think we need to look at Netbooks or convertible type devices that can be purchased at low cost, but provide opportunities for production.

[Mindy Sanjana]

My district put a Promethean board in my classroom last year. It took me about 4 months to figure out how to use it. Now that I have it, however, I love it. Here's why: I teach LITTLE (K-2)kids whose English language proficiency is limited. They don't have cell phones. They don't have e-mail addresses. I can't create interactive learning objects that they could use on a computer. With the IWB, I can create polls and games which a group of 4 to 6 kids can play together. The kids can run up to the board, draw lines or tap with the stylus, and interact with each other and with the material. I can't create this experience with a chalk board or a board game, especially if I want to include audio files. I agree with you that the equipment is expensive for what it does and that it is not user-friendly, but if money is taken out of the equation, and if the teacher is motivated to use the equipment, the IWB really does offer a unique way to provide student-centered learning.

[bioman49]

Mr. Ferriter,

I'm a recent recipient of a Smart Board.

I don't use it at all since it's in the front of my room, which is a high school biology lab, in the worst possible place. I would have to stand by the board to utilize its touch function. I don't ever stand in the front of the room when I lecture. I don't like the barrier of the demonstration table separating me from the kids.

My recommendtion of where I wanted it was ignored by the district and the installers. Of course, why should they listen to me the real user. Bureaucracy at its finest!

I have a giant projection screen in the corner of my room that I use to project my Powerpoint lessons on and use a remote to change the slides. That allows me the freedom of movement around the room while I'm talking. I was taught that that was a good classroom management technique years ago. Plus it allows me to move among the kids, not be separated from them.

They are nice and I can use it to project web sites since you can actually touch the links and they go to that link. They are nice because you can write on them and save the notations.

But improving student performance? Right. We know district officials believe that since they are buying them. But any teacher who believes that really needs to look into buying that bridge in Brooklyn that goes up for sale every now and then.

Public school districts are reaching for any quick fix to the test score dilemma facing them. They will try anything and buy anything that "experts" claim gets students to perform better.

How about spending that money directly in the classroom where it rightly belongs? You know things like lab supplies and ordinary classroom supplies.

[EduNut]

I'm kind of an app queen. I've found that anything that can be done using an IWB can be reproduced without cost using a Web 2.0 app.

The quality of any tech tool can't be measured at a meta-level, in my opinion. There are too many variables given the human element, the various fields of study, and the nature of learning.

Perhaps what teachers need to do (as though it's not already happening in the blogosphere) is share the qualities of "what it looks like" when a tech tool amplifies learning in their classroom. What are the indicators?

How do we know what we're using (whether it be an IWB or a cool collaborative application) is actually enhancing the learning process?