Impact? What impact?

By | May 23, 2014

I wrote this article for the TES. They asked for 1300 polished and considered words. I added some jokes too, for free. They let a subeditor loose on it and, well, you can read the bloodied stump on their site if you want to see the damage that can be wrought by someone in possession of a pair of scissors and charged with making something complex understandable by an international audience… What follows is much better though.

 

Impact? What impact?

“I totally agree with you” the Headteacher says across her office coffee table. “A wireless network makes absolute sense seeing as we’ve already bought the iPads. But…”

There’s always a ‘but’.

“I’m going to need to see the empirical evidence that technology makes a difference before I can justify spending any more money”.

I have a lot of conversations like that and this one, to be clear, is both composite and hyperbolic.

The truth is, schools have invested billions putting technology into our schools over the past twenty-five years yet two facts remain extant:

  1. No one on the ground is really sure what the evidence (you know, proper Social Science done at Universities, using numbers and everything) proves
  2. Everyone on the ground is really sure what the evidence (you know, the stuff you see with your eyes and understand with your brain) from their own lived experience tells them

We need, as a sector and in the technology facet particularly, to get much better at understanding what works and what does not. It is barely credible that ‘effect sizes’[i] obtained through meta-analysis of hundreds of small studies in widely varying contexts are as quantifiable as it gets.

impact image: Flickr

That is not to say we have learned nothing in 25 years. Studies have taken place and teachers have learnt plenty from successful and unsuccessful technological intrusions into the classroom. So while more study is needed, below are five things we can at least say with some confidence about tech and education.

  1. Motivation matters

It’s a truism that learning is always preceded by engagement, and the evidence that technology motivates children is strong. Key studies in the past two decades (Cox 1997[ii], Passey and Rogers 2004[iii]) have demonstrated that learners find the use of technology intrinsically motivating. We probably need to acknowledge the somewhat scary notion that children see school technology as an extrusion of ‘their domain’ into an otherwise adult-controlled world. It’s something they are not only more familiar with, but often ahead of the adults in terms of expertise. Daniel Willingham’s 2010 paper ‘Have Technology and Multitasking Rewired How Students Learn?’[iv] presents a balanced review of the evidence in this area.

 

  1. Agency empowers

Most personal technologies grant the user more control – over what they investigate, how they acquire knowledge and what they do with it. Where empowerment leads, efficacy and autonomy soon follow, with children feeling a greater degree of independence and responsibility for their learning. Recent and relevant evidence of this can be seen in the University of Hull’s 2013 study[v] of tablet schemes across a number of schools in central Scotland. It found that 70% of parents reported their child’s persistence in learning was significantly impacted and that 84% of children were more likely to work at home when using these devices. And if we believe the recent fuss over the importance of grit and effort as success determinants, those are pretty compelling statistics.

 

  1. Accessibility; the Cinderella impact

Regardless of impact measurable at the system scale, technology opens doors for learners who for one reason or another can’t quite reach the handle. Examples abound – a study[vi] published earlier this year demonstrated that the students’ misconceptions about astronomical concepts were reversed through the use of tablets, due to the ability to manipulate virtual models. We’ve all met children whose weak literacy skills utterly undermine their educational experience, and yet who can talk at length and with articulacy on a specialist subject – technology (like being able to speak into a tablet and have it type your words) can be a powerful remover of barriers. Sometimes technology simply makes the impossible possible – there are children in an academy in my Group taking GCSE Astronomy next year. Their teacher is miles away in one of our independent schools. So in outcome terms, their chance of achieving this qualification has just risen to a number higher than zero, which is where it would be without technology. They’d probably call that impact.

  1. Effective schools use technology effectively

It’s important to note that almost any technological tool, from interactive whiteboards to VLEs will – in the hands of a committed enthusiast – be well used and lead to learning gains, or at least the perception that a gain can be attributed to them. The reverse is just as true – no technological tool, no matter how shiny and awesome can transcend poor teaching. It will still just be poor teaching, albeit softly lit by blinking blue LEDs.

 

However, schools which employ technology whenever it is the best tool for the job, embedded through strong leadership, routinely turn out to be better schools for it. Becta’s analysis of the Ofsted reports of ‘ICT Mark’ schools in 2008[vii] found that these schools making effective use of technology were four times more likely to be graded Outstanding. More recently, the Alwoodley Primary School case study[viii] is as good an example of this as any I’ve come across; nothing revolutionary or ‘transformational’, just lots of good ideas, well applied. Successful technology projects in schools almost always rest on the quality of leadership and implementation (including training), and almost never on the quality of the technology.

 

  1. Technology should be applied to proven learning techniques

This is premised on the view of technology as merely a tool for learning and turns the argument on its head. Compared to what we know about technology and learning, there is fairly firm evidence from research about other methods that appear to have impact. Futurelab at NFER[ix]have developed an ‘Innovation Matrix’ which builds on these ‘known knowns’ to try and uncover some of the ‘known unknowns’ of educational technology. “By creating a matrix with ‘effective pedagogy’ (across the top) and ‘uses of technology’ (down the side) it is possible to consider how we might think about the intersections as areas of activity most likely to yield positive results” says Gareth Mills, Head of Learning and Innovation at Futurelab.

 

A good example of this is to be found in Harrogate Grammar School’s mobile learning project[x]. The school knew that effective feedback was an important and well-understood component in students’ progress. They also saw how a tool (a specific set of apps on iPads to create a workflow) that enabled teachers to do this in a rapid, detailed and personalised manner would accelerate learning.

 

Most readers will be able to recall a time when they’ve seen technology have an impact in school beyond what would be possible using other methods. In each case, I’d argue, it would have been because the teacher thought long and hard about what they wanted to achieve, weighed up the tools available to them, and applied the one which was most suited.

 

And often that’s a pencil. Sometimes technology can be part of the answer, as demonstrated by teachers like Daniel Edwards, whose online[xi] commentaries of his work typify the sceptical but open-minded pedagogue described in the paragraph above.

 

So, the evidence suggests that as a teacher you can be confident that your thoughtful and discerning application of technology will lead to Good Things, amplifying your effort and motivating, enabling and accelerating your pupils’ learning. The catch is that there’s no handy template for making this work in every classroom yet.

The nation (and I use that term inclusively; the Government, the Academic community and everyone working in or for a school) must make better progress in identifying the impact of technology in more generalizable ways. We need to reach a position where we know which technologies are most likely to support learning, regardless of the context in which they are applied. Perhaps then we can move past the polarised, self-defeating scuffling between ‘tradition’ and ‘tech’ camps and focus on using the optimal blend of both.

 

At the end of my almost entirely fictionalised chat in the Head teacher’s office, she asks me a door-handle question, like some kind of educational Columbo.

“Just one more thing before you go – what one single technological tool would you recommend we invest in?” I think for a second before answering, aware that a future article in the education press may hinge upon it.

“How about something that would give every pupil access, at the tips of their fingers, whenever they need to refer to it, to the entire canon of human knowledge and culture, explained by experts and curated by the wisdom of the whole species?”

“Exactly!” the Head says “That sounds like the sort of thing that might really help – but how do you know for sure?”

 

 

[i] John Hattie’s 2009 work on effect sizes is well summarized here: http://www.teacherstoolbox.co.uk/T_effect_sizes.html

[ii] Cox, M. (1997) The Effects of IT on Students’ Motivation. Coventry: National Council for Education Technology.

[iii] Passey, D. and Rogers, C. (2004) The Motivational Effect of ICT on Pupils. Nottingham: DfES.

[iv] http://www.aft.org/pdfs/americaneducator/summer2010/Willingham.pdf

[v] http://www2.hull.ac.uk/ifl/ipadresearchinschools.aspx

[vi] http://www.sciencedirect.com/science/article/pii/S0360131513002534

[vii] Becta’s Next Generation Learning website was shut down in 2010 after the agency was closed and its contents dispersed. It’s a bit like the Royal Library at Alexandria, in many respects.

[viii] http://www.ofsted.gov.uk/resources/good-practice-resource-using-modern-technology-improve-english-alwoodley-primary-school

[ix] http://www.futurelab.org.uk/home

[x] Presented at the NAACE Strategic Conference March 2014, http://www.harrogategrammar.co.uk/school/why/ipads-for-learning/

[xi] http://dedwards.me/2012/10/14/ofsted-outstanding-teaching-and-the-ipad/

 

12 thoughts on “Impact? What impact?

  1. BRUCE WILSON (@brucewilson283)

    Excellent post Dominic. As discussed, cross referencing the things that have been proven to make a difference with technologies that we can demonstrate do those things better than we’ve done them before appears to be a powerful way forward.

    Is the NFER work WIP or online somewhere?

    Reply
    1. norrishd833 Post author

      Not that I can find – it was shared at our Leadership Conference in March but was just a PowerPoint slide at that time

      Reply
    1. norrishd833 Post author

      I think it’s simpler than that – proven methods (e.g. Hattie’s effect sizes) down the side, technologically-enhanced methods of improving their effectiveness across the top

      Reply
  2. Mark Anderson

    Another very interesting post. Thank you Dominic.

    Reply
    1. Mark Anderson

      Another very interesting post. Thank you Dominic. You should have a look at mine – we talk about a lot of similar things.

      Reply
  3. Ben

    Bob Harrison and I always have a conversation around this!! IWB’s have never improved any child’s grades…fact. Do they aid some engagement in the classroom or allow children to understand a fact the teacher is putting across….possibly. Depends on who is teaching the student?! As you said the strong leadership team is the key. Speak to IPACA, Streetly, Clare’s court. Technology enables education but is by no means the reason OFSTED rate the school as outstanding. It is a first class leadership team and great teaching……hopefully etag will let us have this discussion finally!!!

    Reply
    1. norrishd833 Post author

      Agreed Ben – technology can be part of the story of a great school, but it’s leadership, culture etc that is behind it all

      Reply
  4. Merlin John

    Oh dear Dominic, it’s hard to believe that your intro was edited out. It speaks volumes and really does reflect what has been happening in many schools. A really good read.
    The problem with most debates about “evidence” is that many of the people who ask for it don’t really want it and wouldn’t know what to do with it if even if you were able to give it to them. If they were honest, they’d say what they really mean: “Talk to the hand.”
    I think the best explorations into the role of technology for learning are coming from Michael Fullan (http://www.michaelfullan.ca/) and his collaborators. He’s hoping to work with 1,000 schools worldwide as part of his “New Pedagogies for Deep Learning” project. I’ll be very interested in the evidence it produces.

    Reply
  5. Ros Walker

    I do enjoy reading your posts! I am reminded of a conversation with a headteacher about impact. He said that the ‘impact’ was not as important for him as making sure that students were ready to face the real world. “One thing is certain,” he said. “If our students don’t make their way in the world, if they don’t get jobs and can’t live to their full potential, then that will have an impact. “

    Reply
    1. norrishd833 Post author

      Thanks Ros! Trouble is, if you can’t put a number on it, some people refuse to accept it exists!

      “Not everything that counts can be counted, and not everything that can be counted counts.”

      Reply

Leave a Reply

Your email address will not be published. Required fields are marked *