The Implementation effect

By | July 29, 2018

I’ve been working with colleagues at United Learning to understand why the educational technology they are using having an impact on learners’ attitudes to/ skills in maths and writing, respectively. The interviews I’ve carried out with staff and pupils have strengthened a view I’ve long held – that the way a technology is implemented has a far greater influence on its impact than the quality or functionality of the product .

I’m not saying that the EdTech product doesn’t need to be good too, just that it takes a truly great implementation to make anything have an impact… and the very best tool poorly implemented will always fail – that’s pretty much what we’ve been living through over the past 20 years of schools using technology. There are a few schools that stand out as reliably using technology well (Shireland, Simon de Senlis, Wymondam, etc) and this isn’t just because they’ve somehow chosen the ‘best’ products, it’s also because they’re practiced at implementing them well and sustaining their use.

What makes for a high-quality implementation then? There are, it seems to me, some things that are always true of good implementations


1. Choose wisely, and then focus
It may seem obvious, but it’s worth restating: if you’re expecting everyone to make something work, it needs to address a genuine priority or challenge that the school faces. For years in the 2000s schools grappled with a government requirement for them to have a Virtual Learning Environment, the very epitome of a solution in search of a problem to solve. In an era of relatively low broadband speeds and limited pupil access to devices at school, it made little sense. The vast majority of schools stopped doing it as soon as it became apparent that they weren’t going to get in trouble. Education is damagingly cynical about technology already, so give yourself a fighting chance by implementing something that you are as sure as you can be will make a difference to something important. Look at what is working in similar contexts and only trust the testimony of your peers who’ve done it. One of the schools I’m currently working with is focused intently on improving writing, and to do that they have to get children to like writing and to do lots of it. They’ve implemented Pobble and it’s really helping to do those things, partially because it’s a good product with a sound concept, but mostly because the school is using it in a structured and rigorous manner.

The other element around choice that I noticed is that quite often this is the only thing a school is doing with technology. This allows staff to focus on a single (significant, high-effort) change, rather than chucking a load of plausible solutions at a problem and hoping that one or a combination of them works. This seems to lead more reliably to impact. It’s certainly cheaper, but it does rely on the choice of that one focus to be very discerningly made.


2. High quality operational management, from a leader with some skin in the game
No change sticks in schools without the sponsorship of someone who can remove barriers and create space for staff to develop, and that person needs to be senior enough to influence others and make decisions. This person must be a details person who isn’t teaching all day every day, because they have to design and oversee a complex programme of change. And change doesn’t happen accidentally, it happens because it’s been carefully planned. They will have trialled things themselves and ironed out the worst of the problems with their own classes. They will have set up structures to both support and to create accountability (see below). Most importantly, they will be resilient against the inevitable resistance and depressing set-backs, displaying positive leadership. If you read back those last few sentences, you’ll probably be able to think of a few Assistant/ Deputy Heads you’ve known who have those qualities and who seem to ‘just make things work’.

My other point in this section, perhaps the most important one, is that you rarely see leadership with the kind of commitment described in the paragraph above unless the person feels highly accountable, that they’ve got something to lose if things go wrong (and something to gain when they go well). When they hear about something going wrong/ not working as it should/ taking longer than planned/ someone simply not doing what was agreed, they need to feel worried, and to take action. That might take the form of additional support for a colleague, or a technical work-around, or a difficult conversation. And you rarely see that level of commitment with EdTech unless the decision to implement a product was this leader’s decision. In short, they need to have picked it. This is why externally imposed solutions (cf. the LA district’s iPad fiasco) almost always fail miserably – there’s no one in school who truly sees this as ‘their problem’, passive resisters aren’t challenged, the unsure aren’t supported… and eventual failure is highly likely.


3. Reduce friction
With something as difficult and fragile as a new use of technology by teachers, it is really easy for small problems to derail things – and small problems lead to the withdrawal of trust by staff. Before you know it, you’re in the ‘gathering dust on a shelf’ phase of the project. This is why successful implementations are all about the detail. Barriers to successful use by teachers and pupils should be anticipated in the planning stage and mitigated through technical changes or training. A good example of this might be a web-based resource that requires staff to log in with a new password. Usually this results in a proportion of people forgetting their credentials and giving up on using the resource. A high-quality implementation would have seen this becoming a potential problem and insisted that the provider use an existing form of identity or pass-through (e.g. Office 365, SAML, a single-sign-on portal). This example of friction is so deleterious to staff uptake that I have in the past rejected potential products purely based on their dispiriting approach to identity management. There are many more possible examples, most often around something’s reliability (whether it’s going to stop working in a lesson). Far, far too often I see these problems persist due to the pace of work in schools – no one has time to come up with a solution or the solution is no longer possible, and teachers hobble on sub-optimally, or just give up.


4. Create structures of support and accountability
This point is about making the various humans (adults and children) feel that they have a stake in what you are doing. Our implementation of Hegarty Maths across the Group’s secondary schools has had to rely on being well structured, as the person leading it works across many schools. She has gathered together a list of techniques that are having success in other schools, so that those rolling HM out can build their own strategy on the work of others. She has also introduced ‘Hegarty books’ for pupils, which are used formally alongside this online resource, to create a record of the work and thinking each pupil has gone through. Along with this, schools usually structure use of the product on a weekly cycle, with a ‘Hegarty Day’ (e.g. every Thursday) being the point at which these books are reviewed and pupils’ progress in the tool shared. The use of ‘league tables’ (most activities completed, etc) has proved motivational and is mirrored at a school level so that staff can see how their efforts compare to colleagues in other schools. None of this is monitoring, but it does create a useful sense that activity is tracked, is noted, and that it matters. Through these steps and others, the lead has helped schools to formalise the use of what could have just been another ‘maths homework site’ in the eyes of pupils and staff, creating a culture where it is sparingly but effectively used to supplement class teaching.


5. Pupil power
This final point is about creating momentum to sustain the change through getting children excited about it. In the school implementing Pobble that I mentioned in Point 1, every class has two ‘Pobble champions’, ‘Published Author’ badges and regular Pobble celebration assemblies where the best pieces of writing are read and their authors can bask in the glory. The champions have many jobs (such as commenting on published pieces) but probably their most important is to notice and congratulate children from any year group who are wearing the badges showing they’ve had a piece of work published. This is a facet of the ‘create structures’ point above, and it helps establish and maintain a culture where this important intervention is talked about and becomes part of the positive story of this school’s use of technology. It’s worth noting that the lead in this school is non-technical (she’s a AP with responsibility for English) and that this is the school’s main thrust of its effort with technology, it’s not doing too much else. The tech is totally peripheral anyway – this is a learning project which happens to take advantage of the connectivity of the Internet to make writing exciting.


If all of this is so obvious, why is effective implementation the exception rather than the rule? I think this goes back to Point 1 more than anything else. Schools have finite energy and patience for new initiatives, so less is very much more in this respect. Unless the EdTech project is absolutely accepted by all involved as adding educational value beyond what would otherwise be possible, few of the other things required for impact are likely to happen. Once it is, the rest could be argued to be simply strong leadership and project management.


6 thoughts on “The Implementation effect

  1. Neil Emery

    I’m a big fan of pupil involvement, making technology part of the learning and real world. With TrilbyTV I always strongly suggest pupil involvement and ownership. Give them a brief to work to and let them design and create content for a schools digital signage. How fantastic at any pupils age is it to say ‘I manage and create content for my schools digital signage’. It’s never Rocket Science but I think control often gets in the way of successful technology implementation in schools. Good piece as always Dominic – thank you and I hope you are well

    1. norrishd833 Post author

      Thanks Neil. Having pupils heavily involved is another feature of success that I often see – initially for additional IT support but increasingly because their enthusiasm and inventiveness are hard for the adults to ignore!

  2. Richard Martin

    This is a great article and I wholeheartedly agree with all of the points made. I would also add the importance of timing and testing to a successful implementation, which in my experience are often under valued or overlooked entirely. The insistence sometimes of new projects being prepared in the summer for a September start is often a recipe for failure. There is little or no opportunity for user or load testing in the summer and introducing unnecessary complexity on the first few days of the school year often leads to initiatives being scrapped and never revisited again. A carefully phased pilot with a few engaged individuals can quickly expand to a full roll out once news of its success and benefits has spread.

    1. norrishd833 Post author

      Thanks Richard – agree. The wrinkle in piloting, I’ve always thought, is where people pilot something in a model totally different to how they actually plan to deploy, e.g. a class set of shared devices, when the vision calls for an eventual 1:1 model. It’s difficult to learn relevant things unless your pilot is truly that.

  3. Pingback: #122 - UK DFE EDTECH ANNOUNCEMENT & Global Teacher Prize Nominees on Robotics & Maths — The Edtech Podcast


Leave a Reply

Your email address will not be published. Required fields are marked *