Pushing EdTech evidence past the era of oral tradition

By | February 10, 2019

When the DfE first got re-interested in educational technology about two years ago, I took part in a round-table session full of practitioners, leaders and people from industry. Seeking to understand the damp squib of EdTech in English schools, the question that the civil servants wanted answering was “Where is the market failure?”

LendEd lets you narrow your focus and then access relevant detailed case studies of potential EdTech solutions

In government-speak, this equates to “Where should we most usefully intervene?” and skips the conversation past all of the things beyond the Department’s remit (e.g. freedoms devolved to Heads, the global assessment landscape, the Ofsted framework, etc). There was lots of conversation around teachers’ skills (misdirection, IMO), budgets and accountability risk.

For me, the biggest ‘broken’ bit of EdTech is that none of us really knows what works and schools are forced to either review the entire EdTech market or to accept the blandishments of the sales people who turn up at their door. This is, partly, because of the current limitations of social research and, mostly, because not enough schools are rigorously and transferably evaluating the EdTech they use.

I’m not blaming schools; they have until now lacked adequate frameworks and tools to make this work achievable and portable – in short, to make it worth spending any time doing. At best, valid information on effective tech has been stuck in the ‘oral tradition’ phase of its history, with anecdata whispered from Head to Head. Where, concluded the DfE, is the TripAdvisor for EdTech?

Off the back of the DfE’s challenge and due to the work of some switched on organisations, schools now have access to a couple of promising online tools which should make identifying the right EdTech solution easier, cheaper, and more reliably replicable.

The first is EdTech Impact, the brainchild of Mike Forshaw, a former school network manager whose company Innovate My School has been publishing teacher innovation stories and providing Heads with EdTech ‘speed date’ events for several years.

EdTech Impact’s starting point is to ask school leaders about their context and the challenge they want EdTech to help to address. This information winnows down the information presented back, meaning that you see evidence from relevant contexts – similar places with similar issues – and can see how each product worked for them.

Screenshot of Learning By Question's page on Edtech Impact
EdTech Impact presents summaries of its structured RCE measures from leaders in schools similar to yours

At the heart of the EdTech Impact project is the process of Rapid Cycle Evaluation (RCE) by which participating schools set impact criteria and evaluate against them across the space of a term, sometimes less. This is to avoid the well-known issue of meaningless 5 or 1 star reviews as well as to gather structured and comparable information that can be applied across all products.

Another tool that has recently been launched with the backing of BESA is called LendEd, a marketplace of educational technology which gives schools the ability to browse and search by subject and function to identify things that have worked well in schools like theirs. An added benefit on top of this is the ability to initiate a free and no-obligation trial of the solution.

A case study from LendEd, combining evidence from multiple reviewers

These are both very welcome and useful tools for school leaders. They’re free to use and, over time, will create real intellectual capital for the sector. The effects, at a system level, will be reduced workload, fewer failed projects, an end to the annual cycle of new EdTech products which never seem to stick and a whole heap of money saved for spending in more impactful ways.

My super-talented and always thought-provoking colleague over at Ark, Lauren Thorpe asked me something recently that I think bears consideration – should the use of an evaluation platform be mandatory for any school implementing a new piece of technology? There are some sound arguments in favour:

Firstly, there’s an ethical imperative to share knowledge if it might help children in other schools, and secondly the tools described above will force a structured consideration of any initiative’s success. Use of EdTech Impact or LendEd will actively improve the quality of implementations by schools and prevent things being trialled without starting with a clear educational driver and firm basis in the experience of similar schools.

The final reason for people to engage is that without schools participating at scale, neither platform can provide useful information to the rest. To date our best source of reliable EdTech recommendation has been via word-of-mouth from the head-down-the-road and all of us engaging in online evaluation may be price that the sector requires in order to breakthrough to widespread efficacious use of Edtech.

One thought on “Pushing EdTech evidence past the era of oral tradition

  1. Peter Burgess

    Thank you for this, very interesting to see your perspective on evaluation platforms, extremely important and although I was sceptical at first I must take my hat off to colleagues at EdTech Impact who have made a damn good job of it in my opinion.


Leave a Reply

Your email address will not be published.