Creating evidence of your impact

When it comes to launching a new service or policy, good intentions are only the beginning. Being able to demonstrate that those good intentions have led to positive outcomes is a vital part of implementation, and this is where evidence is important.

Generating evidence is all about describing what you do, why it matters and whether what you have done has made any difference. The success of any innovation ultimately rests on its ability to demonstrate that it is creating a positive impact, and evidence of that is crucial if the innovation is to become more widely adopted and grow.

Planning how you will create the evidence you need to demonstrate impact is therefore important from the outset, and shouldn’t be left until later down the line. Starting this early will prompt you to think more broadly about the the nature and type of evidence you wish to capture, and will make it easier to build data collection methods into your activities. If organised well, the activities you track will also provide key insights and learnings about what is working in your project – and perhaps more importantly what isn’t, allowing you to make adjustments or improvements as you go.

There is a wide range of evidence types you could collect (see Nesta’s practice guide on Using Research Evidence for more on this), and what you choose will depend on your project. For instance, if you are aiming to increase the number of people who cycle to work rather than drive, you might want to use a quantitative measurement and will need to consider whether you want to measure immediate impact or lasting behaviour change when planning a timeframe for evaluation. But if your aim is to improve physical health generally, by encouraging exercise, more qualitative evaluation (such as interviews) may be a preferable option.

While evidence will often be limited in the early stages of launching a new innovation, as time passes and experience grows, stronger evidence will increase confidence that the innovation warrants further investment to increase its impact.

Creating an evidence plan

When you are setting up any public sector project that involves the redesign or implementation of a new service, make sure to take time at the start to consider how you intend to demonstrate value and impact. Create a clear vision and narrative around the impact you intend to have and build in ways to collect the data that helps you prove this.

As mentioned, there are many different types of data and specific research methods that you could use; the trick is to generate high quality and robust evidence that is appropriate for your particular innovation. You might find that the evidence you collect doesn’t clearly show the results you were hoping for, but that doesn't mean it won't be useful to you.

For more specialist areas of evidence capture like focus groups or cost-benefit analysis, you might want to consider commissioning an independent evaluation, or exploring ways of demonstrating that your project is creating a change through the use of randomisation and control groups.

To help you on your evidence journey, you might want to use a framework such as the Nesta Standards of Evidence.

Nesta Standards of Evidence

Nesta developed its Standards of Evidence to help measure the impact of a range of its practical innovation programmes and investments. Their purpose is help understand how confident you can be that an intervention is having a positive impact by looking at the evidence available.

As the diagram below shows, there are five levels in the Standards of Evidence model. Level 1 represents a low threshold, suitable for very early-stage innovations that may still be at the idea stage. It involves little more than a clear articulation of why the intervention is needed, what it will aim to achieve, and why this is better than what currently happens. This might not sound very in-depth, but it’s important to have a clear rationale on how your idea will improve things and create a positive change. It will also give you a sense of what type of impact you might hope to measure as you move forward.

As you move through the levels, you need to collect data that isolates the impact to the intervention, and then have those findings externally validated. As products and services move up through the levels of the Standards of Evidence, so does the certainty that they will have a positive impact on the intended outcome. At Level 5, which is often very difficult to achieve, there should be demonstrable evidence that the product or the service can be delivered at multiple locations and still deliver a strong, positive impact – in other words, that the positive impact is scalable.

To explore the topic of evidence further, take a look at the resources below:

• Nesta Standards of Evidence
• Using Research Evidence: A Practice Guide

Comments