Liz Hoggarth is a senior research fellow of the Youth Affairs Unit at De Montfort University, Leicester. Liz has been heavily involved in writing and training on outcomes evaluation, especially for those working with children and young people in statutory and third sector organisations. She is co-author of the new book, A Practical Guide to Outcome Evaluation, published by Jessica Kingsley Publishers.
Outcome evaluation – help is on the way.
In 2006, I was asked to work with a group of local Children’s Fund projects to help them evaluate their outcomes. Much of the jargon floating around sounded dry as dust and a long way removed from the practical everyday work with children, young people and their families. That could not have been further from the truth. With Hilary Comfort and other colleagues, I started to unpack the pressures on these projects to demonstrate the positive results of their work, their lack of know-how about evaluation and the real need for ways of building achievable evaluation methods into everyday practice. It soon became evident that those projects that could not show the impact of their work would simply not survive as the commissioning processes kicked in. They desperately needed help in understanding the concept of outcomes and in finding ways to show the difference they made in the lives of clients. That experience has been repeated many times as the drive in national policy to improve outcomes has affected so many settings across the partnerships of social work, education and health.
Many of us react instinctively against further demands to produce evidence, especially quantitative information – we know all too well that progress with clients is made up of tiny, often faltering, steps forward that are extremely difficult to demonstrate or quantify. There are downsides to the outcomes approach as there are to other systems of planning and evaluation. But the question of outcomes is a perfectly legitimate one. The number of visits made to a family is beside the point if the risks are not picked up and appropriate interventions are not identified to begin to help people deal with the problems. The number of counselling sessions provided is hardly important if in the end they made no difference for the person seeking help. We must address outcomes in order to improve services.
My guess is that many readers of this newsletter will also be struggling to grapple with the intricacies of the commissioning process and the demands that they should provide evidence of the positive outcomes of their work. We found that many managers and fieldworkers were having to cope with trying to evaluate outcomes on top of the direct work with clients and that there was very little material available to help them. Whether in statutory services, the voluntary sector or even in private companies engaged in care or education, workers are facing the same questions. How do you define your outcomes? How could you show that users have grown in confidence? How do you help staff to understand what data they must collect and why? How can you enable clients who have communication difficulties or disabilities to identify the progress they have made? How do you structure an evaluation report to satisfy commissioners?
That is how our journey started – to write a practical handbook that could be a reference point for practitioners on outcome evaluation. We have seen some good projects die because they could not produce evidence of their outcomes. We have seen others grasping the nettle and embedding effective systems of evaluation and going from strength to strength as a result. The experience is far from theoretical – we hope it can prove useful to many others.
Copyright © Jessica Kingsley Publishers 2010