Evaluation and influence

Policy development can be a complex and messy business, with many competing demands, considerations, advice and recommendations.

A key consideration for analysts and advisers on policy development is whether their analysis and advice is accepted and used, and how to evaluate policy outcomes.

These issues were considered at the recent Australasian Evaluation Society 2011 International Conference - Evaluation and Influence.

Two clear themes emerged during the conference about influential evaluation:

• Evaluators require a willingness to try and find answers to questions that are especially complex and challenging.
• Great analysis can be undone by poor presentation.

Both themes resonated with the Allen Consulting Group’s experience in undertaking evaluative projects for government clients.

Thirst for evaluation

There is clearly a thirst for evaluation that answers difficult but important policy questions. For example, Professor Peter Shergold’s presentation about the potential of social impact bonds challenged those present to consider how evaluation could demonstrate that a social innovation, funded through such bonds, was actually having an effect on a particular social problem - including some of the most insidious social problems we face.

It was clear from this example and others, that evaluation will have the greatest influence if it can effectively tackle tricky questions of cause and effect in a compelling and timely way.

Listening to all voices

The conference highlighted perspectives on what makes some reports more forceful than others, including the importance of gathering unfiltered perspectives from people who are not usually heard. This is a common feature of our approach.

For example, in recent projects we have consulted with a cross-section of young people throughout Victoria in a variety of settings to ensure we are hearing from those who would not typically be given an opportunity to describe their experiences.

Reaching such groups in an effective and engaging way often forces us to challenge how we have collected information previously.

Inclusive and transparent

Recent examples of our work also illustrate our overall philosophy of being as inclusive and transparent as possible in the conduct of each project, a theme resonating in several instances at the conference.

In a current evaluation, we involved those responsible for delivering an apprentice support program in a workshop developing the evaluation strategy.

We recognise that working alongside the people who are responsible for a program or service is more likely to lead to findings that are informed by practice and are adopted by service providers.

Clear, concise and focussed

Based on many years of experience, we are committed to presenting evidence, findings and recommendations that are clear and focussed.

In an age of information overload, readers appreciate this, even if it means being selective in deciding what information is included and given prominence. People from government agencies who receive reports confirmed that this most likely to influence decision-makers.

At the Allen Consulting Group, we tackle difficult policy problems and methodological challenges to present reports that contribute to the policy development process.

In collecting and analysing data, we challenge established ways of doing things. Reports will have the greatest impact and be most valued if they provide insight regarding a complex issue and present the results in a clear and compelling way.

By Luke Condon