Contents
You can download the full Evaluation how to guide [PDF 540KB]
or view it online.
- Introduction
- Methods of evaluation
- How to prepare an evaluation plan
- Focus on objectives
- Effectiveness
- Audience participation
- Independent evaluation
- Sharing what you learn
Appendices
- An evaluation checklist for consultation managers
- Questions to ask at evaluation meetings with stakeholders and consultees
1. Introduction
Any major communications projects such as a consultation should be evaluated. Evaluation helps to:
- Find out what worked, what did not and why
- Increase learning and improve future practice
- Assess whether involving the public contributes to improved services, actions or decision making
- Judge whether a consultation was cost effective in terms of time and resources.
If you are in a position to be able to answer ‘what would I do differently or the same next time?’ then your evaluation will have been valuable.
2. Methods of evaluation
There are three main approaches to evaluation:
1. Decision audit
- how the views of consultees affected decisions or action
2. Process review
- the extent to which plan timescales, budgets and project objectives were met
- the extent to which the intended audience took part
- whether the methods were effective
- whether there is anything that they would do differently.
3. Stakeholder satisfaction
- the general satisfaction of consultees with the process
The best evaluations use a combination of all three.
3. How to prepare an evaluation plan
The evaluation should be considered at the planning stage of a consultation. It should be:
- proportionate to the scale of the project
- done in good time
- have adequate resources invested in it.
4. Focus on objectives
Clear objectives provide the basis for an evaluation. For each objective there should be some performance indicator and performance level explicitly stated. This can then be assessed at the end of the consultation to give a strong indication about whether the consultation was a success.
Using SMART (Specific, Measurable, Achievable, Relevant and Time bound) objectives helps when it comes to check whether they were achieved.
5. Effectiveness
One key question is ‘has anything changed as a result of the consultation?’ At the end you need to be able to measure whether:
- you got the information that you wanted
- you have used the views obtained
- the consultation has led to some identifiable change in your service or policy
- the consultation has changed the relationship between you and your users and others.
6. Audience participation
It is good practice to offer participants an opportunity to comment on any consultation because they can offer a unique insight. In particular, consultees can be asked, as part of the consultation process, for their views on:
- the information provided (adequacy, ease of understanding, availability in appropriate formats)
- the methods used
- the timescales offered for responses
- whether they felt the consultation was worthwhile.
Regular meetings with key stakeholders provide a good opportunity to ask them for their view on the way the Council consults generally. They should be asked wherever possible to illustrate their answers with specific examples drawn from consultations they have recently been involved in.
7. Independent evaluation
The Cabinet Office and Audit Commission have developed frameworks against which consultations can be evaluated. We have adapted these for use in Nottinghamshire and the checklist appears in Appendix 1.
In circumstances where you are particularly keen to learn the lessons, or where there is the potential for substantial external criticism, it may be worth considering commissioning an independent evaluation of your consultation.
The Consultation Institute offers an external assessment based on its knowledge of best practice and can be contacted at:
21a High Street
Sandy
Bedfordshire
SG19 1AG
tel: 01767 689 600
e-mail: info@consultationinstitute.org
or via the Customer Management Team at :
e-mail: consultation@nottscc.gov.uk
tel: 0115 9772937
Independence can also be brought into the process by establishing a stakeholder steering group to help manage the consultation or by commissioning a third party to summarise the responses and draft the consultation report.
8. Sharing what you learn
The Nottinghamshire Consultation Practitioners Forum allows consultation managers to spread best practice within the Council and throughout Nottinghamshire. We all learn from the successes we achieve and the mistakes we make. Sharing what we learn will allow us to improve our performance.
Examples of good practice will be recorded in the relevant section of the Council’s consultation database and if you need help with a consultation these examples may well help. In addition help and advice is available from the Customer Management Team.
Nottinghamshire County Council's consultation standards
- A consultation mandate should be completed and approved by your departmental co-ordinator/champion before the start of any consultation.(Guides 2 and 11)
- The consultation should be logged on the County Council’s consultation database.(Guides 2,10 and 11)
- For key decisions a public engagement plan should be published with the forward plan.(Guides 2 and 11)
- In planning a consultation, the specific communication needs of groups and individuals who are often otherwise excluded should be considered.(Guides 3 and 4)
- A minimum of 12 weeks should be allowed for consultation on major decisions.(Guide 2)
- The name, address and contact number of the person responsible for the consultation should be published on all consultation materials.(Guides 2 and 10)
- The consultation material should make clear by what date responses are required, in what format and to whom they should be sent.(Guides 2 and 6)
- The consultation should include a face-to-face element where consultees are able to meet, question and put their views to the decision-maker(s).(Guides 4,5,7 and 8)
- Any venue selected for a consultation event should meet the Council’s accessibility code.(Guides 4 and 8)
- Any complaints about the consultation, questions asked, materials or time allowed should be noted in the consultation report.(Guides 9,10 and 11)
- A notice of decision should be published for each consultation.(Guides 10 and 11)
- Feedback regarding the responses, the Council’s decision and how the consultation influenced it should be given to consultees.(Guides 4,7,9,10,11 and 12)
Copyright:
All County Council copyrights are reserved. The supply of Council information does not grant the applicant the right to re-use the information in any way that would infringe the copyright on that information.
For any wider use, the County Council may choose to allow the re-use under licence. This enables the County Council to ensure it is not used in a manner which is inconsistent with the County Council’s copyright.
