real evaluation

Dr. Jane DavidsonKia ora and welcome to Real Evaluation Ltd., a New Zealand-based, world-class evaluation consulting, capacity building and professional development organization that was founded in 1997 by director Dr. E. Jane Davidson. We work independently and collaborate with some of the world’s best evaluators to offer a range of specialist evaluation products, services and resources to organizations in New Zealand and around the globe.

What is Real Evaluation?

Have you ever read through an entire evaluation report and still been none the wiser about whether the programme or initiative was a complete waste of time and money or not? Or one that didn’t actually answer any useful questions of key importance to decision makers?

What if evaluations actually asked important, big picture questions about quality, value and importance? What if the final deliverable was a report that gave clear, concise, direct answers to those questions?

What if the evaluation report could give you an approximate answer to an important question instead of a precise answer to something trivial? What if it could tell you roughly whether you were getting value for money and effort – even when many of your most important outcomes were intangible?

What if the evaluation itself was real value for money? … Seriously!

Millions of dollars are wasted every year on evaluations that may look technically adequate but somehow fail to meet the needs of real-world decision makers.
These fall into four common types:

  • The “no questions” evaluation: No overarching big picture questions (What do we need to know?) to guide the evaluation. No questions => no answers!
  • The “research questions” evaluation: Questions are listed, but they are narrow, they are written at a micro (indicator/data) level, and they are mostly descriptive rather than evaluative. Your end product looks more like a measurement exercise, an indicators report, a rich description, or a publishable research paper than an actionable evaluation.
  • The “no answers” evaluation: Evaluation questions are listed, but they are not directly answered. Data are presented – graphs, tables, interview summaries, observations, … but they are never interpreted adequately or woven back into direct answers to evaluative questions
  • The “nothing new” evaluation: You get some answers to evaluation questions, but unfortunately you already knew the answers from previous work; the evaluation hasn’t answered anything new, anything you can really use; you are still left with the same dangling unanswered questions as last time

Is there any other kind of evaluation out there? How would we find it?

  • Yes, there is an alternative!
  • Yes, it is reasonable to expect direct answers to important evaluative questions.
  • Yes, you can write requests for proposal that are more likely to get you a real evaluation.
  • Yes, your staff can learn how to spot the difference between a proposal for a real evaluation and a waste of evaluation resources.
  • Yes, your staff can be upskiled to be able to effectively manage and ensure your contractor delivers – and to do real evaluation in-house.

… Get Real!