As part of this week’s National Professional Learning and Development Facilitator Training (held in Auckland and sponsored by the Ministry of Education), Nan Wehipeihana and I presented a workshop for PLD providers:
This is an introduction to some of the rubrics developed for the Measurable Gains Framework, designed to evaluate the effectiveness of Ka Hikitia, the Ministry’s strategy for Maori learners.
Click the image above to download a PDF copy of the presentation.
And, click here for the Measurable Gains Framework rubrics themselves (and accompanying documents).
Want to catch any other updates appearing here? Sign for email updates or the RSS feed.
Registration for the Australasian Evaluation Society conference in Wellington has just opened and there’s an impressive pre-conference workshop line-up, including one from Jane Davidson entitled: Actionable answers for real-world decision makers: Real evaluation nuts and bolts that deliver.
Summary of workshops
| Date |
Time |
Topic |
Presenters |
| Monday 30 August |
8.30am to 4.30pm |
Monitoring, Evaluation, Reporting and Improvement (MERI) |
Jess Dart, Clear Horizon |
| Monday 30 August |
8.30am to 4.30pm |
Introduction to Realist Evaluation Design |
Gillian Westhorp, Community Matters |
| Monday 30 August |
8.30am to 4.30pm |
Linking Evaluation and Social Justice |
Donna Mertens, Gallaudet University |
| Monday 30 August |
8.30am to 4.30pm |
Introducing Developmental Evaluation |
Kate McKegg, The Knowledge Institute Ltd; Nan Wehipeihana |
| Monday 30 August |
8.30am to 12.30pm |
Using Evaluation for Policy: Rapid Evidence Assessments (REAs) |
Rowena Cave, Ministry of Justice |
| Monday 30 August |
1.30pm to 5.30pm |
Using Programme Logic in Multi-Level Multi-Site Programme Evaluations |
John Owen, Centre for Program Evaluation |
| Tuesday 31 August |
8.30am to 4.30pm |
Developing Monitoring and Evaluation Frameworks |
Anne Markiewicz, Anne markiewicz and associates |
| Tuesday 31 August |
8.30am to 4.30pm |
Evaluation design and implementation |
Elliot Stern |
| Tuesday 31 August |
8.30am to 4.30pm |
Actionable answers for real-world decision makers: Real evaluation nuts and bolts that deliver |
Jane Davidson, Real Evaluation Ltd. |
| Tuesday 31 August |
8.30am to 4.30pm |
The power of the visual image: Using photo based data collection techniques to enhance participation and dissemination |
Diane McDonald, Centre for Programme Evaluation; Rosalind Hurworth, University of Melbourne; Pamela St Leger, University of Melbourne |
| Tuesday 31 August |
8.30am to 12.30pm |
An Introduction to the Most significant Change – uses and aplications of the technique |
Tracey Delaney, Consultant |
| Tuesday 31 August |
8.30am to 12.30pm |
(Re)Discovering How Applied Systems Theory Improves Evaluation Planning and Performance Reporting when Working with Multi-Level and Multi-Programme Interventions |
Jerome Winston, CeDRE |
| Tuesday 31 August |
1.30pm to 5.30pm |
Evaluative Reasoning, Evidence & Use |
Thomas Schwandt, University of Illinois |
| Tuesday 31 August |
1.30pm to 5.30pm |
Designing improved performance measurement systems |
Graham Smith, Numerical Advantage Pty. Ltd. |
It’s possible to sign up for a workshop without registering for the full conference, but the conference itself is likely to be a great one!
Check out the list of keynote speakers:
- Roger Maaka, an academic who has studied and advised on indigenous issues internationally.
- Donna Mertens is past president of the American Evaluation Association and editor of the Journal of Mixed Methods Research, as well as a professor at Gallaudet University in Washington, DC.
- Thomas Schwandt is a professor at the University of Illinois at Urbana-Champaign and a member of a standing committee of the National Research Council of the US National Academies of Sciences.
- Elliot Stern is Professor of Evaluation Research at Lancaster University and the current editor of Evaluation: the international journal of theory, research and practice.
- Jane Davidson, formerly the Associate Director of the Evaluation Center at Western Michigan University, Jane now heads an evaluation consulting firm in Auckland, New Zealand.
What: Strategic Policy Evaluation: Answering macro-level cross-project questions
Presenters: Nan Wehipeihana & E. Jane Davidson
Hosted by: anzea Regional Symposium
Location: Te Puni Kōkiri House, Wellington
When: May 5, 2010, time TBA
How to sign up: Registrations are due Thurs April 21st! (click link to open PDF brochure & registration form)
Abstract:
Policy-relevant, strategic evaluation asks (and answers) questions that go beyond the evaluation of a single initiative. It addresses such questions as: (1) What is the value of a particular policy initiative as a contributor to strategic policy outcomes? (2) How well does each policy initiative fit with and complement the other initiatives that make up the strategic policy mix? Are there any unnecessary overlaps? (3) What is the collective value of the suite of initiatives that have been deployed to achieve a particular strategic outcome? (4) Have we got the right mix to deliver on the key outcomes? (5) Which approaches to achieving key long-term outcomes are working most cost-effectively for whom, under what conditions, and why? This session will outline five key elements needed to position and effectively engage with the evaluation function so that it delivers high quality, truly useful information to calibrate macro-level policy – and to incisively inform Ministers on policy effectiveness.
Want us to keep you posted?
Please subscribe to the site feed to receive updates about workshops, presentations, publications and keynotes – plus Real Evaluation ideas and techniques. See top right corner for subscription options (email updates highly recommended).
What: Actionable Evaluations: A Bootcamp for Commissioners, Managers and Evaluators
Presenters: E. Jane Davidson & Nan Wehipeihana
Hosted by: anzea Regional Symposium
Location: Te Puni Kōkiri House, Wellington
When: May 5, 2010, time TBA
How to sign up: Registrations are due Thurs April 21st! (click link to open PDF brochure & registration form)
Abstract:
Let’s face it. Every year there are evaluations that bitterly disappoint client organisations by failing to produce actionable answers to important evaluative questions. Weak evaluations frequently lack incisive evaluation questions; employ the wrong methodology for the questions; get lost in the details; skip over the crucial ‘values’ step and therefore can’t draw evaluative conclusions; uncritically accept stated objectives as the only evaluative criteria; focus only on the average effect; fail to adequately triangulate and transparently weave sources of evidence; toss causation into the ‘too hard basket’ while still claiming to have documented ‘outcomes’; and fail to clearly communicate findings. This session will help managers and commissioners of evaluations see the possibilities so they can become informed and demanding consumers of real, genuine, actionable evaluation. You will get important pointers for writing RFPs and managing selection processes with a better chance of attracting a shorter list of high quality proposals; and for effectively managing the evaluation so as to maximise the chances of getting clear, authentic, actionable answers to high priority questions. Evaluators will also be interested in hearing about how their work can better meet the needs of clients and other stakeholders.
Want us to keep you posted?
Please subscribe to the site feed to receive updates about workshops, presentations, publications and keynotes – plus Real Evaluation ideas and techniques. See top right corner for subscription options (email updates highly recommended).
What: Outcomes, impacts and causal attribution – affordable, practical, feasible options for the real world
Hosted by: anzea Regional Symposium
Location: Nga Wai o Horotiu (Auckland University of Technology Marae), Auckland
When: May 21, half day, time TBA
How to sign up: Registrations are due May 12, 2010 (click link to open PDF brochure & registration form)
Abstract:
If a reported “outcome†is not caused by a programme, it is not an outcome at all; it’s a coincidence. Simply measuring variables that may or may not be causally related to a programme (i.e. could just be coincidences – who knows?) doesn’t tell you anything about the quality or value of the programme, therefore it can’t be referred to as outcome evaluation – it’s just measurement.
Isn’t causal attribution heinously expensive, almost never feasible, and doesn’t it require some form of experimental design? Not necessarily. In this interactive seminar, Jane will use case examples to illustrate eight strategies for inferring (or ruling out) causal links between programmes and suspected outcomes: (1) Ask those who have observed or experienced the causal effect, (2) Check if the content of the intervention matches the nature of the outcome; (3) Look for distinctive effect patterns (modus operandi method), (4) Check whether the timing of outcomes makes sense, (5) Examine the relationship between program “dose†and “responseâ€, (6) Use a comparison or control, (7) Control statistically for extraneous variables, and (8) Identify and check the causal mechanisms. These strategies are outlined in Jane’s (2004) book, “Evaluation Methodology Basics: The nuts and bolts of sound evaluation†(Sage).
Want us to keep you posted?
Please subscribe to the site feed to receive updates about workshops, presentations, publications and keynotes – plus Real Evaluation ideas and techniques. See top right corner for subscription options (email updates highly recommended).
|