printUse ctrl + p to print the page

QUESTION 4: How can ToC thinking help in monitoring and evaluation?

The ToC will assist you to have clarity on your outcome chain(s) and explains which strategies have been selected, why this set of strategies and not other strategies, and how they are expected to unfold. ToC’s often are revised and updated in the course of the programme life, taking into account lessons learned and opportunities for improvement of the ToC.

 

ToC and Monitoring

If you have developed the theory of change underpinning your programme or project as described under question 3: ToC, programme design and planning, this will be of great help to design and focus the M&E framework in an early stage of the design process and not in the early implementation phase as is often the case. The main actors, critical assumptions, intended outcomes and some key indicators will be available as a basis for the M&E framework (Kusters et al 2010). The sharper focus of the programme will facilitate the decision which result information is key to collect and who should be involved. As the ToC process enhances the understanding of stakes and stakeholders, this will assist in thinking through the utilisation of the M&E data and lessons and increase the consequence awareness.

 

Monitoring is usually understood as tracking progress against plans, milestones and what we expect to happen. With ToC we take a broader perspective, also looking at the problem the programme is addressing, the wider context and changes in the relationships between the main actors (process indicators) and unintended outcomes. Revisiting the assumptions that have been made at the start during programme implementation is of great importance to know if they prove to be valid. If not, it may be necessary to adapt the strategy, or review the ToC. ToC is helpful to not only measure outcomes but also to understand the role of your programme and other factors in contributing to outcomes.

 

ToC and Evaluation

Evaluation usually takes place either as a midterm review, towards the end of a programme or some time afterwards. A main objective of a midterm review is checking if the programme is contributing to the intended change in line with the underlying ToC and if the ToC needs to be revised. Formative (or process) evaluations would typically  also have an open eye to revise the ToC, whereas summative evaluations would study if lessons learned can be up-scaled or transferred to other domains of practice. ToC can help design evaluations of programmes that have complicated or complex aspects.

 

The results from monitoring (what has happened) feed into the evaluation, but the evaluation often will have a deeper look at why this has happened. ToC is helping to articulate the why element at design, planning and monitoring stages, which provides input for the evaluation. ToC can help to decide what are the main cause effect relations that need to be studied. Sometimes, in the absence of a ToC, the evaluation team could assist key partners to reconstruct the ToC of an ongoing programme.

 

References on ToC and M&E:

A. Funnell & Rogers 2011: Purposeful Program Theory (especially Ch 14)

B. Guijt & Woodhill 2002: IFAD Guide Managing for Impact in Rural Development. A guide for Project M&E;

C. Kusters et al 2010: Making Evaluations Matter

 

Contribution Analysis – An approach to exploring cause and effect

Contribution analysis is an approach to evaluation that can provide credible assessments of cause and effect in complex situations without doing experiments or RCTs. Here you can find a paper by John Mayne about this approach. The author states: “Verifying the theory of change that the programme is based on, and paying attention to other factors that may influence the outcomes, provides reasonable evidence about the contribution being made by the programme.” The paper offers a very concise and practical guide and clearly describes every step of the approach. An updated version of the approach by the same author can be found in the book “Evaluating the Complex: Attribution, Contribution, and Beyond”.

References on contribution analysis:

D. John Mayne (2008). Contribution analysis:An approach to exploring cause and effect, ILAC Brief 16.

E. This is a PowerPoint presentation summarising the ideas as presented in the ILAC Brief 16.

F. “Evaluating the Complex: Attribution, Contribution, and Beyond”, Comparative Policy Evaluation, Volume 18 (2011), by Kim Forss, Mita Marra, Robert Schwartz. Chapter 3: Contribution Analysis: Addressing cause and effect, by John Mayne. Using Contribution Analysis to Address Cause-Effect Questions: Theory and Concepts.


 

Tags: Articles