Toward Learning from Change Pathways
Reviewing Theory of Change and Its Discontents
DOI:
https://doi.org/10.3138/cjpe.69535Keywords:
impact pathway, logic model, program theory, research on evaluation, theory-driven evaluation, theory of changeAbstract
The concept of Theory of Change (ToC) is well established in the evaluation literature, underpinning substantial research and practice efforts. However, its ability to facilitate learning has been increasingly debated. The objective of this paper is to identify, characterize, and evaluate concerns over the use of ToCs based on a review of relevant studies. Seven concerns are found: distinguishing ToCs from other evaluation approaches, conceptual vagueness, under-developed ToCs, under-contribution to theoretical knowledge, uncertainty in stakeholder engagement, neglecting context, and overlooking complexity. Priority areas for improvement include integrating context and complexity throughout the ToC process, contributing theoretical knowledge, and engaging stakeholders as appropriate.
References
Amundsen, C., & D’Amico, L. (2019). Using Theory of Change to evaluate socially-situated, inquiry-based academic professional development. Studies in Educational Evaluation, 61, 196-208. https://doi.org/10.1016/j.stueduc.2019.04.002
Apgar, J. M., Allen, W., Albert, J., Douthwaite, B., Ybarnegaray, R. P., & Lunda, J. (2017). Getting beneath the surface in program planning, monitoring and evaluation: Learning from use of participatory action research and theory of change in the CGIAR Research Program on Aquatic Agricultural Systems. Action Research, 15(1), 15 – 34. https://doi.org/10.1177/1476750316673879
Archibald, T., Sharrock, G., Buckley, J., & Cook, N. (2016). Assumptions, conjectures, and other miracles: The application of evaluative thinking to theory of change models in community development. Evaluation and Program Planning, 59, 119 – 127. https://doi.org/10.1016/j.evalprogplan.2016.05.015. Medline:27324286
Armitage, D., Arends, J., Barlow, N. L., Closs, A., Cloutis, G. A., and Cowley, M. S., ... Wiens, C. (2019). Applying a “theory of change” process to facilitate transdisciplinary sustain-ability education. Ecology and Society, 24(3). https://doi.org/10.5751/es-11121-2403203
Belcher, B., Suryadarma, D., & Halimanjaya, A. (2017). Evaluating policy-relevant research: Lessons from a series of theory-based outcomes assessments. Palgrave Communications, 3(1). https://doi.org/10.1057/palcomms.2017.17
Blamey, A., & Mackenzie, M. (2007). Theories of change and realistic evaluation: Peas in a pod or apples and oranges? Evaluation, 13(4), 439 – 455. https://doi.org/10.1177/1356389007082129
Breuer, E., Lee, L., De Silva, M., & Lund, C. (2016). Using theory of change to design and evaluate public health interventions: A systematic review. Implementation Science, 11(1). https://doi.org/10.1186/s13012-016-0422-6. Medline:27153985
Connell, J. P., & Kubisch, A. C. (1998). Applying a theory of change approach to the evaluation of comprehensive community initiatives: Progress, prospects, and problems. Theory Measurement and Analysis, 2, 15 – 44.
Davies, R. (2018). Representing theories of change: Technical challenges with evaluation consequences. Journal of Development Effectiveness, 10(4), 438 – 461. https://doi.org/10.1080/19439342.2018.1526202
Davis, R., Campbell, R., Hildon, Z., Hobbs, L., & Michie, S. (2015). Theories of behaviour and behaviour change across the social and behavioural sciences: A scoping review. Health Psychology Review, 9(3), 323 – 344. https://doi.org/10.1080/17437199.2014.941722. Medline:25104107
De Silva, M. J., Breuer, E., Lee, L., Asher, L., Chowdhary, N., Lund, C., & Patel, V. (2014). Theory of Change: A theory-driven approach to enhance the Medical Research Council’s framework for complex interventions. Trials, 15(1). https://doi.org/10.1186/17456215-15-267. Medline:24996765
Dhillon, L., & Vaca, S. (2018). Refining theories of change. Journal of MultiDisciplinary Evaluation, 14(30), 64 – 87.
Douthwaite, B., & Hoffecker, E. (2017). Towards a complexity-aware theory of change for participatory research programs working within agricultural innovation systems. Agricultural Systems, 155, 88 – 102. https://doi.org/10.1016/j.agsy.2017.04.002
Douthwaite, B., Mayne, J., McDougall, C., & Paz-Ybarnegaray, R. (2017). Evaluating complex interventions: A theory-driven realist-informed approach. Evaluation, 23(3), 294 – 311. https://doi.org/10.1177/1356389017714382
DuBow, W. M., & Litzler, E. (2019). The development and use of a theory of change to align programs and evaluation in a complex, national initiative. American Journal of Evaluation, 40(2), 231 – 248. https://doi.org/10.1177/1098214018778132
Forsyth, T. (2018). Is resilience to climate change socially inclusive? Investigating theories of change processes in Myanmar. World Development, 111, 13 – 26. https://doi.org/10.1016/j.worlddev.2018.06.023
Freer, G., & Lemire, S . (2019). Can’t see the wood for the logframe: Integrating logframes and theories of change in development evaluation. Canadian Journal of Program Evaluation, 33(3), 336 – 353. https://doi.org/10.3138/cjpe.53007
Funnell, S. C., & Rogers, P. J. (2011). Purposeful program theory: Effective use of theories of change and logic models. San Francisco, CA: Jossey-Bass.
Ghate, D. (2018). Developing theories of change for social programmes: Co-producing evidence-supported quality improvement. Palgrave Communications, 4(1). https://doi.org/10.1057/s41599-018-0139-z
Gilissen, J., Pivodic, L., Gastmans, C., Stichele, R. V., Deliens, L., Breuer, E., & Van den Block, L. (2018). How to achieve the desired outcomes of advance care planning in nursing homes: A theory of change. BMC Geriatrics, 18, 47. https://doi.org/10.1186/s12877-018-0723-5
Green, D. (2017). Theories of change for promoting empowerment and accountability in fragile and conflict-affected settings. Brighton, England: Institute of Development Studies.
Hawe, P. (2015). Lessons from complex interventions to improve health . Annual Review of Public Health, 36(1), 307 – 323. https://doi.org/10.1146/annurev-publhealth031912-114421. Medline:25581153
Hivos. (2015). Theory of change thinking in practice. The Hague, Netherlands: Hivos.
James, C. (2011). Theory of change review: A report commissioned by Comic Relief. London, England: Comic Relief.
Leeuw, F. L., & Donaldson, S. I. (2015). Theory in evaluation: Reducing confusion and encouraging debate. Evaluation, 21(4), 467 – 480. https://doi.org/10.1177/1356389015607712
Maini, R., Mounier-Jack, S., & Borghi, J. (2018). How to and how not to develop a theory of change to evaluate a complex intervention: Reflections on an experience in the Democratic Republic of Congo. BMJ Global Health, 3(1), e000617. https://doi.org/10.1136/bmjgh-2017-000617. Medline:29515919
Makowiecka, K., Marchant, T., Betemariam, W., Chaturvedi, A., Jana, L., Liman, A., ... Schellenberg, J. (2019). Characterising innovations in maternal and newborn health based on a common theory of change: Lessons from developing and applying a characterisation framework in Nigeria, Ethiopia and India. BMJ Global Health, 4(4), e001405. https://doi.org/10.1136/bmjgh-2019-001405. Medline:31406587
Maru, Y. T., Sparrow, A., Butler, J. R. A., Banerjee, O., Ison, R., Hall, A., & Carberry, P. (2018). Towards appropriate mainstreaming of “Theory of Change” approaches into agricultural research for development: Challenges and opportunities. Agricultural Systems, 165, 344 – 353. https://doi.org/10.1016/j.agsy.2018.04.010. Medline:32287945
Mason, P., & Barnes, M. (2007). Constructing theories of change: Methods and sources. Evaluation, 13(2), 151 – 170. https://doi.org/10.1177/135638900707522
Maye, D., Enticott, G., & Naylor, R. (2019). Theories of change in rural policy evaluation. Sociologia Ruralis, 60(1), 198 – 221. https://doi.org/10.1111/soru.12269
Mayne, J. (2015). Useful theory of change models. Canadian Journal of Program Evaluation, 30(2), 119 – 142. https://doi.org/10.3138/cjpe.230
Mayne, J. (2017). Theory of change analysis: Building robust theories of change. Canadian Journal of Program Evaluation, 32(2), 155 – 173. https://doi.org/10.3138/cjpe.31122
Mayne, J., & Johnson, N. (2015). Using theories of change in the CGIAR Research Program on Agriculture for Nutrition and Health. Evaluation, 21(4), 407 – 428. https://doi.org/10.1177/1356389015605198
Moore, G. F., & Evans, R. E. (2017). What theory, for whom and in which context? Reflections on the application of theory in the development and evaluation of complex population health interventions. SSM—Population Health, 3, 132 – 135. https://doi.org/10.1016/j.ssmph.2016.12.005. Medline:29302610
Nilsen, P. (2015). Making sense of implementation theories, models and frameworks. Implementation Science, 10(1), 53. https://doi.org/10.1186/s13012-015-0242-0. Medline:25895742
Nkwake, A. M. (2013). Working with assumptions in international development program evaluation. New York, NY: Springer. https://doi.org/10.1007/978-1-4614-4797-9
Omore, A., Kidoido, M., Twine, E., Kurwijila, L., O’Flynn, M., & Githinji, J. (2019). Using “theory of change” to improve agricultural research: Recent experience from Tanzania. Development in Practice, 29(7), 898 – 911. https://doi.org/10.1080/09614524.2019.1641182
Paina, L., Wilkinson, A., Tetui, M., Ekirapa-Kiracho, E., Barman, D., Ahmed, T., ... Bennett, S. (2017). Using Theories of Change to inform implementation of health systems research and innovation: Experiences of future health systems consortium partners in Bangladesh, India and Uganda. Health Research Policy and Systems, 15(S2). https://doi.org/10.1186/s12961-017-0272-y. Medline:29297374
Pawson, R. (2013). The science of evaluation: A realist manifesto. London, England: SAGE.
Prinsen, G., & Nijhof, S. (2015). Between logframes and theory of change: Reviewing debates and a practical experience. Development in Practice, 25(2), 234 – 246. https://doi.org/10.1080/09614524.2015.1003532
Rolfe, S. (2019). Combining Theories of Change and Realist Evaluation in practice: Lessons from a research on evaluation study. Evaluation, 25(3), 294 – 316. https://doi.org/10.1177/1356389019835229
Sarma, H., Budden, A., Luies, S. K., Lim, S. S., Shamsuzzaman, M., Sultana, T., ... Uddin, M. J. (2019). Implementation of the World’s largest measles-rubella mass vaccination campaign in Bangladesh: A process evaluation. BMC Public Health, 19(1), 925. https://doi.org/10.1186/s12889-019-7176-4. Medline:31291922
Shiell, A., Hawe, P., & Gold, L. (2008). Complex interventions or complex systems? Implications for health economic evaluation. British Medical Journal, 336(7656), 1281 – 1283. https://doi.org/10.1136/bmj.39569.510521.ad. Medline:18535071
Stein, D., & Valters, C. (2012). Understanding theory of change in international development. London, England: Justice and Security Research Programme.
Thompson, G. H., & Moret, W. (2019). Building a theory of change for community development and HIV programming: The impact of social capital, stigma reduction and community-level changes on HIV-related health outcomes for orphans and vulnerable households in Mozambique. Community Development, 50(3), 332 – 351. https://doi.org/10.1080/15575330.2019.1606839
Valters, C. (2015). Theories of change: Time for a radical approach to learning in development. London, England: Overseas Development Institute.
Vogel, I. (2012). Review of the use of ‘theory of change’ in international development. London, England: UK Department for International Development.
Weiss, C. H. (1995). Nothing as practical as good theory: Exploring theory-based evaluation for comprehensive community initiatives for children and families. New Approaches to Evaluating Community Initiatives: Concepts, Methods, and Contexts, 7, 65 – 92. https://doi.org/10.1177/1356389003094007
Weiss, C. H. (1997). Theory-based evaluation: Past, present, and future. New Directions for Evaluation, 1997(76), 41 – 55. https://doi.org/10.1002/ev.1086
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2020 Steven Lam
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Authors contributing to The Canadian Journal of Program Evaluation agree to release their articles under the Creative Commons Attribution-Noncommercial 4.0 (CC-BY-NC) license. This licence allows this work to be copied, distributed, remixed, transformed, and built upon for any purpose provided that appropriate attribution is given, a link is provided to the license, and changes made were indicated.
Authors retain copyright of their work and grant the journal right of first publication.
Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
