Taking Stock of Contribution Analysis
Reflecting on the Past to Inform the Future
DOI:
https://doi.org/10.3138/cjpe-2024-0044Keywords:
causation, contribution analysis, research on evaluation, theory-based evaluation, theory of changeAbstract
Over the past two decades, contribution analysis (CA) has attracted considerable attention as a theory-based evaluation a roach. Over the years, CA has been a lied across a broad and still broadening range of settings and contexts. Informed by a comprehensive review of published articles on CA, this article examines the main methodological and theoretical developments of CA. The findings suggest that developments over the past 10 years have focused on methodological improvements and explication of the theoretical foundation for making contribution claims. The authors suggest avenues for further development of CA, including broadening the empirical base of CA applications and developing quality standards for CA.
References
Anguko, A. (2019). Application of contextual analysis to evaluation methodologies in Africa: The case of contribution analysis (eVALUation Matters Fourth Quarter 2019). Independent Development Evaluation, African Development Bank Group. https://shorturl.at/zI5wm
Befani, B., & Mayne, J. (2014). Process tracing and contribution analysis: A combined approach to generative causal inference for impact evaluation. IDS Bulletin, 45(6), 17–36. https://doi.org/10.1111/1759-5436.12110
Befani, B., Rees, C., Varga, L., & Hills, D. (2016). Testing contribution claims with Bayesian updating (Note Number 2.1). Centre for the Evaluation of Complexity Across the Nexus, University of Surrey.
Befani, B., & Stedman-Bryce, G. (2017). Process tracing and Bayesian updating for impact evaluation. Evaluation, 23(1), 42–60. https://doi.org/10.1177/1356389016654584
Biggs, J. S., Farrell, L., Lawrence, G., & Johnson, J. K. (2014). A practical example of Contribution Analysis to a public health intervention. Evaluation, 20(2), 214–229. https://doi.org/10.1177/1356389014527527
Brousselle, A., & Buregeya, J.-M. (2018). Theory-based evaluations: Framing the existence of a new theory in evaluation and the rise of the 5th generation. Evaluation, 24(2), 153–168. https://doi.org/10.1177/1356389018765487
Buckley, A. P. (2016). Using contribution analysis to evaluate small & medium enterprise support policy. Evaluation, 22(2), 129–148. https://doi.org/10.1177/1356389016638625
Budhwani, S., & McDavid, J. (2017). Contribution analysis: Theoretical and practical challenges and prospects for evaluators. Canadian Journal of Program Evaluation, 32(1), 1–24. https://doi.org/10.3138/cjpe.31121
Buetti, D., Fitzgerald, M., Barber, C., Labelle, P. R., Bourgeois, I., Aubry, T., Cameron, E., & Kendall, C. E. (2024). The use of contribution analysis in evaluating health interventions: A scoping review. Evaluation & the Health Professions. Advanced online publication. https://doi.org/10.1177/01632787241281745
Buregeya, J. M., Brousselle, A., Nour, K., & Loignon, C. (2017). Comment évaluer les effets des évaluations d’impact sur la santé: le potentiel de l’analyse de contribution [How to evaluate the effects of health impact assessments: The potential of contribution analysis]. Canadian Journal of Program Evaluation, 32(1), 25–45. https://doi.org/10.3138/cjpe.31151
Buregeya, J. M., Loignon, C., & Brousselle, A. (2020). Contribution analysis to analyze the effects of the health impact assessment at the local level: A case of urban revitalization. Evaluation and Program Planning, 79, 101746. https://doi.org/10.1016/j.evalprogplan.2019.101746
Cook, A., Morton, S., & Henderson, F. (2023). Interrogating assumptions about the relationship between service providers and recipients: Learning from a new service for survivors of in care abuse. Evaluation and Program Planning, 97, 102260. https://doi.org/10.1016/j.evalprogplan.2023.102260
Coryn, C. L., Noakes, L. A., Westine, C. D., & Schröter, D. C. (2011). A systematic review of theory-driven evaluation practice from 1990 to 2009. American Journal of Evaluation, 32(2), 199–226. https://doi.org/10.1177/1098214010389321
Delahais, T. (2024). Setting the stage for contribution claims. In A. Koleros, M.-H. Adrien, & T. Tyrrell (Eds.), Theories of change in reality (pp. 15–24). Routledge.
Delahais, T., & Lacouette-Fougère, C. (2019). Try again. Fail again. Fail better. Analysis of the contribution of 65 evaluations to the modernisation of public action in France. Evaluation, 25(2), 131–148. https://doi.org/10.1177/1356389018823237
Delahais, T., & Toulemonde, J. (2012). Applying contribution analysis: Lessons from five years of practice. Evaluation, 18(3), 281–293. https://doi.org/10.1177/1356389012450810
Delahais, T., & Toulemonde, J. (2017). Making rigorous causal claims in a real-life context: Has research contributed to sustainable forest management? Evaluation, 23(4), 370–388. https://doi.org/10.1177/1356389017733211
Dinh, K., Worth, H., Haire, B., & Hong, K. T. (2019). Confucian evaluation: Reframing contribution analysis using a Confucian lens. American Journal of Evaluation, 40(4), 562–574. https://doi.org/10.1177/1098214018813008
Downes, A., Novicki, E., & Howard, J. (2019). Using the Contribution Analysis approach to evaluate science impact: A case study of the National Institute for Occupational Safety and Health. American Journal of Evaluation, 40(2), 177–189. https://doi.org/10.1177/1098214018767046
Dybdal, L., Nielsen, S. B., & Lemire, S. (2010). Contribution analysis applied: Reflections on scope and methodology. Canadian Journal of Program Evaluation, 25(2), 29–57. https://doi.org/10.3138/cjpe.25.002
Greenhalgh, T., Pawson, R., Wong, G., Westhorp, G., Greenhalgh, J., Manzano, A., & Jagosh, J. (2016). Quality standards for realist evaluation. For evaluators and peer-reviewers. The RAMESES II Project. http://ramesesproject.org/media/RE_Quality_Standards_for_evaluators_and_peer_reviewers.pdf
Heimlich, J. E., Wasserman, D., Tingley, K., Roberts, S-J., Aloisio, J. (2022). An influence among influences: The perceived influence contribution scale development and use. Evaluation and Program Planning, 92, 102091. https://doi.org/10.1016/j.evalprogplan.2022.102091
Hofmann, C.-A., Roberts, L., Shoham, J., & Harvey, P. (2004). Measuring the impact of humanitarian aid: A review of current practice, HPG Report 17. Overseas Development Institute. https://odi.org/en/about/our-work/measuring-the-impact-of-humanitarian-aid-a-review-of-current-practice/
Junge, K., Cullen, J., & Iacopini, G. (2020). Using contribution analysis to evaluate large-scale, transformation change processes. Evaluation, 26(2), 227–245. https://doi.org/10.1177/1356389020912270
Kane, R., Levine, C., Orians, C., & Reinelt, C. (2021). Contribution analysis: A promising method for assessing advocacy’s impact. New Directions for Evaluation, 2021(171), 45–57. https://doi.org/10.1002/ev.20471
Koleros, A., & Mayne, J. (2019). Using actor-based theories of change to conduct robust contribution analysis in complex settings. Canadian Journal of Program Evaluation, 33(3), 292–315. https://doi.org/10.3138/cjpe.52946.
Kotvojs, F., & Shrimpton, B. (2007). Contribution analysis: A new approach to evaluation in international development. Evaluation Journal of Australasia, 7(1), 27–35. https://doi.org/10.1177/1035719X0700700105
Leeuw, F. L. (2012). Linking theory-based evaluation and contribution analysis: Three problems and a few solutions. Evaluation, 18(3), 348–363. https://doi.org/10.1177/1356389012452051
Leeuw, F. L. (2023). John Mayne and rules of thumb for contribution analysis: A comparison with two related approaches. Canadian Journal of Program Evaluation, 37(3), 403–421. https://doi.org/10.3138/cjpe.75448
Lemire, S., Christie, C. A., & Nielsen, S. B. (2021). Mending the theory gap in evaluation. In M. Palenberg & A. Paulson (Eds.), The realpolitik of evaluation. Why demand and supply rarely intersect (pp. 136–150). Routledge. https://doi.org/10.4324/9781003005162
Lemire, S., Kwako, A. J., Nielsen, S. B., Christie, C. A., Donaldson, S., & Leeuw, F. (2020). What is this thing called a mechanism? Findings from a review of published realist evaluations. New Directions for Evaluation, 2020(167), 73–86. https://doi.org/10.1002/ev.20428
Lemire, S. T., Nielsen, S. B., & Dybdal, L. (2012). Making contribution analysis work: A practical framework for handling influencing factors and alternative explanations. Evaluation, 18(3), 294–309. https://doi.org/10.1177/1356389012450654
Lemire, S., Porowski, A., & Mumma, K. (2023). How we model matters—visualizing program theories. Abt method guide. Abt Associates. https://www.abtglobal.com/files/insights/white-papers/2024/method-guide-visualizing-program-theory.pdf
Lemire, S., Whynot, J., & Montague, S. (2019). How we model matters: A manifesto for the next generation of program theorizing. Canadian Journal of Program Evaluation, 33(3), 414–433. https://doi.org/10.3138/cjpe.53070
Lipski, U., & Ignagni, E. (2001). Evaluation of nutritional interventions: A background document (Report #3). The Prevention Unit, Division of Preventive Oncology Cancer Care Ontario.
Mackay, R., Horton, D., Dupleich, L., & Anderson, A. (2002). Evaluating organizational capacity development. Canadian Journal of Program Evaluation, 17(2), 121–150. https://doi.org/10.3138/cjpe.17.007
Mackie, J. L. (1974). The cement of the universe: A study of causation. Oxford University Press.
Mayne, J. (1999). Addressing attribution through contribution analysis: Using performance measures sensibly [Discussion paper]. Office of the Auditor General of Canada. http://dsp-psd.pwgsc.gc.ca/Collection/FA3-31-1999E.pdf
Mayne, J. (2001). Addressing attribution through contribution analysis: Using performance measures sensibly. Canadian Journal of Program Evaluation, 16(1), 1–24. https://doi.org/10.3138/cjpe.016.001
Mayne, J. (2008). Contribution Analysis: An approach to exploring cause and effect (No. 15; ILAC Brief). ILAC. http://www.pointk.org/resources/files/Contribution_Analysis.pdf
Mayne, J. (2011). Addressing cause and effect in simple and complex settings through contribution analysis. In K. Forss, M. Marra, & R. Schwartz, (Eds.), Evaluating the complex. Attribution, contribution, and beyond. Transaction Publishers.
Mayne, J. (2012). Contribution analysis: Coming of age? Evaluation, 18(3), 270–280. https://doi.org/10.1177/1356389012451663
Mayne, J. (2015). Useful theory of change models. Canadian Journal of Program Evaluation, 30(2), 119–142. https://doi.org/10.3138/cjpe.30.2.142
Mayne, J. (2017). Theory of change analysis: Building robust theories of change. Canadian Journal of Program Evaluation, 32(2), 155–173. https://doi.org/10.3138/cjpe.31122
Mayne, J. (2019a). Revisiting contribution analysis. Canadian Journal of Program Evaluation, 34(2), 171–191. https://doi.org/10.3138/cjpe.68004
Mayne, J. (2019b). Assessing the relative importance of causal factors (CDI Practice Paper 21). Centre for Development Impact.
Mayne, J. (2019c). A brief on contribution analysis: Principles and concepts. https://www.researchgate.net/publication/331159324_A_BRIEF_ON_CONTRIBUTION_ANALYSIS_PRINCIPLES_AND_CONCEPTS
Mayne, J. (2023). Assumptions in theories of change. Evaluation and Program Planning, 98, 102276. https://doi.org/10.1016/j.evalprogpla .2023.102276
McDavid, J. (2023). Enduring themes in John Mayne’s work: Implications for evaluation practice. Canadian Journal of Program Evaluation, 37(3), 355–370. https://doi.org/ 10.3138/cjpe.75429
Michie, S., Van Stralen, M. M., & West, R. (2011). The behaviour change wheel: A new method for characterising and designing behaviour change interventions. Implementation Science, 6(1), 42. https://doi.org/10.1186/1748-5908-6-42
Morton, S. (2015). Progressing research impact assessment: A ‘contributions’ approach. Research Evaluation, 24(4), 405–419. https://doi.org/10.1093/reseval/rvv016
Nakrošis, V. (2014). Theory-based evaluation of capacity-building interventions. Evaluation, 20(1), 134–150. https://doi.org/10.1177/1356389013517763
Newbill, S. L. (2023). Contribution analysis of a complex system during disruptions. Canadian Journal of Program Evaluation, 38(2), 374–395. https://doi.org/10.3138/cjpe.75803
Newcomer, K. E., & Schreirer, M. A. (2001). Using evaluation to support performance management: A guide for federal managers. The Price Waterhouse Coopers Endowment for The Business of Government. http://www.businessofgovernment.org/pdfs/NewcomerReport.pdf
Nielsen, S. B., & Lemire, S. (2025). Nothing as practical as an analytical strategy in realist evaluation. Findings and recommendations from a comprehensive review. In A. Manzano & E. Williams (Eds.), Realist evaluation: Principles and practice (pp. 42–59). Routledge. https://doi.org/10.4324/9781003457077
Nielsen, S. B., Jaspers, S. Ø., & Lemire, S. (2023). The curious case of the realist trial: Methodological oxymoron or unicorn? Evaluation, 30(1), 120–137. https://doi.org/ 10.1177/13563890231200291
Nielsen S. B., Lemire, S., & Montague, S. (2023a). Remembering John Mayne—A practical thinker and a thinking practitioner. Canadian Journal of Program Evaluation, 37(3), 297–305. https://doi.org/10.3138/cjpe.75515
Nielsen S. B., Lemire, S., & Montague, S. (2023b). Using evaluative information sensibly: The enduring contributions of John Mayne. Canadian Journal of Program Evaluation, 37(3), 439–460. https://doi.org/10.3138/cjpe.75444
Nielsen, S. B., Tangsig, S., & Lemire, S. (2023). Mapping the contributions of John Mayne: Bridging the gaps between evaluation, auditing, and performance monitoring. Canadian Journal of Program Evaluation, 37(3), 306–323. https://doi.org/10.3138/cjpe.75441
Nowell, L. S., Norris, J. M., White, D. E., & Moules, N. J. (2017). Thematic analysis: Striving to meet the trustworthiness criteria. International Journal of Qualitative Methods, 16(1). https://doi.org/10.1177/1609406917733847
Palenberg, M. (2023). Causal claims in Contribution Analysis. Canadian Journal of Program Evaluation, 37(3), 389–402. https://doi.org/10.3138/cjpe.75428
Patton, M. Q. (2012). A utilization-focused approach to contribution analysis. Evaluation, 18(3), 364–377. https://doi.org/10.1177/1356389012449523
Ragin, C. C. (2000). Fuzzy-set social science. University of Chicago Press.
Romo Ramos, Y. J. (2021). Contribution tracing for organisational-wide planning, improved collaboration and evaluative thinking. Evaluation, 27(4), 510–526. https://doi.org/10.1177/13563890211020333
Sridharan, S., & Nakaima, A. (2012). Towards an evidence base of theory-driven evaluations: Some questions for proponents of theory-driven evaluation. Evaluation, 18(3), 378–395. https://doi.org/10.1177/1356389012453289
Sridharan, S., & Nakaima, A. (2023). Learning from experiences of evaluators implementing theory-driven evaluations in diverse settings: Building on the contributions of John Mayne. Evaluation and Program Planning, 97, 102257. https://doi.org/10.1016/j.evalprogplan.2023.102257
Sridharan, S, Nakaima, A., & Pereira, A. (2023). Causality and complexity in evaluating equity interventions: Conceptual issues that need to be addressed in theory-driven evaluation approaches. Canadian Journal of Program Evaluation, 37(3), 371–388. https://doi.org/10.3138/cjpe.75430
Ton, G., Mayne, J., Delahais, T., Morell, J., Befani, B., Apgar, M., & O’Flynn, P. (2019). Contribution analysis and estimating the size of effects: Can we reconcile the possible with the impossible? (CDI Practice Paper 20). Centre for Development Impact.
Ton, G., van Rijn, F., & Pamuk, H. (2023). Evaluating the impact of business coaching programmes by taking perceptions seriously. Evaluation, 29(1), 73–90. https://doi.org/10.1177/13563890221137611
Vaessen, J., & Raimondo, E. (2012). Making sense of impact: A methodological framework for assessing the impact of prizes. Evaluation, 18(3), 330–347. https://doi.org/10.1177/1356389012450655
Wimbush, E., Montague, S., & Mulherin, T. (2012). Applications of contribution analysis to outcome planning and impact evaluation. Evaluation, 18(3), 310–329. https://doi.org/10.1177/1356389012452052
Wong, G., Greenhalgh, T., Westhorp, G., & Pawson, R. (2014). Development of methodological guidance, publication standards, and training materials for realist and meta-narrative reviews: The RAMESES (Realist And Meta-narrative Evidence Syntheses: Evolving Standards) project. Health Services and Delivery Research, 2(30). https://doi.org/10.3310/hsdr02300
Wong, G., Westhorp, G., Greenhalgh, J., Manzano, A., Jagosh, J., & Greenhalgh, T. (2017). Quality and reporting standards, resources, training materials and information for realist evaluation: The RAMESES II project. Health Services and Delivery Research, 5(28). https://doi.org/10.3310/hsdr05280
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Bohni Nielsen & Lemire
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Authors contributing to The Canadian Journal of Program Evaluation agree to release their articles under the Creative Commons Attribution-Noncommercial 4.0 (CC-BY-NC) license. This licence allows this work to be copied, distributed, remixed, transformed, and built upon for any purpose provided that appropriate attribution is given, a link is provided to the license, and changes made were indicated.
Authors retain copyright of their work and grant the journal right of first publication.
Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
