Enduring Themes in John Mayne’s Work
Implications for Evaluation Practice
DOI:
https://doi.org/10.3138/cjpe.75429Keywords:
John Mayne, contribution analysis, causality, evaluation practiceAbstract
This paper focuses on three enduring themes in John Mayne’s work. They are causality; balancing learning and accountability as meta-objectives for evaluations; and program complexity. These themes are all central in his development and elaboration of contribution analysis. Although his work was aimed at practitioners, over time, the sophistication of his approach to evaluation raises challenges for practitioners, particularly given the structure of the evaluation field. The paper concludes with a suggestion to make contribution analysis more accessible, taking advantage of the work done by contributors to the Checklist Project at the University of Western Michigan.
References
Aspinal, P., McKeough, D., Meyboom, P., & Nielsen, E. (1986). Task Force on Program Review. Ottawa: Supply and Services, Government of Canada.
Befani, B., & Mayne, J. (2014). Process tracing and contribution analysis: A combined approach to generative causal inference for impact evaluation. IDS bulletin, 45(6), 17–36.
Befani, B., Rees, C., Varga, L., & Hills, D. (2016). Testing contribution claims with Bayesian updating. Centre for the Evaluation of Complexity Across the Nexus (CECAN). Note Number 2.1. University of Surrey.
Befani, B., & Stedman-Bryce, G. (2017). Process tracing and Bayesian updating for impact evaluation. Evaluation, 23(1), 42–60.
Budhwani, S., & McDavid, J. C. (2017). Contribution analysis: Theoretical and practical challenges and prospects for evaluators. Canadian Journal of Program Evaluation, 32(1), 1–24.
Collier, D. (2011). Understanding process tracing. PS: Political Science & Politics, 44(4), 823–830.
Delahais, T., & Toulemonde, J. (2012). Applying contribution analysis: Lessons from five years of practice. Evaluation, 18(3), 281–293.
Dybdal, L., Nielsen, S. B., & Lemire, S. (2010). Contribution analysis applied: Reflections on scope and methodology. The Canadian Journal of Program Evaluation, 25(2), 29–57.
Fetterman, D. M., & Wandersman, A. (Eds.). (2005). Empowerment evaluation principles in practice. New York, NY: Guilford Press.
Funnell, S. C., & Rogers, P. J. (2011). Purposeful program theory: Effective use of theories of change and logic models. Hoboken, NJ: John Wiley & Sons.
Lahey, R. (2023). John Mayne and the origins of evaluation in the public sector in Canada: A shaping of both evaluation and the evaluator. Canadian Journal of Program Evaluation, 37(3), 340–354.
Lahey, R., Elliott, C., & Heath, S. (2018). The evolving market for systematic evaluation in Canada. New Directions for Evaluation, 160, 45–62.
Leeuw, F. L. (2023). John Mayne and rules of thumb for contribution analysis: A comparison with two related approaches. Canadian Journal of Program Evaluation, 37(3), 403–421.
Lemire, S., Nielsen, S. B., & Christie, C. A. (2018). Toward understanding the evaluation market and its industry—Advancing a research agenda. New Directions for Evaluation, 2018(160), 145–163.
Lemire, S. T., Nielsen, S. B., & Dybdal, L. (2012). Making contribution analysis work: A practical framework for handling influencing factors and alternative explanations. Evaluation, 18(3), 294–309.
Mathison, S. (2018). Does evaluation contribute to the public good?. Evaluation, 24(1), 113–119.
Mayne, J. (1986). In defense of program evaluation. Canadian Journal of Program Evaluation. 1(2), 97–102.
Mayne, J. (1999). Addressing attribution through contribution analysis: Using performance measures sensibly. Ottawa: Office of the Auditor General of Canada.
Mayne, J. (2001). Addressing attribution through contribution analysis: using performance measures sensibly. Canadian Journal of Program Evaluation, 16(1), 1–24.
Mayne, J. (2005). Managing for outcomes: Connecting programs to performance. PowerPoint presentation, University of Victoria, ADMN 537 seminar.
Mayne, J. (2007). Challenges and lessons in implementing results-based management. Evaluation, 13(1), 87–109.
Mayne, J. (2008a). Building an evaluative culture for effective evaluation and results management. ILAC Brief, 20, 1–4.
Mayne, J. (2008b). Contribution analysis: An approach to exploring cause and effect. ILAC Brief, 16, 1–4.
Mayne, J. (2011). Contribution analysis: Addressing cause and effect. In K. Forss, M. Marra, & R. Schwartz (Eds.), Evaluating the Complex (pp. 53–96). New Brunswick, NJ: Transaction.
Mayne, J. (2012). Contribution analysis: Coming of age? Evaluation, 18(3), 270–280.
Mayne, J. (2016). An example of ToC analysis. Working Paper. Retrieved from ResearchGate: https://www.researchgate.net/publication/305957815_An_Example_of_ToC_Analysis
Mayne, J. (2017). Theory of change analysis: Building robust theories of change. Canadian Journal of Program Evaluation, 32(2), 155–173.
Mayne, J. (2018). Linking evaluation to expenditure reviews: Not realistic nor a good idea. Canadian Journal of Program Evaluation, 32(3), 316–326.
Mayne, J. (2019). Revisiting contribution analysis. Canadian Journal of Program Evaluation, 34(2), 171–191.
McDavid, J., Brousselle, A., Shepherd, R. P., & Zussman, D. (2018). Linking evaluation and spending reviews: Challenges and prospects. Canadian Journal of Program Evaluation, 32(3), 297–304.
McDavid, J., & Henderson, K. (2021). Hard won lessons learned in the evaluation field: Implications for behavioural insights practice. Canadian Public Administration, 64(1), 7–25.
Miller, R. L., & Campbell, R. (2006). Taking stock of empowerment evaluation: An empirical review. American Journal of Evaluation, 27(3), 296–319.
Nielsen, S. B., & Montague, S. (2023). John Mayne’s Contribution to Results-Based Management Thinking: A Leadership Legacy. Canadian Journal of Program Evaluation, 37(3), 439–460.
Nielsen, S. B., Tangsig, S., & Lemire, S. (2023). Mapping the contributions of John Mayne: Bridging the gaps between evaluation, auditing, and performance monitoring. Canadian Journal of Program Evaluation, 37(3), 306–323.
Office of the Comptroller General. (1981a). Guide on the program evaluation function. Ottawa: Government of Canada.
Office of the Comptroller General. (1981b). Principles for the evaluation of programs by federal departments and agencies. Ottawa: Government of Canada.
Palenberg, M. (2023). Causal claims in contribution analysis. Canadian Journal of Program Evaluation, 37(3), 389–402.
Patton, M. Q. (2010). Developmental evaluation: Applying complexity concepts to enhance innovation and use. New York, NY: Guilford Press.
Pawson, R. (2002). Evidence-based policy: The promise of realist synthesis. Evaluation, 8(3), 340–358.
Pawson, R., & Tilley, N. (1997). Realistic evaluation. Thousand Oaks, CA: Sage Publications.
Pearl, J., & Mackenzie, D. (2018). The book of why: The new science of cause and effect. New York City: Basic Books.
Picciotto, R. (2015). Democratic evaluation for the 21st century. Evaluation, 21(2), 150–166.
Schmitt, J. (2020). The causal mechanism claim in evaluation: Does the prophecy fulfill? New Directions for Evaluation, 167, 11–26.
Segsworth, R. V. (1994). Downsizing and program evaluation: An assessment of the experience in the Government of Canada. In R. Bernier, & J. I. Gow (Eds.), A down-sized state? (pp. 249–260). Montreal: Presses de l’Université du Quebec.
Segsworth, R. V. (2005). Program evaluation in the government of Canada: Plus ca change … The Canadian Journal of Program Evaluation, 20(3), 175–197.
Stern, E., Stame, N., Mayne, J., Forss, K., Davies, R., & Befani, B. (2012). Broadening the range of designs and methods for impact evaluations. Department for International Development, Working Paper 38. London: UK Government.
The Evaluation Checklist Project. (2022). The Evaluation Centre, University of Western Michigan. Accessed February 20, 2022: https://wmich.edu/evaluation/checklists
Ton, G., Mayne, J., Delahais, T., Morell, J., Befani, B., Agpar, M., & O’Flynn, P. (2019). Contribution analysis and estimating the size of effects: Can we reconcile the possible with the impossible? Centre for Development Impact, Practice Paper. Brighton, UK: Institute of Development Studies.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2023 James C. McDavid
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Authors contributing to The Canadian Journal of Program Evaluation agree to release their articles under the Creative Commons Attribution-Noncommercial 4.0 (CC-BY-NC) license. This licence allows this work to be copied, distributed, remixed, transformed, and built upon for any purpose provided that appropriate attribution is given, a link is provided to the license, and changes made were indicated.
Authors retain copyright of their work and grant the journal right of first publication.
Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
