Bridging Evaluation Theory and Practice
The Contributions of John Mayne to Canadian Federal Evaluation
DOI:
https://doi.org/10.3138/cjpe.75457Keywords:
federal government, performance measurement, contribution analysis, evaluation functionAbstract
This paper provides a thematic review of Mayne’s contributions to Canadian federal evaluation in order to describe his influence on how the function has evolved over the past 40 years. Key themes explored in the paper include Mayne’s influence on the development of a results-based culture in federal departments and agencies, as well as his conceptual grounding of evaluation activities through theory-based evaluation approaches and, most notably, Contribution Analysis. We draw on government publications to link policy directions to Mayne’s work and we provide a prospective view on his continued influence on the function for years to come.
References
Alkin, M. C., & King, J. A. (2016). The historical development of evaluation use. American Journal of Evaluation, 37(4), 568–579.
Alkin, M. C., & King, J. A. (2017). Definitions of evaluation use and misuse, evaluation influence, and factors affecting use. American Journal of Evaluation, 38(3), 434–450.
Barrados, M. (2023). John Mayne’s contribution to performance audit. Canadian Journal of Program Evaluation, 37(3), 461–472.
Barrados, M., & Mayne, J. (2003). Can public sector organisations learn? OECD Journal of Budgeting, 3(3), 87–103.
Bourgeois, I. (2021). L’analyse documentaire. In I. Bourgeois (Ed.), Recherche sociale: de la problématique à la collecte de données (7th ed.) (pp. 339–356). Québec: Presses de l’Université du Québec.
Gauthier et al. (2010). The lay of the land: Evaluation practice in Canada in 2009. Canadian Journal of Program Evaluation, 24(1), 1–49.
Government of Canada (n.d.a). Guide to rapid impact evaluation. Retrieved from https://www.canada.ca/en/treasury-board-secretariat/services/audit-evaluation/evaluation-government-canada/guide-rapid-impact-evaluation.html
Government of Canada (n.d.b). Theory-based approaches to evaluation: Concepts and practices. Retrieved from https://www.canada.ca/en/treasury-board-secretariat/services/audit-evaluation/evaluation-government-canada/theory-based-approaches-evaluation-concepts-practices.html
Government of Canada (1981). Guide on the program evaluation function. Retrieved from https://www.tbs-sct.canada.ca/cee/pubs/guide1981-eng.asp
Government of Canada (1984). Principles for the evaluation of programs by the federal departments and agencies. Retrieved from https://www.tbs-sct.canada.ca/cee/pubs/principles-principes1984-eng.asp
Government of Canada (2001). Policy on Evaluation. Archived report available online: https://www.tbs-sct.canada.ca/pol/doc-eng.aspx?id=12309
Government of Canada (2003). The managing for results self-assessment tool. Ottawa: Treasury Board of Canada Secretariat, 2003.
Government of Canada (2004). Study of the evaluation function in the federal government. Archived report available online: https://www.canada.ca/en/treasury-board-secretariat/services/audit-evaluation/study-evaluation-function-federal-government.html
Government of Canada (2016a). Policy on results: What is evaluation. Retrieved from https://www.canada.ca/en/treasury-board-secretariat/services/audit-evaluation/evaluation-government-canada/policy-results-what-evaluation.html
Government of Canada (2016b). Policy on results. Retrieved from https://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=31300
Government of Canada (2016c). Directive on Results. Retrieved from https://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=31306
Government of Canada (2019). Integrating gender-based analysis plus into evaluation: A primer. Retrieved from https://www.canada.ca/en/treasury-board-secretariat/services/audit-evaluation/evaluation-government-canada/gba-primer.html
Lahey, R. (2023). John Mayne and the origins of evaluation in the public sector in Canada: A shaping of both evaluation and the evaluator. Canadian Journal of Program Evaluation, 37(3), 340–354.
Mayne, J. (1986). Ongoing program performance information systems and program evaluation in the government of Canada. Canadian Journal of Program Evaluation, 1(1), 29–37. https://evaluationcanada.ca/system/files/cjpe-entries/01-1-029.pdf
Mayne, J. (1994). Utilizing evaluation in organizations: The balancing act. In F. Leeuw, R. C. Rist, & R. Sonnichsen (Eds.), Can Governments Learn? (pp. 17–43). Routledge.
Mayne, J. (1997a). Accountability for program performance: A key to effective performance monitoring and reporting. In Monitoring performance in the public sector (1st ed., pp. 157–176). Routledge.
Mayne, J. (1997b). Effective performance monitoring: A necessary condition for public sector growth. In Monitoring performance in the public sector (1st ed.) (pp. 3–30). Routledge.
Mayne, J. (2001). Addressing attribution through contribution analysis: Using performance measures sensibly. Canadian Journal of Program Evaluation, 16(1), 1–24.
Mayne, J. (2004). Reporting on outcomes: Setting performance expectations and telling performance stories. Canadian Journal of Program Evaluation 19(1), 31–60.
Mayne, J. (2005). Ensuring quality for evaluation: Lessons from auditors. Canadian Journal of Program Evaluation, 20(1), 37–64.
Mayne, J. (2006a). Audit and evaluation in public management: Challenges, reforms, and different roles. Canadian Journal of Program Evaluation, 21(1), 11–45.
Mayne, J. (2006b). Performance studies: The missing link? Canadian Journal of Program Evaluation, 21(2), 201–208.
Mayne, J. (2007a). Challenges and lessons in implementing results-based management. Evaluation, 13(1), 87–109.
Mayne, J. (2007b). Evaluation for accountability: Reality or myth? In M. Bemelmans-Videc, J. Lonsdale & B. Perrin (Eds.), Making accountability work: Dilemmas for evaluation and for audit. Transaction.
Mayne, J. (2009). Results management: Can results evidence gain a foothold in the public sector. In O. Rieper, F. Leeuw & T. Ling (Eds.), The evidence book (pp. 117–149). Routledge.
Mayne, J. (2010). Building an evaluative culture: The key to effective evaluation and results management. The Canadian Journal of Program Evaluation, 24(2), 1–30.
Mayne, J. (2011). Contribution analysis: Addressing cause and effect. In R. Schwartz, K. Forss & M. Marra (Eds.), Evaluating the complex. Attribution, contribution and beyond (pp. 53–95). Transaction.
Mayne, J. (2012). Independence in evaluation and the role of culture. In J. Barbier & P. Hawkins (Eds.), Evaluation and culture (pp. 105–138). Routledge.
Mayne, J. (2014a). Issues in enhancing evaluation use. In M. L. Loud & J. Mayne (Eds.), Enhancing evaluation use: Insights from internal evaluation units (pp. 1–14). Sage.
Mayne, J. (2014b). Using theories of reach to enhance equity considerations in evaluation. In K. Forss & M. Marra (Eds.), Speaking justice to power: Ethical and methodological challenges for evaluators (pp. 133–152). Routledge.
Mayne, J. (2015). Useful theory of change models. Canadian Journal of Program Evaluation, 30(2), 119–142.
Mayne, J. (2017). Theory of change analysis: Building robust theories of change. Canadian Journal of Program Evaluation, 32(2), 155–173.
Mayne, J. (2018). Linking evaluation to expenditure reviews: Neither realistic nor a good idea. Canadian Journal of Program Evaluation, 32(3), 316–326.
Mayne, J. (2020a). Contribution analysis and the long-term perspective: Challenges and opportunities. In K. Forss, I. Lindkvist, & M. McGillivray (Eds.), Long term perspectives in evaluation. Increasing relevance and utility (pp. 46–68). Routledge.
Mayne, J. (2020b). Sustainability analysis of intervention benefits: A theory of change approach. The Canadian Journal of Program Evaluation, 35(2), 204–222.
Mayne, J., Divorski, S., & Lemaire, D. (1999). Locating evaluation: Anchoring evaluation in the executive or the legislature, or both or elsewhere. In R. Boyle & D. Lemaire (Eds.), Building effective evaluation capacity: Lessons from practice. Transaction.
Mayne, J., & Rist, R. C. (2006). Studies are not enough: The necessary transformation of evaluation. Canadian Journal of Program Evaluation, 21(3), 93–120. https://evaluationcanada.ca/system/files/cjpe-entries/21-3-093.pdf
McDavid, J. (2023). Enduring themes in John Mayne’s work: Implications for evaluation practice. Canadian Journal of Program Evaluation, 37(3), 355–370.
Nielsen, S. B., & Montague, S. (2023). John Mayne’s contribution to results-based management thinking: A leadership legacy. Canadian Journal of Program Evaluation, 37(3), 439–460.
Nielsen, S. B., Tangsig, S., & Lemire, S. (2023). Mapping the contributions of John Mayne: Bridging the gaps between evaluation, auditing, and performance monitoring. Canadian Journal of Program Evaluation, 37(3), 306–323.
Patton, M. Q. (2020). Evaluation use theory, practice, and future research: Reflections on the Alkin and King Aje series. American Journal of Evaluation, 41(4), 581–602.
Robinson, M. (2018). The role of evaluation in spending review. Canadian Journal of Program Evaluation, 32(3), 305–315. https://evaluationcanada.ca/system/files/cjpe-entries/32-3-305.pdf
Rutman, L., & Mayne, J. (1985). Institutionalization of program evaluation in Canada: The federal level. New directions for program evaluation, 25, 61–68.
Schwartz, R., & Mayne, J. (2005a). Assuring the quality of evaluative information: Theory and practice. Evaluation and Program Planning, 28(1), 1–14.
Schwartz, R., & Mayne, J. (2005b). Does quality matter? Who cares about the quality of evaluative information? In R. Schwartz & J. Mayne (Eds.), Quality matters (pp. 301–322). Routledge.
Shepherd, R., & Charbonneau, E. (2021). Making internal accountability relevant to public managers during and after the pandemic: A practitioner-derived research agenda. Report.
Whynot, J. (2021). Where’s Betty?: Integrating gender-based analysis plus in Canada’s federal government evaluation function. University of Ottawa.
Whynot, J., Lemire, S., & Montague, S. (2018). The current landscape of program theorizing. Canadian Journal of Program Evaluation, 33(3), 287–291.
Zapico Goni, E., & Mayne, J. (1997). Performance monitoring. Implications for the future. In Monitoring performance in the public sector (pp. 253–276). Routledge.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2023 Isabelle Bourgeois, Jane Whynot
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Authors contributing to The Canadian Journal of Program Evaluation agree to release their articles under the Creative Commons Attribution-Noncommercial 4.0 (CC-BY-NC) license. This licence allows this work to be copied, distributed, remixed, transformed, and built upon for any purpose provided that appropriate attribution is given, a link is provided to the license, and changes made were indicated.
Authors retain copyright of their work and grant the journal right of first publication.
Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
