Building Capacity With Evaluation Standards and Guidelines in Prince Edward Island
Responding to Academics’ “Call to Action”
DOI:
https://doi.org/10.3138/cjpe.74373Keywords:
evaluation policy, leadership, governmentAbstract
The purpose of this practice note is to promote knowledge sharing and insight for evaluation practitioners and academics in other jurisdictions by reflecting on evaluation policy development in Prince Edward Island. The author describes the process of developing evaluation standards and guidelines for the Government of Prince Edward Island and the benefits of a systematic development process. The author reviews each phase of the process, including “Identify,” “Consult,” “Study,” “Consensus,” “External Review,” and “Endorsement.” The author concludes that the benefit of developing evaluation standards and guidelines is two-fold: A new resource is added to the government’s evaluation toolkit, and the development process provides an opportunity for capacity and interdepartmental community building.
References
Ameen, C. (1990). Use of the Joint Committee standards: Benefits gained and lessons learned. Evaluation Practice, 11(1), 33–38.
Arminio, J. (2009). Applying professional standards. In G. McClellan & J. Stringer (Eds.), The Handbook of Student Affairs Administration, 3rd ed., (pp. 187–205). Jossey-Bass.
Aucoin, P. (2005). Decision-making in government: The role of program evaluation. Treasury Board of Canada Secretariat. https://www.tbs-sct.gc.ca/cee/tools-outils/aucoin-eng.asp
Baer, W. (1986). Expertise and professional standards. Work and Occupations, 13(4), 532–552.
Bustelo, M. (2006). The potential role of standards and guidelines in the development of an evaluation culture in Spain. Evaluation, 12(4), 437–453.
Coakes, E. & Smith, P. (2007). Developing communities of innovation by identifying innovation champions. The Learning Organization, 14(1), 74–85.
Dobell, R., & Zussman, D. (2018). Sunshine, scrutiny, and spending review in Canada, Trudeau to Trudeau: From program evaluation and policy to commitment and results. Canadian Journal of Program Evaluation, 32(3), 371–393.
Foote, R. (1986). The case for centralized program evaluation function within the Government of Canada. Canadian Journal of Program Evaluation, 1(1), 89–95.
Gugerty, M., Mitchell, G., & Santamarina, F. (2021). Discourses of evaluation: Institutional logics and organizational practices among international development agencies. World Development, 146(11), 1–11.
Hatanaka, M., & Konefal, J. (2012). Legitimacy and standard development in multistakeholder initiatives: A case study of the Leonardo Academy’s sustainable agriculture standard initiative. International Journal of Sociology of Agriculture & Food, 20(2), 155–173.
Khalil, K., Ardoin, N., & Wojcik, D. (2016). The company you keep: Networks in a community of informational education evaluators. Studies in Educational Evaluation, 51, 7–16.
Koldsgaard, M., Crooks, D., & Durrant, M. (2008). Innovations in nursing education: Standard development for nurse educator practice. Journal for Nurses in Staff Development, 24(3), 119–123.
Lahey, R. & Nielsen, S. (2013). Rethinking the relationship among monitoring, evaluation, and results-based management: Observations from Canada. New Directions for Evaluation, 137, 45–56.
Lahey, R., MacDonald, W., Brower, K., Chaytor, K., Hurstfield-Meyer, R., Jacob, J., Jordan, F., Kishchuk, P., Kuji-Shikatani, K., Lamari, M., Lee, L., McDavid, J., Miller, T., Morin, S., & Woodman, K. (2020). Evaluation in the provinces and territories: A cross-Canada snapshot and call to action. Canadian Journal of Program Evaluation, 35(3), 111–126.
Lau, F. (2019). Cultivating eHealth evaluation leadership. Healthcare Management Forum, 32(4), 192–196.
Linn, R. (1981). A preliminary look at applicability of the educational evaluation standards. Educational Evaluation and Policy Analysis, 3(2), 87–91.
Maxwell, N. (1986). Linking ongoing performance measurement and program evaluation in the Canadian federal government. Canadian Journal of Program Evaluation, 1(1), 39–46.
Mayne, J. (1986). Ongoing program performance information systems and program evaluation in the Government of Canada. Canadian Journal of Program Evaluation, 1(1), 8–17.
McDonald, R., & Teather, G. (1997). Science and technology evaluation practices in the Government of Canada. In Policy evaluation in innovation and technology: Towards best practices (pp. 393–417). OECD.
Millerand, F., & Baker, K. (2010). Who are the users? Who are the developers? Webs of users and developers in the development of a technical standard. Information Systems Journal, 20(2), 137–161.
Mowry, S., Clough, K., MacDonald, B., Pranger, T., & Griner, D. (2001). Evaluation policy and practice in the provincial government of Prince Edward Island. Canadian Journal of Program Evaluation, 16(special issue), 89–100.
Müller-Clemm, W., & Barnes, M. (1997). A historical perspective on federal program evaluation in Canada. Canadian Journal of Program Evaluation, 12(1), 47–70.
Nowakowski, J. (1990). Exploring the role of professional standards in evaluations: Areas of needed research. Studies in Educational Evaluation, 16(2), 271–296.
Penny, D. (1979). Provincial evaluation policy and the Ontario assessment instrument pool. School Guidance Worker, 34(4), 5–10.
Russon, C. (Ed.). (2000). The program evaluation standards in international settings (The Evaluation Center Occasional Paper Series). Western Michigan University.
Rutman, L. (1986). Some thoughts on federal level evaluation. Canadian Journal of Program Evaluation, 1(1), 19–27.
Rutman, L., & Mayne, J. (1985). Institutionalization of program evaluation in Canada: The federal level. New Directions for Program Evaluation, 25(special issue), 61–68.
Schmidt, C. (2017). Guidelines for the design, socialization, formalization, and adoption of new technical standards. MITRE. https://www.mitre.org/publications/technical-papers/best-practices-for-technical-standard-creation
Schwartz, R. & Mayne, J. (2005). Insuring the quality of evaluative information: Theory and practice. Evaluation and Program Planning, 28(1), 1–14.
Schwarzman, J., Bauman, A., Gabbe, B., Rissel, C., Shilton, T., & Smith, B. (2021, October 8). How practitioner, organisational and system-level factors act to influence health promotion evaluation capacity: Validation of a conceptual framework. Evaluation and Program Planning. Advanced online publication.
Segsworth, R. (1990). Auditing and evaluation in the Government of Canada: Some reflections. Canadian Journal of Program Evaluation, 5(1), 41–56.
Segsworth, R. (2005). Program evaluation in the Government of Canada: Plus ça change. Canadian Journal of Program Evaluation, 20(3), 175–197.
Shepherd, R. (2012). In search of a balanced Canadian federal evaluation function: Getting to relevance. The Canadian Journal of Program Evaluation, 26(2), 1–45.
Shepherd, R. (2018). Expenditure reviews and the federal experience: Program evaluation and its contribution to assurance provision. The Canadian Journal of Program Evaluation, 32(3), 347–370.
Stake, R. (1981). Setting standards for educational evaluators. American Journal of Evaluation, 2(2), 148–152.
Stufflebeam, D. (1982). A next step: Discussion to consider unifying the ERS and Joint Committee standards. In Rossi (Ed.), Standards for evaluation practice (pp. 67–81). Jossey-Bass.
Stufflebeam, D. (1985). The relevance of the Joint Committee standards for improving evaluations in continuing education in the health professions. In S. Abrahamson (Ed.), Evaluation of continuing education in the health professions (pp. 15–52). Springer.
Stufflebeam, D. (1986, April 16-20). Standards of practice for evaluators [Paper presentation]. Annual Meeting of the American Educational Research Association, San Francisco, California. USA. https://eric.ed.gov/?id=ED291768
Stufflebeam, D. (2004). A note on the purposes, development, and applicability of the Joint Committee evaluation standards. American Journal of Evaluation, 25(1), 99–102.
Stufflebeam, D. & Madaues, G. (1983). The standards for evaluation of educational programs, projects, and materials, In G. Madaus, M. Scriven, & D. Stufflebeam (Eds.), Evaluation models: Viewpoints on educational and human services evaluation (pp. 395–404). Springer.
Taut, S. (2007). Studying self-evaluation capacity building in a large international development organization. American Journal of Evaluation, 28(1), 45–59.
Thompson, B. (1994). The revised program evaluation standards and their correlation with the evaluation use literature. The Journal of Experimental Evaluation, 63(1), 54–82.
Wade, J. & Kallemeyn, L. (2020). Evaluation capacity building (ECB) interventions and the development of sustainable evaluation practice: An exploratory study. Evaluation and Program Planning, 79(119), 1–9.
Wargo, M. (1981). The standards: A federal perspective. American Journal of Evaluation, 2(2), 157–162.
Wingate, L. (2009). The program evaluation standards applied for metaevaluation purposes: Investigating interrater reliability and implications for use [Doctoral thesis, Western Michigan University]. https://www.proquest.com/docview/305029057
Yarbrough, D. (2017). Developing the program evaluation utility standards: Scholarly foundations and collaborative practices. The Canadian Journal of Program Evaluation, 31(special issue), 284–304.
Yarbrough, D., Shulha, L. M., & Caruthers, F. (2004). Background and history of the Joint Committee’s program evaluation standards. New Directions for Evaluation, 104, 15–30.
Yarbrough, D., Shulha, L., Hopson, R., & Caruthers, F. (2011). The program evaluation standards: A guide for evaluators and evaluation users (3rd ed.). Sage.
Young, G. (2006). Evaluation can cross the boundaries: The case of Transport Canada. Canadian Journal of Program Evaluation, 21(3), 73–92.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2023 Bobby Thomas Cameron
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Authors contributing to The Canadian Journal of Program Evaluation agree to release their articles under the Creative Commons Attribution-Noncommercial 4.0 (CC-BY-NC) license. This licence allows this work to be copied, distributed, remixed, transformed, and built upon for any purpose provided that appropriate attribution is given, a link is provided to the license, and changes made were indicated.
Authors retain copyright of their work and grant the journal right of first publication.
Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
