Behind the Virtual Curtain
Techniques to Bring the CAE Principles to Life
DOI:
https://doi.org/10.3138/cjpe.75786Keywords:
CAE principles, virtual evaluation, technology, evaluation practiceAbstract
In this practice note, the authors discuss techniques they used to bring the collaborative approaches to evaluation (CAE) principles to life in a virtual setting. Following their CAE evaluation of an extracurricular post-secondary program, the authors examined artifacts of their work using CAE as a deductive framework. Their analysis highlighted numerous strategies to enact the CAE principles and the importance of modern technologies to facilitate crucial evaluation processes in a virtual setting. The authors conclude by discussing four recurring practices that aligned with the CAE principles and sharing technologies they used to enhance evaluation activities with clients, stakeholders, and each other.
References
Acree, J. (2019). Applying the collaborative approaches to Evaluation (CAE) principles in an educational evaluation: Reflections from the field. Canadian Journal of Program Evaluation, 34(2), 272–281.
Bowen, G. A. (2009). Document analysis as a qualitative research method. Qualitative Research Journal, 9(2), 27–40.
Bowman-Farrell, N. R. (2018). Looking backward but moving forward: Honoring the sacred and asserting the sovereign in Indigenous evaluation. American Journal of Evaluation. Advanced online publication.
Brandon, P. R. (1998). Stakeholder participation for the purpose of helping ensure evaluation validity: Bridging the gap between collaborative and non-collaborative evaluations. American Journal of Evaluation, 19(3), 325–337.
Coburn, C. E., & Penuel, W. R. (2016). Research–practice partnerships in education: Outcomes, dynamics, and open questions. Educational Researcher, 45(1), 48–54. ×16631750
Cousins, J. B., & Earl, L. M. (1995). Participatory evaluation in education: studies in evaluation use and organizational learning . Falmer Press.
Cousins, J. B., Shulha, L., Whitmore, E., Al Hudib, H., & Gilbert, N. (2020). Situating evidence-based principles to guide practice in collaborative approaches to evaluation (CAE). In Cousins, J. B. (Ed.), Collaborative approaches to evaluation: Principles in use, 3–46. SAGE.
Davies, A. (1992). Together is better: collaborative assessment, evaluation and reporting. Portage and Main Press.
Donaldson, S. I., & Donaldson, S. I. (2016). Visions for using evaluation to develop more equitable societies. In S. I. Donaldson & R. Picciotto (Eds.), Evaluation for an equitable society, 1–11. Information Age.
Fetterman, D. M., Rodríguez-Campos, L., Wandersman, A., O’Sullivan, R. G., & Zukoski, A. P. (2018). An Introduction to collaborative, participatory, and empowerment evaluation approaches. In D. M. Fetterman, L. Rodríguez-Campos, & A. P. Zukoski (Eds.), Collaborative, participatory, and empowerment evaluation, 1–9. Guilford Press.
Galen, M., & Grodzicki, D. (2011). Utilizing emerging technology in program evaluation. In Mathison, S. (Ed.), Really new directions in evaluation: Young evaluators’ perspectives (Issue 131, pp. 123–128). New Directions for Evaluation.
Kivunja, C., & Kuyini, A. B. (2017). Understanding and applying research paradigms in educational contexts. International Journal of Higher Education, 6(5), 26–41.
Kyei-Mensah, G., Klugman, B., Feyissa-Azasoo, Z. A., Mathais, J., & Boarden, S. (2021). Power in evaluation: Ways of seeing it and entry points for doing things differently. American Evaluation Association Conference. https://evaluation.secure-platform.com/a/gallery/rounds/1/schedule/items/562
Lee, K., Krenn, H., Avey, H., Bowman, N., Fraser, E., Hall, M., Parker, C. S., Pineda, D., & Ricci, C. (2021). Doing evaluation in service of racial equity: Deepen community engagement. W.K. Kellogg Foundation.
Leslie, D., Holosko, M., & Dunlop, J. (2006). Using information technology in planning program evaluation. Journal of Evidence-Based Social Work, 3, 73–90. _06
Materia, F. T., Miller, E. A., Runion, M. C., Chesnut, R. P., Irvin, J. B., Richardson, C. B., & Perkins, D. F. (2016). Let’s get technical: Enhancing program evaluation through the use and integration of internet and mobile technologies. Evaluation and Program Planning, 56, 31–42.
McLuhan, M. (1964). Understanding media: The extensions of man. McGraw-Hill.
Poth, C. A., & Shulha, L. (2008). Encouraging stakeholder engagement: A case study of evaluator behavior. Studies in Educational Evaluation, 34(4), 218–223.
Raţiu, L., Chirică, S., & Rus, C. L. (2014). Participatory evaluation: An intended catalyst to learning within university. Procedia—Social and Behavioral Sciences, 142, 140–145.
Saldaña, J. (2021). The coding manual for qualitative researchers (J. Seaman, C. Bush, & I. Antcliff, Eds.; 4th ed.). SAGE Publications.
Schwandt, T. A. (2009). Toward a practical theory of evidence for evaluation. In S. I. Donaldson, C. A. Christie, & M. M. Mark (Eds.), What counts as credible evidence in applied research and evaluation practice?, 197–212. SAGE Publications.
Searle, M., Merchant, S., Chalas, A., & Lam, C. Y. (2017). A case study of the guiding principles for collaborative approaches to evaluation in a developmental evaluation context. Canadian Journal of Program Evaluation, 31(3), 350–373.
Shulha, L. M., Whitmore, E., Cousins, J. B., Gilbert, N., & al Hudib, H. (2016). Introducing evidence-based principles to guide collaborative approaches to evaluation: Results of an empirical process. American Journal of Evaluation, 37(2), 193–215.
Stickl Haugen, J., & Chouinard, J. A. (2019). Transparent, translucent, opaque: Exploring the dimensions of power in culturally responsive evaluation contexts. American Journal of Evaluation, 40(3), 376–394.
Whitmore, E. (1998). Understanding and practicing participatory evaluation. Jossey-Bass.
Whitmore, E., Al Hudib, H., Cousins, J. B., Gilbert, N., & Shulha, L. M. (2017). Reflections on the meaning of success in collaborative approaches to evaluation: Results of an empirical study. Canadian Journal of Program Evaluation, 31(3), 328–349.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2023 Paisley Worthington, Cheryl Mak, Michael Holden, Michelle Searle
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Authors contributing to The Canadian Journal of Program Evaluation agree to release their articles under the Creative Commons Attribution-Noncommercial 4.0 (CC-BY-NC) license. This licence allows this work to be copied, distributed, remixed, transformed, and built upon for any purpose provided that appropriate attribution is given, a link is provided to the license, and changes made were indicated.
Authors retain copyright of their work and grant the journal right of first publication.
Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
