Learning From Evaluation Data

Discoveries From the Inaugural Evaluation Capacity Case Challenge

Authors

  • Michelle Searle Faculty of Education, Queen’s University, Kingston, Ontario
  • Jennifer Hughes Faculty of Education, Queen’s University, Kingston, Ontario
  • Katrina Carbone Faculty of Education, Queen’s University, Kingston, Ontario
  • Leslie A. Fierro Max Bell School of Public Policy, McGill University, Montreal, Quebec
  • Melissa Tremblay Faculty of Education, University of Alberta, Edmonton, Alberta
  • Rebecca Gokiert School of Public Health, University of Alberta, Edmonton, Alberta
  • Isabelle Bourgeois Faculty of Education, University of Ottawa, Ottawa, Ontario

DOI:

https://doi.org/10.3138/cjpe-2024-0005

Keywords:

community-engaged learning, evaluation capacity building, evaluation competition, evaluation education, experiential learning

Abstract

Evaluation competitions, a process that has existed for more than three decades, are a mechanism to promote program evaluation and enhance evaluation capacity. Integrating these competitions into evaluation education extends traditional teaching models beyond theory-dense learning contexts. This practice note delves into the Evaluation Case Competition Challenge (EC3), a case challenge designed to advance knowledge on evaluation capacity building (ECB). Insights from an end-of-experience survey are distilled into three discoveries: successful elements of the multifaceted competition structure and exploring the case challenge as a valuable opportunity for ECB, as well as a focus on student celebrations and suggestions. These discoveries provide insight into the learner experiences and recommendations for the future trajectory of EC3. Five guiding principles for envisioning evaluation learning competitions are offered for consideration.

References

Bourgeois, I., Lemire, S. T., Fierro, L. A., Castleman, A. M., & Cho, M. (2023). Laying a solid foundation for the next generation of evaluation capacity building: Findings from an integrative review. The American Journal of Evaluation, 44(1), 29–49.

Braun, V., & Clarke, V. (2012). Thematic analysis. American Psychological Association.

Burguillo, J. C. (2010). Using game theory and competition-based learning to stimulate student motivation and performance. Computers & Education, 55(2), 566–575.

Canadian Evaluation Society. (2018). Competencies for Canadian evaluation practice. https://evaluationcanada.ca/career/evaluator-competencies.html

Cascio, M. A., Lee, E., Vaudrin, N., & Freedman, D. A. (2019). A team-based approach to open coding: Considerations for creating intercoder consensus. Field Methods, 31(2), 116–130.

DiMenichi, B. C., & Tricomi, E. (2015). The power of competition: Effects of social motivation on attention, sustained physical effort, and learning. Frontiers in Psychology, 6, 1282–1282.

Fenton, L., & Gallant, K. (2016). Integrated experiential education: Definitions and a conceptual model. The Canadian Journal for the Scholarship of Teaching and Learning, 7(2), 1–15.

Fierro, L. A., Bourgeois, I., Gokiert, R., Searle, M., & Tremblay, M. (2024). Strengthening evaluation capacity building practice through competition: The Max Bell School of Public Policy’s Evaluation Capacity Case Challenge. Canadian Journal of Program Evaluation, 38(3), 521–531.

Gullickson, A. M., King, J. A., LaVelle, J. M., & Clinton, J. M. (2019). The current state of evaluator education: A situation analysis and call to action. Evaluation and Program Planning, 75, 20–30.

Kassan, S., Gaber, J., & Lorenz, P. (2018). Game theory based distributed clustering approach to maximize wireless sensors network lifetime. Journal of Network and Computer Applications, 123, 80–88.

Kolb, A. Y., & Kolb, D. A. (2017). Experiential learning theory as a guide for experiential educators in higher education. Experiential Learning & Teaching in Higher Education, 5(1), 10–23.

LaVelle, J., Lovato, C., & Stephenson, C. L. (2020). Pedagogical considerations for the teaching of evaluation. Evaluation and Program Planning, 79, 101786–101711.

LaVelle, J. M., Stephenson, C. L., Donaldson, S. I., & Hackett, J. D. (2022). Findings from an empirical exploration of evaluators’ values. American Journal of Evaluation. Advanced online publication.

Lingard, L. (2019). Beyond the default colon: Effective use of quotes in qualitative research. Perspectives on Medical Education, 8, 360–364.

Neubauer, L. C., LaVelle, J. M., Boyce, A. S., & Archibald, T. (2023). The evaluator and evaluation: Critical considerations for the future of education and training. New Directions for Evaluation, 2023, 153–162.

Nowell, L., Dhingra, S., Carless, S., Davidson, S., & Carr, E. (2022). Nursing students’ experiences and perceptions of an innovative graduate level healthcare grand challenge course: A qualitative study. International Journal of Nursing Education Scholarship, 18(1).

Obrecht, M., Porteous, N., & Haddock, B. (1998). Casing out evaluation: Expanding student interest in program evaluation through case competitions. Canadian Journal of Program Evaluation, 13(2), 129–134.

Saldaña, J. (2016). The coding manual for qualitative researchers (3E [Third edition]). Sage.

Seaman, J., Brown, M., & Quay, J. (2017). The evolution of experiential learning theory: Tracing lines of research in the JEE. The Journal of Experiential Education, 40(4), NP1–NP21.

Sung, H. Y. (2022). A competition-based problem-posing approach for nursing training. Healthcare (Basel), 10(6), 1132.

Thomas, D. R. (2006). A general inductive approach for analyzing qualitative evaluation data. American Journal of Evaluation, 27(2), 237–246.

Trevisan, M. S. (2004). Practical training in evaluation: A review of the literature. American Journal of Evaluation, 25(2), 255–272.

Verhoeff, T. (1997). The role of competitions in education. In Proceedings of the future world: Educating for the 21st century, a Conference and Exhibition at IOI, Cape Town, South Africa, 97, https://olympiads.win.tue.nl/ioi/ioi97/ffutwrld/competit.html

Warren, J. A., Hof, K. R., McGriff, D., & Morris, L. B. (2012). Five experiential learning activities in addictions education. Journal of Creativity in Mental Health, 7(3), 272–288.

Whitmore, C. B. (2023). Teaching evaluation through community-engaged learning courses. American Journal of Evaluation, 44(2), 270–281.

Worthington, P., Stasel, R. S., Carbone, K., Hughes, J., & Searle, M. (2023). Learning by linking the Canadian Evaluation Society’s student case competition within a graduate evaluation course. New Directions for Evaluation, 2023(177), 105–113.

Downloads

Published

2024-05-06

How to Cite

Searle, M., Hughes, J., Carbone, K., Fierro, L. A., Tremblay, M., Gokiert, R., & Bourgeois, I. (2024). Learning From Evaluation Data: Discoveries From the Inaugural Evaluation Capacity Case Challenge. Canadian Journal of Program Evaluation, 38(3), 542–554. https://doi.org/10.3138/cjpe-2024-0005

Issue

Vol. 38 No. 3 (2024): Special Issue

Section

Research and Practice Notes

License

Copyright (c) 2024 Michelle Searle, Jennifer Hughes, Katrina Carbone, Leslie A. Fierro, Melissa Tremblay, Rebecca Gokiert, Isabelle Bourgeois

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

Authors contributing to The Canadian Journal of Program Evaluation agree to release their articles under the Creative Commons Attribution-Noncommercial 4.0 (CC-BY-NC) license. This licence allows this work to be copied, distributed, remixed, transformed, and built upon for any purpose provided that appropriate attribution is given, a link is provided to the license, and changes made were indicated.

Authors retain copyright of their work and grant the journal right of first publication.

Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.

Most read articles by the same author(s)

1 2 3 4 5 6 7 > >>