Artificial Intelligence in Program Evaluation
Insights and Applications
DOI:
https://doi.org/10.3138/cjpe-2024-0027Keywords:
artificial intelligence, machine learning, program evaluation, data analysis, predictive modeling, data visualization, dashboards, evaluation frameworkAbstract
The practice note outlines six approaches to integrating artificial intelligence (AI) and machine learning (ML) into program evaluation, enhancing traditional methods with data-driven insights and improved efficiency. These approaches aim to address the growing need for evaluators to analyze complex datasets accurately while reducing manual effort. They include identifying patterns in data to uncover trends and outliers, using predictive models for forecasting outcomes, pinpointing areas for improvement by analyzing performance, simplifying data interpretation through visualizations, automating data analysis for efficiency, and leveraging dashboards for real-time monitoring and decision-making. The note highlights that while it does not offer guidance for evaluators with limited technical expertise, it provides a framework for integrating AI into evaluation practices. Successful implementation requires understanding stakeholder needs, fostering client engagement, ensuring tool usability, and maintaining effective communication. While AI can enhance evaluation quality and innovation, ethical considerations and biases in AI algorithms must be carefully addressed. These AI-powered techniques can enable more robust, evidence-based decision-making, supporting positive social impact.
References
Alborhamy, Y. (2020). How do data dashboards affect evaluation use in a knowledge network? A study of stakeholder perspectives in the Centre for Research on Educational and Community Services (CRECS) [Master’s thesis]. University of Ottawa.
Bangare, M. L., Bangare, P. M., Ramirez-Asis, E., Jamanca-Anaya, R., Phoemchalard, C., & Bhat, D. A. R. (2022). Role of machine learning in improving tourism and education sector. Materials Today: Proceedings, 51, 2457–2461.
Bulathwela, S., Pérez-Ortiz, M., Holloway, C., Cukurova, M., & Shawe-Taylor, J. (2024). Artificial intelligence alone will not democratise education: On educational inequality, techno-solutionism and inclusive tools. Sustainability, 16(2), 781.
Cesaro, M. (2021, November). The future of learning [paper presentation]. Abu Dhabi International Petroleum Exhibition & Conference, Abu Dhabi, United Arab Emirates.
Davenport, T., & Harris, J. (2017). Competing on analytics: Updated, with a new introduction: The new science of winning. Harvard Business Press.
Davenport, T. H., & Kirby, J. (2016). Only humans need apply: Winners and losers in the age of smart machines. Harper Business.
Felix, E. A., & Lee, S. P. (2017). Integrated approach to software defect prediction. IEEE Access, 5, 21524–21547.
Hassani, H., & Silva, E. S. (2023). The role of ChatGPT in data science: How AI-assisted conversational interfaces are revolutionizing the field. Big Data and Cognitive Computing, 7(2), 62.
Islam, M. (2020). Data analysis: Types, process, methods, techniques and tools. International Journal on Data Science and Technology, 6(1), 10–15.
Izdihar, K., Karim, M. K. A., Aresli, N. N., Radzi, S. F. M., Sabarudin, A., Yunus, M. M., Rahman, M. A. A., & Shamsul, S. (2021). Detection of novel coronavirus from chest X-Ray radiograph images via automated machine learning and CAD4COVID [Paper presentation]. 2021 International Congress of Advanced Technology and Engineering (ICOTEN), Taiz, Yemen.
Labarthe, H., Luengo, V., & Bouchet, F. (2018). Analyzing the relationships between learning analytics, educational data mining and AI for education.
Lee, Y., & Cho, J. (2021). Development of an artificial intelligence education model of classification techniques for non-computer majors. Joiv International Journal on Informatics Visualization, 5(2), 113–119. https://www.joiv.org/index.php/joiv/article/view/552
Lohit, V. S., Mujahid, M. M., & Sai, G. K. (2022). Use of machine learning for continuous improvement and handling multi-dimensional data in service sector. Computational Intelligence and Machine Learning, 3(2), 30–35.
Mueller, B., Kinoshita, T., Peebles, A. T., Graber, M. A., & Lee, S. (2022). Artificial intelligence and machine learning in emergency medicine: A narrative review. Acute Medicine & Surgery, 9(1), e740.
Provost, F., & Fawcett, T. (2013). Data science for business: What you need to know about data mining and data-analytic thinking. O’Reilly Media.
Rasheed, Z., Waseem, M., Ahmad, A., Kemell, K.-K., Xiaofeng, W., Duc, A. N., & Abrahamsson, P. (2024). Can large language models serve as data analysts? A multi-agent assisted approach for qualitative data analysis.
Seth, I., Guleria, K., & Panda, S. N. (2022). Introducing intelligence in vehicular ad hoc networks using machine learning algorithms. ECS Transactions, 107(1), 8395.
Solanki, A., Jain, K., & Jadiga, S. (2024). Building a data-driven culture: Empowering organizations with business intelligence. International Journal of Computer Trends and Technology, 72(2), 46–55.
Thu, T. N. T., Nguyen, B.-L., & Nguyen, T. (2022). Framework of using machine learning techniques in human resource management in Vietnamese companies. In Vu Dinh Khoa, Shivani Agarwal, Gloria Jeanette Rincon Aponte, Nguyen Thi Hong Nga, Vijender Kumar Solanki, Ewa Ziemba (eds), Proceedings of the 2021 International Conference on Research in Management & Technovation, ACSIS 28, 295–300.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Sidney J. Shapiro, Vinh Lam
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Authors contributing to The Canadian Journal of Program Evaluation agree to release their articles under the Creative Commons Attribution-Noncommercial 4.0 (CC-BY-NC) license. This licence allows this work to be copied, distributed, remixed, transformed, and built upon for any purpose provided that appropriate attribution is given, a link is provided to the license, and changes made were indicated.
Authors retain copyright of their work and grant the journal right of first publication.
Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
