Exploring the Edges
Identifying the Next Generation of Evaluation Capacity Building Research and Practice Through Adjacency
DOI:
https://doi.org/10.3138/cjpe-2024-0014Keywords:
evaluation capacity, evaluation capacity building, evaluation theory, integrative review, research on evaluationAbstract
Evaluation capacity building (ECB) is a well-established domain of scholarship and practice within the broader field of evaluation. Stemming from a concern to ensure sustainable and useful evaluation practices in organizations, ECB draws on several evaluation theories and approaches such as collaborative evaluation and utilization-focused evaluation. In a recent integrative review, the authors and their colleagues identified other theories, frameworks, and models that inform ECB; many of these were developed in other fields, such as social sciences and public administration. The present exploratory study sought to identify some of the knowledge and theories that inform ECB from other fields through a review of 47 “ECB-adjacent” articles. This article describes the various themes and topics included in these articles to formulate potential future avenues for ECB research and practice.
References
Al Hudib, H., & Cousins, J. B. (2022). Understanding evaluation policy and organizational capacity for evaluation: An interview study. American Journal of Evaluation, 43(2), 234–254.
Alkin, M. C. & Christie, C. A. (Eds.). (2023). Evaluation roots: Theory influencing practice (3rd ed.). Guilford Press
Archibald, T., Sharrock, G., Buckley, J., & Cook, N. (2016). Assumptions, conjectures, and other miracles: The application of evaluative thinking to theory of change models in community development. Evaluation and Program Planning, 59, 119–127.
Argyris, C. (1997, September–October). Double-loop learning in organizations. Harvard Business Review, 115–125.
Argyris, C. (2003). A life full of learning. Organization Studies, 24, 1178–1192.
Argyris, C., & Schön, D. A. (1996). Organizational learning 11: Theory, method and practice. Addison Wesley.
Baruch, B., Ling, T., Warnes, R., & Hofman, J. (2018). Evaluation in an emerging field: Developing a measurement framework for the field of counter-violent-extremism. Evaluation, 24(4), 475–495.
Baughman, S., Boyd, H. H., & Franz, N. K. (2012). Non-formal educator use of evaluation results. Evaluation and Program Planning, 35, 329–336.
Bourgeois, I. (2016). Performance measurement as a precursor to organizational evaluation capacity building. Evaluation Journal of Australasia, 16(1), 11–18.
Bourgeois, I., Lemire, S., Fierro, L. A., Castleman, A. M., & Cho, M. (2023). Laying a solid foundation for the next generation of evaluation capacity building: Findings from a comprehensive review. American Journal of Evaluation, 44(1), 29–49.
Buckley, J., Hargraves, M., & Moorman, L. (2021). The relational nature of evaluation capacity building: Lessons from facilitated evaluation partnerships. New Directions for Evaluation, 2021(169), 47–64.
Clinton, J. (2014). The true impact of evaluation: Motivation for ECB. American Journal of Evaluation, 35(1), 120–127.
Cooksy, L. J., & Caracelli, V. J. (2005). Quality, context, and use: Issues in achieving the goals of metaevaluation. American Journal of Evaluation, 26(1), 31–42.
Coryn, C. L. S., Wilson, L. N., Westine, C. D., Hobson, K. A., Ozeki, S., Fiekowsky, E. L., Greenman II, G. D., & Schroter, D. C. (2017). A decade of research on evaluation: A systematic review of research on evaluation published between 2005 and 2014. American Journal of Evaluation, 38(3), 329–347.
Cousins J. B., Goh S. C., Clark S., & Lee L. E. (2004). Integrating evaluative inquiry into the organizational culture: A review and synthesis of the knowledge base. Canadian Journal of Program Evaluation, 19(2), 99–141.
Dillman, L. M., & Christie, C. A. (2017). Evaluation policy in a nonprofit foundation: A case study exploration of the Robert Wood Johnson Foundation. American Journal of Evaluation, 38(1), 60–79.
Donaldson, S. I. (2017). Empowerment evaluation: An approach that has literally altered the landscape of evaluation. Evaluation and Program Planning, 63, 136–137.
Donnelly, C., & Searle, M. (2017). Optimizing use in the field of program evaluation by integrating learning from the knowledge field. Canadian Journal of Program Evaluation, 31(3), 305–327.
Fierro, L. A., Bourgeois, I., Gokiert, R., Searle, M., & Tremblay, M. (2024). Strengthening evaluation capacity building practice through competition: The Max Bell School of Policy’s Evaluation Capacity Case Challenge. Canadian Journal of Program Evaluation, 38(3), 521–531.
Fierro, L. A., & Christie, C. A. (2017). Evaluator and program manager perceptions of evaluation capacity and evaluation practice. American Journal of Evaluation, 38(3), 376–392.
Fierro, L.A., Codd, H., Gill, S., Pham, P. K., Grandjean Targos, P. T., & Wilce, M. (2018). Evaluative thinking in practice: The National Asthma Control Program. New Directions for Evaluation, 2018(158), 49–72.
Garcia-Iriarte, E., Suarez-Balcazar, Y., Taylor-Ritzler, T., & Luna, M. (2010). A catalyst-for-change approach to evaluation capacity building. American Journal of Evaluation, 32(2), 168–182.
Gates, E. F., Walton, M., & Vidueira, P. (Eds.). (2021). Systems and complexity-informed evaluation: Insights from practice [Special issue]. New Directions for Evaluation, 170.
Harper, L. M., & Dickson, R. (2019). Using developmental evaluation principles to build capacity for knowledge mobilization in health and social care. Evaluation, 25(3), 330–348.
Higa, T. A. F., & Brandon, P. R. (2009). Participatory evaluation as seen in a Vygotskian framework. Canadian Journal of Program Evaluation, 23(3), 103–125.
Hopia, H., Latvala, E., & Liimatainen, L. (2016). Reviewing the methodology of an integrative review. Scandinavian Journal of Caring Sciences, 30(4), 662–669.
Huffman, D., Thomas, K., & Lawrenz, F. (2008). A collaborative immersion approach to evaluation capacity building. American Journal of Evaluation, 29(3), 358–368.
Jacob, S. (2005). Réflexions autour d’une typologie des dispositifs institutionnels d’évaluation [Reflecting on a model of institutional evaluation approaches]. Canadian Journal of Program Evaluation, 20(2), 49–68.
Janzen, R., Ochocka, J., Turner, L., Cook, T., Franklin, M., & Deichert, D. (2017). Building a community-based culture of evaluation. Evaluation and Program Planning, 65, 163–170.
Khalil, K., Ardoin, N. M., & Wojcik, D. (2017). Social learning within a community of practice: Investigating interactions about evaluation among zoo education professionals. Evaluation and Program Planning, 61, 45–54.
King, J. A., & Rohmert-Hirt, J.A. (2011). Internal evaluation in American public school districts: The importance of externally driven accountability mandates. New Directions for Evaluation, 2011(132), 73–86.
Kusak, J. Z., & Rist, R. C. (2001). Building a performance-based monitoring and evaluation system: The challenges facing developing countries. Evaluation Journal of Australasia, 1(2), 14–23.
Labin, S. N., Duffy, J. L., Meyers, D. C., Wandersman, A., & Lesesne, C. A. (2012). A research synthesis of the evaluation capacity building literature. American Journal of Evaluation, 33(3), 307–338.
Levin-Rozalis, M., & Rosenstein, B. (2005). The changing role of the evaluator in the process of organizational learning. Canadian Journal of Program Evaluation, 20(1), 81–104.
Lopez, A. A. (2018). Leadership in evaluation capacity building: Influencing organizational change and learning about evaluation (Publication No. 13422121) [Doctoral dissertation, Claremont Graduate University]. ProQuest Dissertations Publishing.
Mark, M. M. (2008). Building a better evidence base for evaluation theory, beyond general calls to a framework of types of research on evaluation. In N. L. Smith & P. R. Brandon (Eds.), Fundamental issues in evaluation (pp. 111–134). Guilford Press.
Mark, M. M., Donaldson, S. I., & Campbell, B. (Eds.). (2011). Social psychology and evaluation. Guilford Press.
Maxwell, N. L., Rotz, D., & Garcia, C. (2016). Data and decision making: Same organization, different perceptions; different organizations, different perceptions. American Journal of Evaluation, 37(4), 463–485.
Mayne, J. (2017). Theory of change analysis: Building robust theories of change. Canadian Journal of Program Evaluation, 32(2), 155–173.
Moreau, K. A., & Cousins, J. B. (2014). A survey of program evaluation practices in family-centered pediatric rehabilitation settings. Evaluation and Program Planning, 43, 1–8.
Newcomer, K., & Brass, C. T. (2016). Forging a strategic and comprehensive approach to evaluation within public and nonprofit organizations: integrating measurement with analytics within evaluation. American Journal of Evaluation, 37(1), 80–99.
Newcomer, K., Olejniczak, K., & Hart, N. (2022). Learning agendas: Motivation, engagement, and potential. New Directions for Evaluation, 2022(173), 63–83.
Newell, S., & Graham, A. (2012). Using an empowerment evaluation approach with community-based programs: Reflections from the front line. Evaluation Journal of Australasia, 12(2), 15–27.
Nielsen, S. B., Fierro, L. A., Bourgeois, I., & Lemire, S. (2024). The science and practice of evaluation capacity building. Canadian Journal of Program Evaluation, 38(3), 425–432.
Nielsen, S. B., & Hunter, D. E. K. (Eds.). (2013). Performance management and evaluation [Special issue]. New Directions for Evaluation, 137.
Nielsen, S.B., Lemire, S., Bourgeois, I., Fierro, L.A. (2023). Mapping the evaluation capacity landscape: A bibliometric analysis of scholarly communities and themes. Evaluation and Program Planning, 99, 102318.
Nielsen, S. B., Lemire, S., Bourgeois, I. & Fierro, L. A. (2024). Capturing evaluation capacity: Findings from a mapping of evaluation capacity instruments. Canadian Journal of Program Evaluation, 38(3), 433–460.
Patton, M.Q. (2018). A historical perspective on the evolution of evaluative thinking. New Directions for Evaluation, 2018(158), 11–28.
Peterman, K., & Gathings, M. J. (2019). Using a community-created multisite evaluation to promote evaluation use across a sector. Evaluation and Program Planning, 74, 54–60.
Phillips II, G., Lindeman, P., Adames, C.N., Bettin, E., Bayston, C., Stonehouse, P., Kern, D., Johnson, A. K., Hendricks Brown, C., & Greene, G. J. (2019). Empowerment evaluation: A case study of citywide implementation within an HIV prevention context. American Journal of Evaluation, 40(3), 318–334.
Piggot-Irvine, E. (2010). Confronting evaluation blindness: Evidence of impact of action science-based feedback. American Journal of Evaluation, 31(3), 314–325.
Preskill, H. (2014). Now for the hard stuff: Next steps in ECB research and practice. American Journal of Evaluation, 35(1), 116–119.
Preskill, H., & Boyle, S. (2008). A multidisciplinary model of evaluation capacity building. American Journal of Evaluation, 29(4), 443–459.
Raimondo, E. (2018). The power and dysfunctions of evaluation systems in international organizations. Evaluation, 24(1), 26–41.
Rugg, D., Carael, M., Boerma, J. T., & Novak, J. (2004). Global advances in monitoring and evaluation of HIV/AIDS: From AIDS case reporting to program improvement. New Directions for Evaluation, 2004(103), 33–48.
Shaw, J., & Campbell, R. (2014). The “process” of process use: Methods for longitudinal assessment in a multisite evaluation. American Journal of Evaluation, 35(2), 250–260.
Stockdill S. H., Baizerman M., & Compton D. W. (2002). Toward a definition of the ECB process: A conversation with the ECB literature. New Directions for Evaluation, 2002(93), 7–26.
Swope, M. W. (2021). From checked out to bought in: Exploring individual level motivation in evaluation capacity building (Publication No. 28546061) [Doctoral dissertation, Claremont Graduate University]. ProQuest Dissertations Publishing.
Szijarto, B., & Cousins, J. B. (2019). Making space for adaptive learning. American Journal of Evaluation, 40(2), 160–176.
Thomas, V. G., & Parsons, B. A. (2017). Culturally responsive evaluation meets systems-oriented evaluation. American Journal of Evaluation, 38(1), 7–28.
Torraco, R. (2005). Writing integrative literature reviews: Guidelines and examples. Human Resource Development Review, 4(3), 356–367.
Vang, K., Moore, M., & Nicklin, C. (2021). Applying a systems and complexity lens to building evaluation capacity: Learning from a multicountry donor’s experience. New Directions for Evaluation, 2021(170), 101–111.
Vengrin, C., Westfall-Rudd, D., Archibald, T., Rudd, R., & Singh, K. (2018). Factors affecting evaluation culture within a non-formal educational organization. Evaluation and Program Planning, 69, 75–81.
Wenger, E. (1998). Communities of practice: Learning as a social system. Systems Thinker, 9(5), 2–3.
Wilson, D. (2004). World Bank contribution to building national HIV/AIDS monitoring and evaluation capacity in Africa: Going beyond indicator development and conceptual training. New Directions for Evaluation, 2004(103), 101–115.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Leslie A. Fierro, Isabelle Bourgeois
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Authors contributing to The Canadian Journal of Program Evaluation agree to release their articles under the Creative Commons Attribution-Noncommercial 4.0 (CC-BY-NC) license. This licence allows this work to be copied, distributed, remixed, transformed, and built upon for any purpose provided that appropriate attribution is given, a link is provided to the license, and changes made were indicated.
Authors retain copyright of their work and grant the journal right of first publication.
Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
