Leveraging Mixed Methods Designs for Promoting Evaluation and Evaluation Capacity Building
DOI:
https://doi.org/10.3138/cjpe.71288Keywords:
evaluation, evaluation capacity building, higher education evaluation capacity, mixed methods evaluation, mixed methods integrationAbstract
Limited evaluation capacity, power dynamics, and resource constraints act as organizational barriers that inhibit evaluations in higher education contexts. Evaluation capacity promotes evaluation and minimizes the impact of these barriers. Embedding a synergistic combination of qualitative and quantitative methods within an evaluation and evaluation capacity building (ECB) initiative was our attempt to address these organizational barriers. In this practice note, we illustrate how MM designs catalyze evaluative thinking that promotes evaluation and supports ECB efforts. Our use and integration of MM strategies to build evaluation capacity and infuse evaluation into organizational culture informs higher education evaluation capacity-building initiatives.
References
Alkin, M. C., & Taut, S. M. (2002). Unbundling evaluation use. Studies in Educational Evaluation, 29(1), 1 – 12. https://doi.org/10.1016/s0191-491x(03)90001-0
Ancona, D. G., Kochan, T. A., Scully, M., & Van Maanen, J. (2004). Managing for the future: Organizational behavior and processes (3rd ed.). South Western College Publications.
Bickman, L. (1979). Program evaluation and social psychology: Toward the achievement of relevancy. Personality and Social Psychology Bulletin, 5(4), 483 – 490. https://doi.org/10.1177/014616727900500410
Compton, D. W., Glover ‐Kudon, R., Smith, I. E., & Avery, M. E. (2002). Ongoing capacity building in the American Cancer Society (ACS) 1995–2001. New Directions for Evaluation, 2002(93), 47 – 62. https://doi.org/10.1002/ev.41
Cousins, J. B., Elliott, C., Amo, C., Bourgeois, I., Chouinard, J. A., Goh, S. C., & Lahey, R. (2008). Organizational capacity to do and use evaluation: Results of a pan-Canadian survey of evaluators. Canadian Journal of Program Evaluation, 23(3), 1 – 35.
Creswell, J. W., & Plano Clark, V. L. (2011). Designing and conducting mixed methods research. Sage.
Greene, J. C. (2015). The emergence of mixing methods in the field of evaluation. Qualitative Health Research, 25(6), 746 – 750. https://doi.org/10.1177/1049732315576499. Medline:25800716
Labin, S. N., Dufy, J. L., Meyers, D. C., Wandersman, A., & Lesesne, C. A. (2012). A research synthesis of the evaluation capacity building literature. American Journal of Evaluation, 33(3), 307 – 338. https://doi.org/10.1177/1098214011434608
Mahato , S. (2020). An empirical study of the process of evaluation capacity building in higher education (Unpublished doctoral dissertation). Ohio University, Athens, OH.
Mertens, D. M. (2017). Mixed methods design in evaluation. Sage.
Nastasi, B. K., Hitchcock, J. H., & Brown, L. M. (2010). An inclusive framework for conceptualizing mixed methods design typologies: Moving toward fully integrated synergistic research models. In A. Tasakkori & C. Teddlie (Eds.), Handbook of mixed methods in social & behavioral research (vol. 2 , pp. 305 – 338). Sage.
Patton, M. Q. (2002). Qualitative research and evaluation methods. Sage.
Patton, M. Q. (2012). Essentials of utilization-focused evaluation. Sage.
Posavac, E. J. (1994). Misusing program evaluation by asking the wrong question. New Directions for Program Evaluation, 1994(64), 69 – 78. https://doi.org/10.1002/ev.1696
Poth, C. N. (2018). Innovation in mixed methods research: A practical guide to integrative thinking with complexity. Sage.
Preskill, H., & Boyle, S. (2008). A multidisciplinary model of evaluation capacity building. American Journal of Evaluation, 29(4), 443 – 459. https://doi.org/10.1177/1098214008324182
Shulha, L. M., & Cousins, J. B. (1997). Evaluation use: Theory, research, and practice since 1986. American Journal of Evaluation, 18(3), 195 – 208. https://doi.org/10.1016/s08861633(97)90027-1
VeneKlasen, L., & Miller, V. (2002). Power and empowerment. PLA Notes, 43, 39 – 41. https://doi.org/10.3362/9781780444208.004
Walser, T. M. (2015). Evaluability assessment in higher education: Supporting continuous improvement, accountability, and a culture of assessment. Journal of Assessment and Institutional Effectiveness, 5(1), 58 – 77. https://doi.org/10.5325/jasseinstefe.5.1.0058
Wholey, J. S. (2012). Using evaluation to improve program performance and results . In M. Alkin (Ed.), Evaluation roots: A wider perspective of theorists’ views and influences (2nd ed., pp. 261 – 267). Sage.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2022 Seema Mahato, Krisanna Machtmes, Bradley Cohen
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Authors contributing to The Canadian Journal of Program Evaluation agree to release their articles under the Creative Commons Attribution-Noncommercial 4.0 (CC-BY-NC) license. This licence allows this work to be copied, distributed, remixed, transformed, and built upon for any purpose provided that appropriate attribution is given, a link is provided to the license, and changes made were indicated.
Authors retain copyright of their work and grant the journal right of first publication.
Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
