Advancing a Competency-Based Mixed Methods Tool to Assess a Course-Based Service-Learning Model Integrating Real-World Experience
DOI:
https://doi.org/10.3138/cjpe.71043Keywords:
assessment tool, evaluation competency, mixed methods, program evaluation education, service-learningAbstract
Given increasingly complex societal issues such as global aging, evaluators must develop essential competencies responding to the messiness of reality. This practice note advances an argument for the inclusion of service learning as a form of experiential learning that engages learners in hands-on practice with guided reflection, and shares a competency-based mixed methods (MM) tool for generating learning evidence in a course-based service-learning model. I aligned key design elements with pedagogical approaches with evaluation competencies. Competency-based guidance is critical to ensure effective assessment and quality course delivery. I conclude with lessons learned for integration of MM within program evaluation education.
References
American Evaluation Association (AEA). (2018 , April 5). AEA evaluator competencies. https://www.eval.org/About/Competencies-Standards/AEA-Evaluator-Competencies
Canadian Evaluation Society (CES). (2018). Competencies for Canadian evaluators. https://evaluationcanada.ca/competencies-canadian-evaluators
Christie, C. A., Quiñones, P., & Fierro, L. (2014). Informing the discussion on evaluator training: A look at evaluators’ course taking and professional practice. American Journal of Evaluation, 35(2), 274 – 290. https://doi.org/10.1177/1098214013503697
Creswell, J., & Plano Clark, V. L. (2018). Designing and conducting mixed methods research (3rd ed.). Sage.
Dillman, L. M. (2012). Evaluator skill acquisition: Linking educational experiences to competencies. American Journal of Evaluation, 34(2), 270 – 285. https://doi.org/10.1177/1098214012464512
El Hassar, B. E., Poth, C., Gokiert, R. J., & Bulut, O. (2020). Toward an evidence-based approach to building evaluation capacity. Canadian Journal of Program Evaluation, 36(1). https://doi.org/10.3138/cjpe.69191
Eyler, J., & Giles, D. E., Jr. (1999). Where’s the learning in service-learning? Jossey-Bass.
Galport, N., & Azzam, T. (2016). Evaluator training needs and competencies: A gap analysis . American Journal of Evaluation, 38(1), 80 – 100. https://doi.org/10.1177/1098214016643183
Hou, S.-I. (2009). Measuring the infusion of service-learning on student outcomes: The Service-Learning Self-Efficacy Scale on Program Planning Competencies (SL-SEPP). California Journal of Health Promotion, 7 (2), 123–128. https://doi.org/10.32398/cjhp.v7i2.2020
Hou, S.-I. (2021). A mixed methods process evaluation of an integrated course design on teaching mixed methods research . International Journal of Scholarship of Teaching and Learning, 15(2), article 8. https://digitalcommons.georgiasouthern.edu/ij-sotl/vol15/iss2/8
Hou , S.-I. (forthcoming). A mixed methods evaluation on teaching evaluation: Innovative course-based service-learning model on program evaluation competencies . Journal of the Scholarship of Teaching and Learning.
Hou, S.-I. , & Pereira , V. (2017). Measuring the infusion of Service-Learning (SL) on student development and implementation competencies . Journal of Experiential Education, 40(2), 170 – 186. https://doi.org/10.1177/1053825917699518
Hou, S.-I., & Wilder, S. (2015). Changing pedagogy: Faculty adoption of service-learning: Motivations, barriers, and strategies among service-learning faculty at a public research Institution. SAGE Open, 5(1), 1 – 7. https://doi.org/10.1177/2158244015572283
Kane, M. (2013). The argument-based approach to validation. School Psychology Review, 42(4), 448 – 457. https://doi.org/10.1080/02796015.2013.12087465
Kolb, D. A. (1984). Experiential learning: Experience as the source of learning and development. Prentice-Hall.
Linfeld , K. J. (2019). Hands-on program evaluation training: The evolution of a doctoral course. Journal of Prevention and Intervention in the Community, 1 – 17. https://doi.org/10.1080/10852352.2019.1643580. Medline:31364929
Mertens, D. M. (2018). Mixed methods design in evaluation. Sage.
Newcomer, K. E., Hatry, H. P., & Wholey, J. S. (2015). Handbook of practical program evaluation (4th ed.). Jossey-Bass.
Novak, J. M., Markey, V., & Allen, M. (2007). Evaluating cognitive outcomes of service learning in higher education: A meta-analysis . Communication Research Reports, 24(2), 149 – 157. https://doi.org/10.1080/08824090701304881
Poth, C. N., Searle, M., Aquilina, A. M., Ge, J., & Elder, A. (2020, January 27). Assessing competency-based evaluation course impacts: A mixed methods case study. Evaluation and Program Planning, 79, 101789. https://doi.org/10.1016/j.evalprogplan.2020.101789. Medline:32032893
Rossi, P., Lipsey, M., & Freeman, H. (2003). Evaluation: A systematic approach (7th ed.). Sage.
Stevahn, L., King, J., Ghere, G., & Minnema , J. (2005). Establishing essential competencies for program evaluators. American Journal of Evaluation, 26(1), 43 – 59. https://doi.org/10.1177/1098214004273180
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2022 Su-I Hou
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Authors contributing to The Canadian Journal of Program Evaluation agree to release their articles under the Creative Commons Attribution-Noncommercial 4.0 (CC-BY-NC) license. This licence allows this work to be copied, distributed, remixed, transformed, and built upon for any purpose provided that appropriate attribution is given, a link is provided to the license, and changes made were indicated.
Authors retain copyright of their work and grant the journal right of first publication.
Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
