Pinpointing Where to Start
A Reflective Analysis on the Introductory Evaluation Course
DOI:
https://doi.org/10.3138/cjpe.69697Keywords:
assessment in higher education, evaluation education, evaluation educator, Gibbs reflective cycleAbstract
This reflective analysis details four approaches to an introductory course for evaluation learners within a methodologically focused graduate-level program on statistics, measurement, and research design. Evidence of student learning outcomes, or SLOs, was utilized within Gibbs’ reflective cycles to redesign the course using Fink’s integrated course design process. The purpose of each approach varied along a theory-practice continuum, including theory, theory-to-practice, practice, and evidence building. The purpose, SLOs, and learning experiences of each approach are accompanied by longitudinal reflections on evaluation learners, course purposes, and the creation of a multi-course learning progression. This exploration offers perspectives and lessons learned that may assist new and experienced instructors in determining how an introductory course may best fit the learning needs of their students.
References
Bakker, J. L. (2010). Interpretivism. In A. Mills, G. Durepos, & E. Wiebe (Eds.), Encyclopedia of case study research. Sage. https://doi.org/10.4135/9781412957397.n180
Banta, T. W., & Palomba, C. A. (2014). Assessment essentials: Planning, implementing, and improving assessment in higher education (2nd ed.). Jossey-Bass & Pfeiffer Imprints.
Centers for Disease Control and Prevention (CDC). (2007). Putting together an evaluation matrix. US Department of Health and Human Services. https://www.cdc.gov/std/program/pupestd.htm
Centers for Disease Control and Prevention (CDC). (2011). Introduction to program evaluation for public health programs: A self-study guide. Centers for Disease Control and Prevention.
Chouinard, J. A., Boyce, A. S., Hicks, J., Jones, J., Long, J., Pitts, R., & Stockdale, M. (2017). Navigating theory and practice through evaluation fieldwork: Experiences of novice evaluation practitioners. American Journal of Evaluation, 38(4), 493 – 506. https://doi.org/10.1177/1098214016667582
Chouinard, J. A., & Cram, F. (2019). Culturally responsive approaches to evaluation: Empirical implications for theory and practice. Sage.
Davidson, E. J. (2005). Evaluation methodology basics: The nuts and bolts of sound evaluation. Sage.
Dreyfus, S. E., & Dreyfus, H. L. (1980). A five-stage model of the mental activities involved in directed skill acquisition. Defense Technical Information Center. https://doi.org/10.21236/ADA084551
Duncan, R. G., & Hmelo -Silver, C. E. (2009). Learning progressions: Aligning curriculum, instruction, and assessment. Journal of Research in Science Teaching, 46(6), 606 – 609. https://doi.org/10.1002/tea.20316
Duschl, R., Maeng, S., & Sezen, A. (2011). Learning progressions and teaching sequences: A review and analysis. Studies in Science Education, 47(2), 123 – 182. https://doi.org/10.1080/03057267.2011.604476
Fink, L. D. (2013). Creating significant learning experiences: An integrated approach to designing college courses. John Wiley & Sons.
Fink, L. D. (2016). Designing courses for significant learning [PowerPoint Presentation]. Workshop presented at the University of Denver.
Fisher, D., & Frey, N. (2008). Better learning through structured teaching: A framework for the gradual release of responsibility. ASCD.
Gibbs, G. (1998). Learning by doing: A guide to teaching and learning methods. Further Education Unit, Oxford Polytechnic.
Gullickson, A. M. (2020, June 16). Snakes and ladders: Evaluation education from a learner’s perspective. International Society for Evaluation Education, Online Conference. https://vimeo.com/429426018
Gullickson, A. M., King, J. A., LaVelle, J. M., & Clinton, J. M. (2019). The current state of evaluator education: A situation analysis and call to action. Evaluation and Program Planning, 75, 20 – 30. https://doi.org/10.1016/j.evalprogplan.2019.02.012
Gurlitt, J. (2012). Advance organizer. In N. M. Seel (Ed.), Encyclopedia of the sciences of learning (pp. 148 – 151). Springer. https://doi.org/10.1007/978-1-4419-1428-6_157
Herman, J. (2013). Formative assessment for next generation science standards: A proposed model. National Center for Research on Evaluation, Standards and Student Testing (CRESST). http://www.ets.org/Media/Research/pdf/herman.pdf
Hunter, M. T., & McDavid, J. C. (2018). 2018 Inventory of Canadian graduate evaluation education. Consortium of Universities for Evaluation Education. http://evaluationeducation.ca/wp-content/uploads/2018/10/Inventory_Full_Canadian_Graduate_Evaluation_Education_July-2018.pdf
Hunter, M. T., & McDavid, J. C. (2019). Comparison of Canadian and American graduate evaluation education programs. The Canadian Journal of Program Evaluation, 34(2). https://doi.org/10.3138/cjpe.56989
Jin, H., Shin, H., Johnson, M. E., Kim, J., & Anderson, C. W. (2015). Developing learning progression-based teacher knowledge measures. Journal of Research in Science Teaching, 52(9), 1269 – 1295. https://doi.org/10.1002/tea.21243
King, J. A., & Ayoo, S. (2020). What do we know about evaluator education? A review of peer-reviewed publications (1978–2018). Evaluation and Program Planning, 79, 101785. https://doi.org/10.1016/j.evalprogplan.2020.101785
Kuh, G. D., Ikenberry, S. O., Jankowski, N. A., Cain, T. R., Ewell, P. T., Hutchings, P., & Kinzie, J. (2015). Using evidence of student learning to improve higher education. Jossey-Bass.
LaVelle, J. M. (2018). 2018 directory of evaluator education programs in the United States. University of Minnesota Libraries Publishing, 153.
LaVelle, J. M. (2019). Educating evaluators 1976–2017: An expanded analysis of university-based evaluation education programs. American Journal of Evaluation, 41(4), 494 – 509. https://doi.org/10.1177/1098214019860914
Pearson, P. D., & Gallagher, G. (1983). The gradual release of responsibility model of instruction. Contemporary Educational Psychology, 8, 112 – 123.
Preskill, H., & Russ-Eft, D. F. (2005). Building evaluation capacity: 72 activities for teaching and training. Sage.
Rickards, W. H., & Stitt-Bergh, M. (2016). Editors’ notes. New Directions for Evaluation, 2016(151), 7 – 9. https://doi.org/10.1002/ev.20194
Rossi, P. H., Lipsey, M. W., & Freeman, H. E. (2004). Evaluation: A systematic approach (7th ed.). Sage.
Rossi, P. H., Lipsey, M. W., & Henry, G. T. (2018). Evaluation: A systematic approach (8th ed.). Sage.
Scriven, M. (2003). Evaluation in the new millennium: The transdisciplinary vision. In S. I. Donaldson & M. Scriven (Eds.), Evaluating social programs and problems: Visions for the new millennium (pp. 19 – 41). Lawrence Erlbaum Associates.
Shulha, L. M., Whitmore, E., Cousins, J. B., Gilbert, N., & al Hudib, H. (2016). Introducing evidence-based principles to guide collaborative approaches to evaluation results of an empirical process. American Journal of Evaluation, 37(2), 193 – 215. https://doi.org/10.1177/1098214015615230
Suskie, L. (2009). Assessing student learning: A common sense guide (2nd ed.). Jossey-Bass.
Vantini, I., & Benini, L. (2008). Models of learning, training and progress evaluation of medical students. Clinica Chimica Acta, 393(1), 13 – 16. https://doi.org/10.1016/j.cca.2008.03.015
Yarbrough, D. B., Shulha, L. M., Hopson, R. K., & Caruthers, F. A. (2011). The program evaluation standards: A guide for evaluators and evaluation users. Sage.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2021 Robyn Thomas Pitts
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Authors contributing to The Canadian Journal of Program Evaluation agree to release their articles under the Creative Commons Attribution-Noncommercial 4.0 (CC-BY-NC) license. This licence allows this work to be copied, distributed, remixed, transformed, and built upon for any purpose provided that appropriate attribution is given, a link is provided to the license, and changes made were indicated.
Authors retain copyright of their work and grant the journal right of first publication.
Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
