A Transdisciplinary Model of Program Outcomes for Enhanced Evaluation Practice
DOI:
https://doi.org/10.3138/cjpe.61660Keywords:
evaluation, measurement, outcomes, research on evaluation, taxonomyAbstract
Evaluation is a transdiscipline with a focus upon asking and answering important questions about programs, policies, and interventions. A unified taxonomy of evaluation-specific program outcomes would be a helpful tool to evaluators, program designers, implementers, and policymakers, but one has not yet been proposed and validated in an evaluation context. This study builds from a grounded analysis of 125 programs and over 850 individual program outcomes, from which was developed a taxonomy of nine outcomes specific to evaluating programs and interventions: attitude, affect, behavior, cognition, status, relationship, biological, environmental, and economic. The article defines key terms and directionality and suggests specific fields that provide a helpful lens for understanding programs and improving evaluation practice.
References
Ajzen, I. (1985). From intentions to actions: A theory of planned behavior. In J. Kuhl & J. Beckmann (Eds.), Action control: From cognition to behavior (pp. 11 – 39). Berlin, Germany: Springer-Verlag.
Ajzen, I. (1991). The theory of planned behavior. Organizational Behavior and Human Decision Processes, 50(2), 179 – 211. https://doi.org/10.1016/0749-5978(91)90020-t
Allport, G. W. (1935). Attitudes. In C. Murchison (Ed.), A handbook of social psychology (pp. 798 – 844). Worcester, MA: Clark University Press.
American Psychological Association (APA). (2018a). Affect. In APA Dictionary of Psychology. Retrieved from https://dictionary.apa.org/affect
American Psychological Association (APA). (2018b). Behavior. In APA Dictionary of Psychology. Retrieved from https://dictionary.apa.org/behavior
American Psychological Association (APA). (2018c). Emotion. In APA Dictionary of Psychology. Retrieved from https://dictionary.apa.org/emotion
Backhouse, R. E., & Medema, S. G. (2009). Retrospectives: On the definition of economics. Journal of Economic Perspectives, 23(1), 221 – 233. https://doi.org/10.1257/jep.23.1.221
Barnett, E., & Casper, M. (2001). A definition of “social environment.” American Journal of Public Health, 91(3), 465. https://doi.org/10.2105/ajph.91.3.465a. Medline:11249033
Carlson, N. R. (2005). Foundations of physiological psychology (6th ed.). Boston, MA: Pearson.
Centers for Disease Control and Prevention (CDC). (1999). A framework for program evaluation. Retrieved from https://www.cdc.gov/eval/framework/index.htm
Chen, H. T. (1990). Theory-driven evaluations. Newbury Park, CA: Sage.
Chen, H. T. (2014). Practical program evaluation: Theory-driven evaluation and the integrated evaluation perspective. Thousand Oaks, CA: Sage.
Denzin, N., & Lincoln, Y. S. (1994). Handbook of qualitative research. Thousand Oaks, CA: Sage.
Donaldson, S. I. (2007). Program theory-driven evaluation science: Strategies and applications. New York, NY: Taylor & Francis.
Donaldson, S. I., & Lipsey, M. W. (2006). Roles for theory in contemporary evaluation practice: Developing practical knowledge. In I. F. Shaw, J. C. Greene, & M. M. Mark (Eds.), The handbook of evaluation: Policies, programs, and practices (pp. 56 – 75). Thousand Oaks, CA: Sage.
Fitzpatrick, J. L., Sanders, J. R., & Worthen, B. R. (2011). Program evaluation: Alternative approaches and practical guidelines (4th ed.). Upper Saddle River, NJ: Pearson.
Funnell, S. C., & Rogers, P. J. (2011). Purposeful program theory: Effective use of theories of change and logic models. San Francisco, CA: Jossey-Bass.
Janz, N. K., & Becker, M. H. (1984). The Health Belief Model: A decade later. Health Education & Behavior, 11(1), 1 – 47. https://doi.org/10.1177/109019818401100101. Medline:6392204
Kalat, J. W. (2005). An introduction to psychology (7th ed.). Belmont, CA: Thomson Wadsworth.
Katz, L. (1953). A new status index derived from sociometric analysis. Psychometrika, 18(1), 39 – 43. https://doi.org/10.1007/bf02289026
Kirkpatrick, D. L. (1987). Evaluation. In R. L. Craig (Ed.), Training and development handbook (3rd ed.). New York, NY: McGraw-Hill.
LaVelle, J. M. (2019). Teaching the science of theory-driven evaluation. Manuscript submitted for publication.
Mark, M. (2008). Building a better evidence base for evaluation theory: Beyond general calls to a framework of types of research on evaluation. In N. L. Smith & P. Brandon (Eds.), Fundamental issues in evaluation (pp. 111 – 134). New York, NY: The Guilford Press.
Martin, E., & Hine, R. (2008). Biology. In A dictionary of biology (6th ed.). Oxford, England: Oxford University Press.
Mathison, S. (2005). Outcome. In S. Mathison (Ed.), Encyclopedia of evaluation (p. 287). Thousand Oaks, CA: Sage.
Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis: An expanded source-book. Thousand Oaks, CA: Sage.
Patton, M. Q. (2008). Utilization-focused evaluation. Thousand Oaks, CA: Sage.
Podolny, J. M. (1993). A status-based model of market competition. American Journal of Sociology, 98(4), 829 – 872. https://doi.org/10.1086/230091
Pooley, J. A., & Cohen, L. (2010). Resilience: A definition in context. The Australian Community Psychologist, 22(1), 30 – 37.
Prochaska, J. O., & DiClemente, C. C. (1986). Toward a comprehensive model of change. In W. R. Miller & N. Heather (Eds.), Treating addictive behaviors: Processes of change (pp. 3 – 27). Berlin, Germany: Springer.
Rockwell, K., & Bennett, C. (2004). Targeting outcomes of programs: A hierarchy for targeting outcomes and evaluating their achievement. Lincoln, NE: University of Nebraska–Lincoln. Retrieved from https://digitalcommons.unl.edu/cgi/viewcontent.cgi?article=1047&context=aglecfacpub
Russ-Eft, D., & Preskill, H. (2009). Evaluation in organizations a systematic approach: A systematic approach to enhancing learning, performance, and change. New York, NY: Basic Books.
Scriven, M. (1994). Evaluation thesaurus (4th ed.). Thousand Oaks, CA: Sage.
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. New York: Houghton Mifflin.
Sleet, D. A., Trifiletti, L. B., Gielen, A. C., & Simons-Morten, B. (2006). Individual-level behavior change models: Applications to injury problems. In A. C. Gielen, D. A. Sleet, & R. J. DiClemente (Eds.), Injury and violence prevention: Behavioral science theories, methods, and applications (pp. 19 – 40). San Francisco, CA: Jossey-Bass.
Stufflebeam, D. L. (2001). Evaluation models. New Directions for Evaluation, 2001(89), 7 – 98. https://doi.org/10.1002/ev.3
Stufflebeam, D. L., & Shinkfield, A. J. (2007). Evaluation theory, models, and applications. San Francisco, CA: Jossey-Bass.
Suchman, E. A. (1967). Evaluation research. New York, NY: Russell Sage Foundation.
Waggoner, J., Carline, J. D., & Durning, S. J. (2016). Is there a consensus on consensus methodology? Descriptions and recommendations for future consensus research. Academic Medicine, 91(5), 663 – 668. https://doi.org/10.1097/acm.0000000000001092. Medline:26796090
Weiss, C. H. (1997). Theory-based evaluation: Past, present, and future. New Directions for Evaluation, 1997(76), 41 – 55. https://doi.org/10.1002/ev.1086
Weiss, C. H. (1998). Evaluation: Methods for studying programs and policies. Upper Saddle River, NJ: Prentice Hall.
Yarbrough, D. B., Shula, L. M., Hopson, R. K., & Cauthers, F. A. (2011). Joint committee on standards on educational evaluation: The program evaluation standards: A guide for evaluators and users. Thousand Oaks, CA: Sage.
Yardley, S., & Dornan, T. (2012). Kirkpatrick’s levels and education “evidence.” Medical Education, 46(1), 97 – 106. https://doi.org/10.1111/j.1365-2923.2011.04076.x. Medline:22150201
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2020 John M. LaVelle, Satlaj Dighe
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Authors contributing to The Canadian Journal of Program Evaluation agree to release their articles under the Creative Commons Attribution-Noncommercial 4.0 (CC-BY-NC) license. This licence allows this work to be copied, distributed, remixed, transformed, and built upon for any purpose provided that appropriate attribution is given, a link is provided to the license, and changes made were indicated.
Authors retain copyright of their work and grant the journal right of first publication.
Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
