Reporting Lines of Inquiry
Documenting Evaluations and Making Values Explicit
DOI:
https://doi.org/10.3138/cjpe-2023-0055Keywords:
evaluation reporting, values, lines of inquiry, criteria, evaluation practiceAbstract
The Program Evaluation Standards call for “rigorous documentation of evaluations” and emphasize evaluators’ responsibility to make explicit the key values that shape evaluations. Clear, complete, values-attentive evaluation reporting is necessary so constituents can understand, learn from, critique, improve, and take action based on evaluations. Yet, scholars have documented gaps in reporting that can limit understanding of evaluations. In this practice note, we provide guidance for reporting a central, value-laden component of evaluations: lines of inquiry. An evaluative line of inquiry is a linked set of an evaluation question, associated criteria or constructs/variables, data collection or analysis method(s), findings, and evaluative conclusion(s) that addresses a specific dimension of quality. We draw on a recent empirical analysis to outline the elements evaluators should include when reporting lines of inquiry and share exemplars that illustrate ways to communicate the links among these elements. We offer this guidance to assist evaluators in documenting their evaluations and making key values explicit to ensure meaningful understanding of evaluations.
References
Alkin, M. C., Christie, C. A., & Rose, M. (2006). Communicating evaluation. In I. Shaw, J. C. Greene, & M. M. Mark (Eds.), The SAGE handbook of evaluation (pp. 360–383). SAGE.
American Evaluation Association. (n.d.). The program evaluation standards. Retrieved from https://www.eval.org/About/Competencies-Standards/Program-Evaluation-Standards
Boyce, A., Avent, C. M., Adetogun, A., Servance, L., DeStefano, L., Nerem, R., & Platt, M. O. (2019). Implementation and evaluation of a biotechnology research experience for African-American high school students. Evaluation and Program Planning, 72, 162–169.
Canadian Evaluation Society. (2024). Evaluation standards. Retrieved from https://evaluationcanada.ca/career/evaluation-standards.html
Centers for Disease Control and Prevention. (n.d.). Putting together an evaluation matrix. Retrieved from https://www.cdc.gov/std/program/pupestd/pupestd_eval-matrix_shortguide_final.pdf
Davidson, E. J. (2005). Evaluation methodology basics: The nuts and bolts of sound evaluation. SAGE.
Fournier, D. M. (2005). Evaluation. In S. Mathison (Ed.), Encyclopedia of evaluation (pp. 139–140). SAGE.
Fu, A. C., Kannan, A., Shavelson, R. J., Peterson, L., & Kurpius, A. (2016). Room for rigor: Designs and methods in informal science education evaluation. Visitor Studies, 19(1), 12–38.
Grack Nelson, A., & Tranby, Z. (2015). Reporting with an evaluator audience in mind: A BISE synthesis paper. Retrieved from https://www.visitorstudies.org/bise
Greene, J. C. (2012). Values-engaged evaluations. In M. Segone (Ed.), Evaluation for equitable development results (pp. 192–206). UNICEF. Retrieved from http://www.clear-la.cide.edu/sites/default/files/Evaluation_for_equitable%20results_web.pdf
Grob, G. F. (2015). Writing for impact. In K. E. Newcomer, H. P. Hatry, & J. S. Wholey (Eds.), Handbook of practical program evaluation (4th ed., pp. 739–764). Jossey-Bass.
Gullickson, A. M., & Hannum, K. M. (2019). Making values explicit in evaluation practice. Evaluation Journal of Australasia, 19(4), 162–178.
Hurteau, M., Houle, S., & Mongiat, S. (2009). How legitimate and justified are judgments in program evaluation?. Evaluation, 15(3), 307–319.
Lucas, R. I., Promentilla, M. A., Ubando, A., Tan, R. G., Aviso, K., & Yu, K. D. (2017). An AHP-based evaluation method for teacher training workshop on information and communication technology. Evaluation and Program Planning, 63, 93–100.
Montrosse-Moorhead, B., & Griffith, J. C. (2017a). Checklist for evaluation-specific standards (Version 2.0). Retrieved from https://chess.uconn.edu/
Montrosse-Moorhead, B., & Griffith, J. C. (2017b). Toward the development of reporting standards for evaluations. American Journal of Evaluation, 38(4), 577–602.
Nielsen, J. V., Skovgaard, T., Bredahl, T. V. G., Bugge, A., Wedderkopp, N., & Klakk, H. (2018). Using the RE-AIM framework to evaluate a school-based municipal programme tripling time spent on PE. Evaluation and Program Planning, 70, 1–11.
Pawlowski, C. S., Schmidt, T., Nielsen, J. V., Troelsen, J., & Schipperijn, J. (2019). Will the children use it?—A RE-AIM evaluation of a local public open space intervention involving children from a deprived neighbourhood. Evaluation and Program Planning, 77, 101706.
Robertson, K. N., & Wingate, L. A. (2017). Checklist for program evaluation report content. Retrieved from https://wmich.edu/evaluation/checklists
Roorda, M. (2020). Developing defensible criteria for public sector evaluations (Unpublished dissertation). University of Melbourne.
Roorda, M., & Gullickson, A. M. (2019). Developing evaluation criteria using an ethical lens. Evaluation Journal of Australasia, 19(4), 179–194.
Schwandt, T. A., & Gates, E. F. (2021). Evaluating and valuing in social research. Guilford.
Teasdale, R. M. (2021). Evaluative criteria: An integrated model of domains and sources. American Journal of Evaluation, 42(3), 354–376.
Teasdale, R. M. (2022). How do you define success? Evaluative criteria for informal STEM education. Visitor Studies, 25(2), 163–184.
Teasdale, R. M., Strasser, M., Moore, C., & Graham, K. (2023). Evaluative criteria in practice: Findings from an analysis of evaluations published in Evaluation and Program Planning. Evaluation and Program Planning, 97, 102226.
Torres, R. T., Preskill, H., & Piontek, M. E. (2004). Evaluation strategies for communicating and reporting: Enhancing learning in organizations. SAGE.
Tran, V. D., Jancey, J., Lee, A., James, A., Howat, P., & Thi Phuong Mai, L. (2017). Physical activity and nutrition program for adults with metabolic syndrome: Process evaluation. Evaluation and Program Planning, 61, 128–133.
Yarbrough, D. B., Shulha, L. M., & Hopson, R. K. (2011). The program evaluation standards: A guide for evaluators and evaluation users (3rd ed.). SAGE.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Rebecca M. Teasdale, Ceily L. Moore, Mikayla Strasser
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Authors contributing to The Canadian Journal of Program Evaluation agree to release their articles under the Creative Commons Attribution-Noncommercial 4.0 (CC-BY-NC) license. This licence allows this work to be copied, distributed, remixed, transformed, and built upon for any purpose provided that appropriate attribution is given, a link is provided to the license, and changes made were indicated.
Authors retain copyright of their work and grant the journal right of first publication.
Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
