The “What” and “Why” of (Un)Ethical Evaluation Practice
A Meta-Narrative Review and Ethical Awareness Framework
DOI:
https://doi.org/10.3138/cjpe-2023-0023Keywords:
evaluation ethics, program evaluation, stewardship, values, principles, ethical frameworkAbstract
There is growing recognition of the complex moral and ethical tensions associated with evaluation practice. However, there are scant evidence-informed frameworks for cultivating ethical awareness or informing ethical deliberation across the evaluation landscape. Thus, we aimed to synthesize research evidence on evaluation ethics, and draw on these findings to develop an evidence-informed evaluation ethics framework. Our methodological approach involved, first, conducting a meta-narrative review of empirical studies on evaluation ethics. Specifically, we conducted a systematic peer-reviewed and grey literature search, then identified, extracted, and thematically organize data from 20 studies that meet inclusion criteria. Second, in consultation with an ethicist, we curated findings on ethical concerns within an integrated evaluation ethics framework. Our results illustrate six thematic patterns of research inquiry on evaluation ethics and highlight trends, and gaps. The ethics framework (ACAP) we develop includes four multi-faceted categories. It outlines six Accountabilities (where ethical consideration is owed), illustrates how ethical Concerns can manifest in practice, and outlines diverse stakeholder groups’ Agency over the management of ethical concerns. Critically, it outlines five meta-categories of ethical principles (P) including systematic and transparent inquiry, accordant self-determination, fairness, beneficence and non-maleficence, and reflexive stewardship. Implications for priming ethical awareness, navigating ethical conflicts, and advancing evaluation ethics education and research are discussed.
References
Abma, T. A., Visse, M., Hanberger, A., Simons, H., & Greene, J. C. (2020). Enriching evaluation practice through care ethics. Evaluation, 26(2), 131–146.
Alexander, L. B., & Richman, K. A. (2008). Ethical dilemmas in evaluations using indigenous research workers. American Journal of Evaluation, 29(1), 73–85.
American Evaluation Association. (2018). Guiding Principles. Retrieved from https://www.eval.org/Portals/0/Docs/AEA_289398-18_GuidingPrinciples_Brochure_2.pdf
Anthias, F., & Lloyd, C. (2002). Rethinking anti-racisms: From theory to practice. Routledge. https://books.google.ca/books?id=nQjKgBu-_hAC
Alkin, M. C., & King, J. A. (2017). Definitions of evaluation use and misuse, evaluation influence, and factors affecting use. American Journal of Evaluation, 38(3), 434–450.
Arakawa, S., Sachdeva, S., & Shandas, V. (2018). Environmental stewardship. In J. Marques (Ed.), Handbook of engaged sustainability (pp. 1–23). Springer International Publishing.
Asmal, L., Lamp, G., & Tan, E. J. (2022). Considerations for improving diversity, equity and inclusivity within research designs and teams. Psychiatry Research, 307, 114295.
Averill, J. B. (2002). Matrix analysis as a complementary analytic strategy in qualitative inquiry. Qualitative Health Research, 12(6), 855–866.
Azzam, T., Wanzer, D. L., Knight, C., & Codd, H. (2021). The manifestations of politics in evaluation: An exploratory study across the evaluation process. Evaluation and Program Planning, 88, 101947.
Chilisa, B., & Mertens, D. M. (2021). Indigenous made in Africa evaluation frameworks: Addressing epistemic violence and contributing to social transformation. American Journal of Evaluation, 42(2), 241–253.
Baldwin, T., Brownsword, R., & Schmidt, H. (2009). Stewardship, paternalism and public health: Further thoughts. Public Health Ethics, 2(1), 113–116.
Banks, S. (2013). Negotiating personal engagement and professional accountability: professional wisdom and ethics work. European Journal of Social Work, 16(5), 587–604.
Barnett, D. J., Taylor, H. A., Hodge, J. G., & Links, J. M. (2009). Resource allocation on the frontlines of public health preparedness and response: Report of a summit on legal and ethical issues. Public Health Reports, 124(2), 295–303.
Bateson, G. (1972). Steps to an ecology of mind. New York, NY: Ballantine.
Beauchamp, T. L. (2008). The principle of beneficence in applied ethics (Stanford Encyclopedia of Philosophy). Stanford Encyclopedia of Philosophy. Retrieved from http://seop.illc.uva.nl/entries/principle-beneficence/
Beever, J., & Brightman, A. O. (2015). Reflexive principlism as an effective approach for developing ethical reasoning in engineering. Science and Engineering Ethics, 22(1), 275–291.
Bies, R. J. (2001). Interactional (in)justice: The sacred and the profane. In J. Greenberg and R. Cropanzano (Eds.), Advances in organization justice (pp. 89–118). Stanford University Press.
Bonnett, A. (2000). Anti-racism. Routledge. https://books.google.ca/books?id=PcccemO4GbUC
Botes, A. (2000). A comparison between the ethics of justice and the ethics of care. Journal of Advanced Nursing, 32(5), 1071–1075.
Bourgeois, I., Hotte, N., Simmons, L., & Osseni, R. (2016). Measuring evaluation capacity in Ontario public health units. Canadian Journal of Program Evaluation, 31(2), 165–183.
Bourgeois, I., Toews, E., Whynot, J., & Lamarche, M. K. (2013). Measuring organizational evaluation capacity in the Canadian federal government. Canadian Journal of Program Evaluation, 28(2), 1–19.
Bourgeois, I., & Whynot, J. (2018). Strategic evaluation utilization in the Canadian Federal Government. Canadian Journal of Program Evaluation, 32(3), 327–346.
Bozeman, B., & Hirsch, P. (2005). Science ethics as a bureaucratic problem: IRBs, rules, and failures of control. Policy Sciences, 38(4), 269–291.
Bracht, N., Finnegan Jr, J. R., Rissel, C., Weisbrod, R., Gleason, J., Corbett, J., & Veblen-Mortenson, S. (1994). Community ownership and program continuation following a health demonstration project. Health Education Research, 9(2), 243–255.
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3, 77–101.
Brebels, L., de Cremer, D., van Dijke, M., van Hiel, A. (2011). Fairness as social responsibility: A moral self-regulation account of procedural justice enactment. British Journal of Management, 22(SUPPL. 1), S47–S58.
Brear, M. & Gordon, R. (2021). Translating the Principle of Beneficence into Ethical Participatory Development Research Practice. Journal of International Development, 33(1), 109–126.
Brownsword, R. (2013). Public health interventions: Liberal limits and stewardship responsibilities. Public Health Ethics, 6(3), 235–240. http://www.jstor.org/stable/26645029
Buchanan, H., Babcock, K., & MacDonald, W. (2011). Ethical Challenges in Evaluation: 2010 Canadian Survey. Retrieved from https://www.perfeval.pol.ulaval.ca/es/node/1370
Canadian Evaluation Society (CES). 2020. CES guidance for ethical evaluation practice. Retrieved from https://evaluationcanada.ca/files/pdf/CES_Guidance_for_Ethical_Evaluation_approved_09_01_20.pdf
Canadian Institutes of Health Research, Natural Sciences and Engineering Research Council of Canada, & Social Sciences and Humanities Research Council of Canada. (2022). Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans—TCPS 2 (2022). Retrieved from https://ethics.gc.ca/eng/documents/tcps2-2022-en.pdf
Cetron, M., & Landwirth, J. (2005). Public health and ethical considerations in planning for quarantine. The Yale Journal of Biology and Medicine, 78, 329–334.
Chi, C., Núñez, A., & Tuepker, A. (2022). From sharing voices to making decisions: The case for centring community ownership in evaluation of health programme planning and management. The International Journal of Health Planning and Management, 37(S1), 8–19.
Chouinard, J. A., & Cousins, J. B. (2021). Developing an ethical rationale for collaborative approaches to evaluation. Evaluation, 27(3), 364–381.
Chouinard, J. A., & Cram, F. (2021). Introduction to the ethics, values, and culture section. American Journal of Evaluation, 42(2), 237–240.
Chyung, S. Y., Winiecki, D. J., & Downing, J. L. (2010). Training and performance improvement professionals’ perspectives on ethical challenges during evaluation. Performance Improvement Quarterly, 23(1), 7–29.
Coryn, C. L. S., Wilson, L. N., Westine, C. D., Hobson, K. A., Ozeki, S., Fiekowsky, E. L., … Schröter, D. C. (2017). A decade of research on evaluation: A systematic review of research on evaluation published between 2005 and 2014. American Journal of Evaluation, 38(3), 329–347.
Cottone, R. R. (2014). On replacing the ethical principle of autonomy with an ethical principle of accordance. Counseling and Values, 59(2), 238–248.
Coughlin, S. S., Barker, A., & Dawson, A. (2012). Ethics and scientific integrity in public health, epidemiological and clinical research. Public Health Reviews, 34(1), 71–83.
Covidence systematic review software. (2013). Veritas Health Innovation, Melbourne, Australia. Retrieved from www.covidence.org
Cushman, E. (1996). The rhetorician as an agent of social change. College Composition and Communication, 47(1), 7–28.
Dahler-Larsen, P. (2000). Surviving the routinization of evaluation: The administrative use of evaluations in Danish municipalities. Administration & Society, 32(1), 70–92.
Datnow, A., & Park, V. (2015). Data use—for equity. Educational Leadership, 72(5), 48–54.
Dencik, L. & Hintz, A. Redden, J. & Trere, E. (2019). Exploring data justice: Conceptions, applications and directions, information, communication & society. Applications and Directions, Information, Communication & Society, 22, 873–881.
Desautels, G., & Jacob, S. (2012). The ethical sensitivity of evaluators: A qualitative study using a vignette design. Evaluation, 18(4), 437–450.
Dodgson, J. E. (2019). Reflexivity in Qualitative Research. Journal of Human Lactation, 35(2), 220–222.
Dunn, A. G., Coiera, E., Mandl, K. D., Bourgeois, F. T. (2016). Conflict of interest disclosure in biomedical research: a review of current practices, biases, and the role of public registries in improving transparency. Research Integrity and Peer Review, 1, 1.
Eckstein, L., Garrett, J. R., & Berkman, B. E. (2014). A framework for analyzing the ethics of disclosing genetic research findings. The Journal of Law, Medicine & Ethics, 42(2), 190–207.
Floridi, L., & Taddeo, M. What is data ethics? Philosophical Transactions, Series A, 374(2083).
Fraser, N., & Honneth, A. (2003). Redistribution or recognition? A political-philosophical exchange. London; New York: Verso Books.
Gaddy, M., & Scott, K. (2020). Principles for advancing equitable data practice. Washington, DC: Urban Institute.
Garcia, J., Froehlic, R., McGuire-Kuletz, M., & Rejiester, N. (2009). Comparing two training strategies to increase competence in solving ethical dilemmas. Rehabilitation Research, Policy, and Education 23(4), 223–232.
Gawthrop, J. C., & Uhlemann, M. R. (1992). Effects of the problem-solving approach in ethics training. Professional Psychology: Research and Practice, 23(1), 38–42.
Geelhoed, R. J., Phillips, J. C., Fischer, A. R., Shpungin, E., & Gong, Y. (2007). Authorship decision making: An empirical investigation. Ethics & Behavior, 17(2), 95–115.
Gorman, D. M., & Conde, E. (2007). Conflict of interest in the evaluation and dissemination of “model” school-based drug and violence prevention programs. Evaluation and Program Planning, 30(4), 422–429.
de Graaf, G., & Hertogh, M. (2020). Policy research under pressure: the case of the Ministry of Justice in the Netherlands. Journal of European Public Policy, 29(3), 468–489.
Grant, M. J., & Booth, A. (2009). A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information & Libraries Journal, 26(2), 91–108.
Greenhalgh, T., Robert, G., MacFarlane, F., Bate, P., Kyriakidou, O., & Peacock, R. (2005). Storylines of research in diffusion of innovation: A meta-narrative approach to systematic review. Social Science and Medicine, 61(2), 417–430.
Gubazire, B. B. (2022). Stewardship: An ethics for environmental respectability in Africa. E-Journal of Humanities, Arts and Social Sciences, 2, 299–309.
Guillemin, M., & Gillam, L. (2004). Ethics, reflexivity, and “Ethically Important Moments” in research. Qualitative Inquiry, 10(2), 261–280.
Haig, B. (2005). Exploratory factor analysis, theory generation and scientific method. Multivariate Behavioral Research, 40(3), 305–329.
Hayden, E, Rundell, D., & Smyntek-Gworek, S. (2013) Adaptive expertise: a view from the top and the ascent, Teaching Education, 24:4, 395–414.
Häyry, M. (2005). Precaution and solidarity. Cambridge Quarterly of Healthcare Ethics, 14(2), 199–206.
Health Alliance of Canada, C. (2013). Health Ethics Guide THIRD EDITION. Retrieved from www.chac.ca
Heimer, C. A., & Petty, J. (2010). Bureaucratic ethics: IRBS and the legal regulation of human subjects research. Annual Review of Law and Social Science, 6, 601–626.
Hood, S., Hopson, R. K., & Kirkhart, K. E. (2015). Culturally responsive evaluation. In Handbook of Practical Program Evaluation (pp. 281–317). John Wiley & Sons, Ltd.
Hopson, R. K. (2014). Justice signposts in evaluation theory, practice, and policy. New Directions for Evaluation, 142, 83–94.
House, E., & Howe, K. (1999). Values in evaluation and social research. California: SAGE.
Howe, B., Jagadish, H. V., & Stoyanovich, J. (2020). The future of privacy is data equity. Report for Frameworks for Integrative Data Equity Systems (FIDES) and Foundations of Responsible Data Science Workshop. Washington, DC.
International Organization for Cooperation in Evaluation (IOCE). (n.d.). VOPE directory. Retrieved from https://ioce.net/vope-directory/
Johnson, S. B., Lucivero, F., Zimmermann, B. M., Stendahl, E., Samuel, G., Phillips, A., & Hangel, N. (2022). Ethical reasoning during a pandemic: results of a five country European study. AJOB Empirical Bioethics, 13(2), 67–78.
Karni, E. (1998). Impartiality: Definition and Representation. Econometrica, 66, 1405–1416.
Lamping, J. L. (2016). Ethical dilemmas and obedience to authority: Examining evaluators’ ethical decision making (PhD Thesis). University of Wisconsin-Stout, WI.
Lather, P. (1986). Research as praxis. Harvard Educational Review 56(3), 257–278.
Lavallée, L. (2014). Anti-oppression research. The SAGE Encyclopedia of Action Research (pp. 41–44).
Lawrence, D. J. (2007). The four principles of biomedical ethics: A foundation for current bioethical debate. Journal of Chiropractic Humanities, 14, 34–40.
Lee, M. (2005). Public reporting. In J. Rabin (Ed.), Encyclopedia of public administration & public policy (pp. 239–243). Taylor & Francis.
Leone, L., Stame, N., & Tagle, L. (2016). Exploring ethical issues and conditions for institutionalizing evaluation in the public sector. Evaluation, 22(2), 149–167.
LSE GV314 Group and Page EC. (2014). Evaluation under contract: Government pressure and the production of policy research. Public Administration, 92(1): 224–239.
Mabry, L. (1999). Circumstantial ethics. American Journal of Evaluation, 20(2), 199–212.
MacKay, E., & O’Neill, P. (1992). What creates the dilemma in ethical dilemmas? Examples from psychological practice. Ethics & Behavior, 2(4), 227–244.
McClelland A. (2017). To report, or not to report? In Concordia University. Retrieved from https://www.concordia.ca/cunews/offices/vprgs/sgs/public-scholars/2017/08/15/to-report-or-not-to-report.html
McKillip, J., & Garberg, R. (1986). Demands of the joint committee’s standards for educational evaluation. Evaluation and Program Planning, 9, 325–333.
Maxwell, J. A., & Mittapalli, K. (2015). Realism as a stance for mixed methods research. In A. Tashakkori and C. Teddlie (Eds.), SAGE Handbook of Mixed Methods in Social & Behavioral Research (pp. 145–168). SAGE Publications.
Mertens, D. M., & Catsambas, T. (2021). Ethical Practice through a Transformative Lens and Methodological Implications in Evaluation. In R. D. van den Berg, P. Hawkins, & N. Stame (Eds.), Ethics for evaluation beyond “doing no harm” to “tackling bad” and “doing good” (1st ed., pp. 164–187).
Meyer, S. B., & Lunnay, B. (2013). The application of abductive and retroductive inference for the design and analysis of theory-driven sociological research. Sociological Research Online, 18(1), 86–96.
Miller, F. A., Christensen, R., Giacomini, M., & Robert, J.S. (2008). Duty to disclose what? Querying the putative obligation to return research results to participants. Journal of Medical Ethics, 34(3), 210–213.
Mohai, P., Pellow, D., & Roberts, J. T. (2009). Environmental justice. Annual Review of Environment and Resources, 34(1), 405–430.
Moher, D., Liberati, A., Tetzlaff, J., & Altman, D. G. (2009). Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ, 339, b2535.
Moore, M. (2008). The ethics of care and justice. Women & Politics, 20(2), 1–16.
Morey, B., Chang, R. C., Thomas, K. B., Tulua, A., Penaia, C., Tran, … Ponce, N. (2021). No Equity without data equity: Data reporting gaps for Native Hawaiians and Pacific Islanders as structural racism. Journal of Health Politics, Policy and Law, 47(2), 159–200.
Morris, M. (2005). Ethics. In S. Mathison (Ed.) Encyclopedia of Evaluation. University of British Columbia: SAGE Publications, Inc.
Morris, M. (2006). Foundation officers, evaluation, and ethical problems: A pilot investigation. Evaluation and Program Planning, 30(4), 410–415.
Morris, M. (2015). Research on evaluation ethics: Reflections and an agenda. New Directions for Evaluation, 148, 31–42.
Morris, M., & Clark, B. (2013). You want me to do what? Evaluators and the pressure to misrepresent findings. American Journal of Evaluation, 34(1), 57–70.
Morris, M., & Cohn, R. (1993). Program evaluators and ethical challenges: A national survey. Evaluation Review, 17(6), 621–642.
Morris, M., & Jacobs, L. R. (2000). You got a problem with that? Exploring evaluators’ disagreements about ethics. Evaluation Review, 24(4), 384–406.
Mylopoulos, M., Kulasegaram, K., & Woods, N. N. (2018). Developing the experts we need: Fostering adaptive expertise through education. Journal of Evaluation in Clinical Practice, 24(3), 674–677.
Narasimhan, S., & Chandanabhumma, P. P. (2021). A scoping review of decolonization in Indigenous-focused health education and behavior research. Health Education and Behavior, 48(3), 306–319.
Newey, C. (2022). Impartiality in Moral and Political Philosophy. Oxford University Press.
Newman, K. L. (1993). Procedural justice and ethical decision making. Social Justice Research, 6(1), 113–134.
Newman, D. L., & Brown, R. D. (1992). Violations of evaluation standards: Frequency and seriousness of occurrence. Evaluation Review, 16(3), 219–234.
Nijhawan, L., Janodia, M., Muddukrishna, B., Bhat, K. M., Bairy, K. L., Udupa, N., & Musmade, P. B. (2013). Informed consent: Issues and challenges. Journal of Advanced Pharmaceutical Technology & Research, 4(3), 134–140.
Okougbo, P. O., & Okike, E. N. (2021). Teaching strategies in accounting ethics: a students’ assessment of preferred teaching styles. Congent Social Sciences, 7(1), 1950306.
Ontario Human Rights Commission. (2013). The Ontario Human Rights Code. Retrieved from https://www.ohrc.on.ca/en/ontario-human-rights-code
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T.C., Mulrow, … Moher, D. (2021). The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ, 372, n71.
Palmer, C. (2006). Stewardship: A case study in environmental ethics. In R. J. Berry (Ed.), Environmental Stewardship: Critical Perspectives, Past and Present (1st ed., pp. 67–86). Bloomsbury 3PL.
Picciotto, R. (2021). Evaluation as a social practice: Disenchantment, rationalities and ethics. Evaluation and Program Planning, 87, 101927.
Place, K. R. (2015). Exploring the role of ethics in public relations program evaluation. Journal of Public Relations Research, 27(2), 118–135.
Pleger, L., & Sager, F. (2018). Betterment, undermining, support and distortion: A heuristic model for the analysis of pressure on evaluators. Evaluation and Program Planning, 69, 166–172.
Quayle, A. F., & Sonn, C. C. (2019). Amplifying the voices of Indigenous Elders through community arts and Narrative inquiry: Stories of oppression, psychosocial suffering, and survival. American Journal of Community Psychology, 64(1–2), 46–58.
Qualtrics. (2005). Qualtrics, Provo, Utah. Retrieved from https://www.qualtrics.com/
Rallis, S. F., & Rossman, G. B. (2009). Ethics and trustworthiness. In J. Heigham & R. A. Croker (Eds.), Qualitative Research in Applied Linguistics: A Practical Introduction (pp. 263–287). Palgrave Macmillan UK.
Reynolds, S. J. (2006). A neurocognitive model of the ethical decision-making process: Implications for study and practice. Journal of Applied Psychology, 91(4), 737–748.
Richardson, H. S. (1990). Specifying Norms as a way to resolve concrete ethical problems. Philosophy & Public Affairs, 19(4), 279–310. https://www.jstor.org/stable/2265316
Roach, C. M., Hollis, T. I., McLaren, B. E., & Bavington, D. L. Y. (2006). Ducks, bogs, and guns: a case study of stewardship ethics in Newfoundland. Ethics & the Environment, 11(1), 43+.
Rodwin, M. A. (2018) Attempts to redefine conflicts of interest. Accountability in Research, 25(2), 67–78.
Rosenbaum, S. (2010). Data governance and stewardship: Designing data stewardship entities and advancing data access. Health Services Research, 45(5p2), 1442–1455.
Schlosberg, D., & Collins, L. B. (2014). From environmental to climate justice: climate change and the discourse of environmental justice. WIREs Climate Change, 5(3), 359–374.
Schröder-Bäck, P., Duncan, P., Sherlaw, W., Brall, C., & Czabanowska, K. (2014). Teaching seven principles for public health ethics: towards a curriculum for a short course on ethics in public health programmes. BMC Medical Ethics, 15(1), 73.
Schroeder, D., Chatfield, K., Singh, M., Chennells, R., & Herissone-Kelly, P. (2019). The four values framework: Fairness, respect, care and honesty. In D. Schroeder, K. Chatfield, M. Singh, R. Chennells, & P. Herissone-Kelly (Eds.), Equitable Research Partnerships: A Global Code of Conduct to Counter Ethics Dumping (pp. 13–26). Springer International Publishing.
Schwandt, T. A. (1989). Recapturing moral discourse in evaluation. Educational Researcher, 18(8), 11–17.
Schwandt, T. A. (2017). Professionalization, ethics, and fidelity to an evaluation ethos. American Journal of Evaluation, 38(4), 546–553.
Schweigert, F. J. (2007). The priority of justice: A framework approach to ethics in program evaluation. Evaluation and Program Planning, 30(4), 394–399.
Scriven, M. (2008). The concept of a transdiscipline: And of evaluation as a transdiscipline. Journal of MultiDisciplinary Evaluation, 5(10), 65–66. https://journals.sfu.ca/jmde/index.php/jmde_1/article/view/161.
Scriven, M. (2016) Roadblocks to recognition and revolution. American Journal of Evaluation 37(1): 27–44.
Sheinfeld, S. N., & Lord, G. L. (1981). The ethics of evaluation researchers: An exploration of value choices. Evaluation Review, 5(3), 377–416.
Silver, D. (2021). Re-purposing evaluation to learn about social justice: Reconfiguring epistemological politics through the regulative ideal of “participatory parity.” Evaluation, 27(3), 382–399.
Smith-Morris, C. (2007). Autonomous individuals or self-determined communities? The changing ethics of research among Native Americans. Human Organization, 66(3).
Spielthenner, G. (2015). The principle-based method of practical ethics. Health Care Analysis, 25(3), 275–289.
Steinke, E. E. (2004). Research ethics, informed consent, and participant recruitment. Clinical Nurse Specialist CNS, 18(2), 88–97.
The Arthur W. Page Center. (2022). Public Relations Ethics: Ethical principles of responsibility and accountability. Retrieved from https://www.pagecentertraining.psu.edu/public-relations-ethics/ethics-in-crisis-management/lesson-1-prominent-ethical-issues-in-crisis-situations/ethical-principles-of-responsibility-and-accountability/
The Ontario Human Rights Code | Ontario Human Rights Commission. (n.d.). Retrieved from https://www.ohrc.on.ca/en/ontario-human-rights-code
Toulmin, S. (1981). The tyranny of principles. Hastings Center Report, 11(6), 31–39.
Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans (TCPS 2). (2018). Chapter 1: ethics framework. Retrieved from https://ethics.gc.ca/eng/tcps2-eptc2_2018_chapter1-chapitre1.html#b
Tuhiwai Smith, L. (2012). Decolonizing methodologies: Research and Indigenous peoples. (2nd ed.). London: Zed Books.
Turner, D. (2003). Evaluation ethics and quality: Results of a survey of Australasian Evaluation Society member. Report, AES Ethics Committee, November.
van den Berg, R. D. (2021). Ethical guidance enabling evaluation’s role in tackling bad and doing good. In R. D. van den Berg, P. Hawkins, & N. Stame (Eds.), Ethics for Evaluation Beyond “doing no harm” to “tackling bad” and “doing good” (1st ed., pp. 17–44).
van den Berg, R. D., Hawkins, P., & Stame, N. (2021). A Framework for Approaching Ethics in Evaluation. In R. D. van den Berg, P. Hawkins, & N. Stame (Eds.), Ethics for evaluation beyond “doing no harm” to “tackling bad” and “doing good” (1st ed., pp. 2–16).
Varelius, J. (2006). The value of autonomy in medical ethics. Medicine, Health Care, and Philosophy, 9(3), 377–388.
Westaby, K., Williams, T. M., Robinson, N., & Connors, E. (2019). Being responsive: The first assessment of culturally responsive evaluation in Wisconsin: Findings from the 2017 survey. Milwaukee, WI: ¡Milwaukee Evaluation!.
Walter, M., & Suina, M. (2018). Indigenous data, indigenous methodologies and indigenous data sovereignty. International Journal of Social Research Methodology, 22, 1–11.
Watts, B., Castillo, R., Akwetey, J., & Pham, D. (2023). Program Evaluation Standards in Evaluation Scholarship and Practice. Journal of MultiDisciplinary Evaluation, 19(43), 103–114.
Wong, G., Greenhalgh, T., Westhorp, G., Buckingham, J., & Pawson, R. (2013). RAMESES publication standards: meta-narrative reviews. BMC Medicine, 11(20), 1–15.
Yarbrough, D. B., Shulha, L. M., Hopson, R., & Caruthers, F. A. (2010). The program evaluation standards: A guide for evaluators and evaluation users (3rd ed.). Thousand Oaks, CA: Corwin Press.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2023 Betty Onyura, Emilia Main, Claudia Barned, Alexandra Wong, Tin D. Vo, Nivetha Chandran, Nazi Torabi, Deena M. Hamza
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Authors contributing to The Canadian Journal of Program Evaluation agree to release their articles under the Creative Commons Attribution-Noncommercial 4.0 (CC-BY-NC) license. This licence allows this work to be copied, distributed, remixed, transformed, and built upon for any purpose provided that appropriate attribution is given, a link is provided to the license, and changes made were indicated.
Authors retain copyright of their work and grant the journal right of first publication.
Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
