Missing in Action
Indigenous Knowledge Systems in Evaluation of Comprehensive Community Initiatives
DOI:
https://doi.org/10.3138/cjpe.69099Keywords:
Aboriginal, child welfare, community development, multisector collaboration, social change, systems, systems thinkingAbstract
Comprehensive community initiatives (CCIs) coordinate social and structural change across multiple community sectors and represent promising approaches to complex social problems. Nowhere is this more relevant than for Indigenous children and families. However, strategies to evaluate initiatives to enhance Indigenous well-being must be meaningful to Indigenous communities. A review of literature regarding evaluation of CCIs identified various principles, case studies, methodologies, and methods grounded in Western ways of knowing and approaches to research. Research that engages with Indigenous-led comprehensive community initiatives is needed to enhance evaluation practices for CCIs that enables resurgence of Indigenous traditions and worldviews.
References
Abboud, R., & Claussen, C. (2016). The use of Outcome Harvesting in learning-oriented and collaborative inquiry approaches to evaluation: An example from Calgary, Alberta. Evaluation and Program Planning, 59, 47 – 54. https://doi.org/10.1016/j.evalprogplan.2016.08.005. Medline:27591484
Amed, S., Shea, S., Pinkney, S., Wharf Higgins, J., & Naylor, P. J. (2016). Wayfinding the live 5-2-1-0 initiative: At the intersection between systems thinking and community-based childhood obesity prevention. International Journal of Environmental Research and Public Health, 13(6), 614 – 630. https://doi.org/10.3390/ijerph13060614. Medline:27338432
Archibald, T., Sharrock, G., Buckley, J., & Cook, N. (2016). Assumptions, conjectures, and other miracles: The application of evaluative thinking to theory of change models in community development. Evaluation and Program Planning, 59, 119 – 127. https://doi.org/10.1016/j.evalprogplan.2016.05.015. Medline:27324286
Auspos, P., & Kubisch, A. C. (2012). Performance management in complex, place-based work: What it is, what it isn’t, and why it matters. Washington, DC: The Aspen Institute. Retrieved from http://www.aspeninstitute.org/sites/default/files/content/images/rcc/Aspen_Performance_Management.pdf
Australian Institute of Health and Welfare. (2010). Child Protection Australia 2008–09. Retrieved from https://www.aihw.gov.au/getmedia/b0df902e-0df3-43fd-a9f32679cf753f91/10859.20100901.pdf.aspx
Bennett, M., & Auger, A. (2013). First Nations and non-Aboriginal children in child protection services. Prince George, BC: National Collaborating Centre for Aboriginal Health. Retrieved from http://www.nccah-ccnsa.ca/Publications/Lists/Publications/Attachments/7/protective_services_EN_web.pdf
Blackstock, C., & Trocmé, N. (2005). Community-based child welfare for Aboriginal children: Supporting resilience through structural change. In M. Ungar (Ed.), Handbook for working with children and youth: Pathways to resilience across cultures and contexts (pp. 105 – 120 ). Thousand Oaks, CA: Sage.
Blair, E., Busain, L. J., & Tucker, P. (2015). Pathways to opportunity: Applying the principles of an effective, Indigenous comprehensive community initiative toward a thriving community. Retrieved from https://conservancy.umn.edu/bitstream/handle/11299/173965/Blair_Busian_Tucker-Pathways%20to%20Opportunity.pdf?sequence=1&isAllowed=y
Blamey, A., & Mackenzie, M. (2007). Theories of change and realistic evaluation: Peas in a pod or apples and oranges? Evaluation, 13(4), 439 – 455. https://doi.org/10.1177/1356389007082129
Bradford, N. (2005). Place-based public policy: Towards a new urban and community agenda for Canada. Family Network Research Report F51. Ottawa, ON: Canadian Policy Research Networks. Retrieved from http://neighbourhoodchange.ca/wp-content/uploads//2011/06/Bradford-2005-Place-based-public-policy-agenda.pdf
Bradford, N., & Chouinard, J. A. (2009). Learning through evaluation? Reflections on two federal community-building initiatives. Canadian Journal of Program Evaluation, 24(1), 51 – 77.
Brant Castellano, M. (2004). Ethics of Aboriginal research. International Journal of Indigenous Health, 1(1), 98-114.
Brown, P. (2010). Evaluating and learning from community change. In A. Kubisch, P. Auspos, P. Brown, & T. Dewar (Eds.), Voices from the field III: Lessons and challenges from two decades of community change efforts (pp. 95 – 105). Washington, DC: The Aspen Institute.
Cabaj, M. (2014). Evaluating collective impact: Five simple rules. The Philanthropist, 26(1), 109 – 124. Retrieved from https://www.tamarackcommunity.ca/hubfs/Resources/Publications/Evaluating%20Collective%20Impact%205%20Simple%20Rules.pdf
Canadian Centre for Policy Alternatives (CCPA) Manitoba. (2018). Na-gah mo Waabishkizi Ojijaak Bimise Keetwaatino: Singing white crane flying north: Gathering a bundle for Indigenous evaluation. Winnipeg, MB. Retrieved from https://www.policyalternatives.ca/publications/reports/na-gah-mo-waasbishkizi-ojijaak-bimise-keetwaatino-singingwhite-crane-flying
Carter, V. (2010). Factors predicting placement of urban American Indian/Alaskan Natives into out-of-home care. Children and Youth Services Review, 32(5), 657 – 663. https://doi.org/10.1016/j.childyouth.2009.12.013
Chandna, K., Vine, M. M., Snelling, S., Harris, R., Smylie, J., & Manson, H. (2019). Principles, approaches, and methods for evaluation in indigenous contexts: A grey literature scoping review. Canadian Journal of Program Evaluation, 34(1), 21 - 47. https://doi.org/10.3138/cjpe.43050
Coffman, J. (2007). A framework for evaluating systems initiatives. Build Strong Foundations for Our Youngest Children. Retrieved from http://pathways.nccp.org/assets/pdf/Coffman2007.pdf
Connell, W., Kubisch, A., Schorr, L., & Weiss, C. (1995). New approaches to evaluating community initiatives: Concepts, methods, and contexts. New York, NY: The Aspen Institute.
Cook, J. R. (2015). Using evaluation to effect social change: Looking through a community psychology lens. American Journal of Evaluation, 36(1), 107 – 117. https://doi.org/10.1177/1098214014558504
Coombe, C. (2012). Participatory approaches to evaluating community organizing and coalition building. In M. Minkler (Ed.), Community organizing and community building for health and welfare (pp. 346 – 365). New Brunswick, NJ: Rutgers University Press.
Delfabbro, P., Hirte, C., Rogers, N., & Wilson, R. (2010). The over-representation of young Aboriginal or Torres Strait Islander people in the South Australian child system: A longitudinal analysis. Children and Youth Services Review, 32, 1418 – 1425. https://doi.org/10.1016/j.childyouth.2010.06.011
Farrow, F., & Schorr, L. (2011). Expanding the evidence universe: Doing better by knowing more. Washington, DC: Center for the Study of Social Policy.
Fiester, L. (2011). Measuring change while changing measures: Learning in, and from, the evaluation of Making Connections. Baltimore, MD: The Annie E. Casey Foundation. Retrieved from http://www.aecf.org/m/resourcedoc/aecf-MeasuringChangeWhile-ChangingMeasures-2011.pdf
Foster-Fishman, P. G., Nowell, B., & Yang, H. (2007). Putting the system back into systems change: A framework for understanding and changing organizational and community systems. American Journal of Community Psychology, 39(3–4), 197 – 215. https://doi.org/10.1007/s10464-007-9109-0. Medline:17510791
Foster-Fishman, P. G., & Watson, E. R. (2012). The ABLe change framework: A conceptual and methodological tool for promoting systems change. American Journal of Community Psychology, 49(3–4), 503 – 516. https://doi.org/10.1007/s10464-011-9454-x. Medline:21735334
Gillespie, J., Supernault, G., & Abel, M. (2014). Community networking: A promising practice for Aboriginal child welfare. In D. Badry, M. Montgomery, D. Fuchs, & S. McKay (Eds.), Reinvesting in families, securing a brighter future: Voices from the Prairies (pp. 201 – 219). Edmonton, AB: Prairie Child Welfare Consortium/Alberta Centre for Child, Family and Community Research.
Hill, J. R., & Thies, J. (2010). Program theory and logic model to address the co-occurrence of domestic violence and child maltreatment. Evaluation and Program Planning, 33(4), 356 – 364. https://doi.org/10.1016/j.evalprogplan.2010.03.001. Medline:20452671
Kania, J., & Kramer, M. (2011). Collective impact. Stanford Social Innovation Review (Winter), 36 – 41. Retrieved from http://www.ssireview.org/articles/entry/collective_impact
Kaufman, J. S., Crusto, C. A., Quan, M., Ross, E., Friedman, S. R., O’Rielly, K., & Call, S. (2006). Utilizing program evaluation as a strategy to promote community change: Evaluation of a comprehensive, community-based, family violence initiative. American Journal of Community Psychology, 38(3–4), 191- 200. https://doi.org/10.1007/s10464-006-9086-8. Medline:17086485
Kawakami, A., Aton, K., Cram, F., Lai, M., & Porima, L. (2007). Improving the practice of evaluation through Indigenous values and methods: Decolonizing evaluation practice—Returning the gaze from Hawai'i and Aotearoa. Hūlili: Multidisciplinary Research on Hawaiian Well-Being, 4(1), 319 – 348.
Kelly, T. (2010). Five simple rules for evaluating complex community initiatives. Community Investments, 22(1), 19 – 22. Retrieved from http://www.frbsf.org/communitydevelopment/files/Spring_CI_2010a.pdf
Kingsley, G. T., Coulton, C. J., & Pettit, K. L. S. (2014). Strengthening communities with neighborhood data. Washington, DC: Urban Institute. Retrieved from http://www.urban.org/sites/default/files/13805_urban-kingsley.pdf
Komro, K. A., Flay, B. R., Biglan, A., & Wagenaar, A. C. (2016). Research design issues for evaluating complex multicomponent interventions in neighborhoods and communities. Translational Behavioral Medicine, 6(1), 153 – 159. https://doi.org/10.1007/s13142-015-0358-4. Medline:27012263
Kovach, M. (2009). Indigenous methodologies: Characteristics, conversations, and contexts. Toronto, ON: University of Toronto Press.
Kubisch, A., Auspos, P., Brown, P., Buck, E., & Dewar, T. (2011). Voices from the field III: Lessons and challenges for foundations based on two decades of community-change efforts. The Foundation Review, 3(1), 138 – 149. https://doi.org/10.4087/foundationreview-d-11-00010
LaFrance, J., & Nichols, R. (2010). Reframing evaluation: Defining an Indigenous evaluation framework. Canadian Journal of Program Evaluation, 23(2), 13 – 31.
Lavergne, C., Dufour, S., Trocmé, N., & Larrivée, M. (2008). Visible minority, Aboriginal and Caucasian children investigated by Canadian child protective services. Child Welfare, 87(2), 59 – 76.
Lee, K. S., & Chavis, D. M. (2012). Cross-case methodology: Bringing rigour to community and systems change research and evaluation. Journal of Community & Applied Social Psychology, 22(5), 428 – 438. https://doi.org/10.1002/casp.1131
Martin, K., & Mirraboopa, B. (2009). Ways of knowing, being and doing: A theoretical framework and methods for Indigenous and Indigenist re-search. Journal of Australian Studies, 27(76), 203 – 214. https://doi.org/10.1080/14443050309387838
Patton, M. Q. (2010). Developmental evaluation: Applying complexity concepts to enhance innovation and use. New York, NY: Guilford Press.
Peters, R. D., Petrunka, K., Khan, S., Howell-Moneta, A., Nelson, G., Pancer, S. M., & Loomis, C. (2016). Cost-savings analysis of the Better Beginnings, Better Futures community-based project for young children and their families: A 10-year follow-up. Prevention Science, 17(2), 237 – 247. https://doi.org/10.1007/s11121-015-0595-2. Medline:26319616
Provan, K. G., Veazie, M. A., Staten, L. K., & Teufel-Shone, N. I. (2005). The use of network analysis to strengthen community partnerships. Public Administration Review, 65(5), 603 – 613. https://doi.org/10.1111/j.1540-6210.2005.00487.x
Public Health Agency of Canada. (2010). Canadian incidence study of reported child abuse and neglect—2008: Major findings. Ottawa, ON. Retrieved from https://cwrp.ca/sites/default/files/publications/en/CIS-2008-rprt-eng.pdf
Rogers, P. J. (2008). Using programme theory to evaluate complicated and complex aspects of interventions. Evaluation, 14(1), 29 – 48. https://doi.org/10.1177/1356389007084674
Royal Commission on Aboriginal Peoples. (1997). Report of the Royal Commission on Aboriginal Peoples. Volume 3: Gathering strength. Ottawa, ON: Government of Canada.
Sinha, V., Trocmé, N., Fallon, B., MacLaurin, B., Fast, E., Thomas Prokop, S., ... Kenn, R. (2011). Kiskisik Awasisak: Remember the children: Understanding the overrepresentation of First Nations children in the child welfare system. Ottawa, ON: Assembly of First Nations. Retrieved from http://cwrp.ca/sites/default/files/publications/en/FNCIS-2008_March2012_RevisedFinal.pdf
Sridharan, S., Campbell, B., & Zinzow, H. (2006). Developing a stakeholder-driven anticipated timeline of impact for evaluation of social programs. American Journal of Evaluation, 27(2), 148 – 162. https://doi.org/10.1177/1098214006287990
Thoburn, J. (2007). Globalisation and child welfare: Some lessons from a cross-national study of children in out-of-home care. Social Work Monographs. Norwich, England: UEA. Retrieved from https://www.onpe.gouv.fr/system/files/ao/j_thoburn_rapport.pdf
Torjman, S., & Leviten-Reid, E. (2003). Comprehensive community initiatives. Ottawa, ON: Caledon Institute of Social Policy. Retrieved from http://fcssaa.org/sites/default/files/documents/Comprehensive%20Community%20Initiatives,%202003.pdf
Trocmé, N., Knoke, D., & Blackstock, C. (2004). Pathways to the overrepresentation of Aboriginal children in Canada’s child welfare system. Social Service Review, 78(4), 577 – 600. https://doi.org/10.1086/424545
Tuhiwai-Smith, L. (2012). Decolonizing methodologies: Research and Indigenous peoples (2nd ed.). New York, NY: Zed Books.
U.S. Department of Health and Human Services, Administration for Children and Families, Administration on Children, Youth and Families, Children’s Bureau. (2009). The AFCARS report: Preliminary estimates for FY 2008 as of October 2009 (Report #16). Retrieved from https://www.acf.hhs.gov/sites/default/files/cb/afcarsreport16.pdf
Wilson, S. (2008). Research is ceremony: Indigenous research methods. Halifax, NS: Fernwood.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2020 Judy Gillespie, Jason Albert, Shelaine Grant, Tanya MacKeigan
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Authors contributing to The Canadian Journal of Program Evaluation agree to release their articles under the Creative Commons Attribution-Noncommercial 4.0 (CC-BY-NC) license. This licence allows this work to be copied, distributed, remixed, transformed, and built upon for any purpose provided that appropriate attribution is given, a link is provided to the license, and changes made were indicated.
Authors retain copyright of their work and grant the journal right of first publication.
Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
