Application of an Evaluation Framework for Extra-Organizational Communities of Practice
Assessment and Refinement
DOI:
https://doi.org/10.3138/cjpe.69796Keywords:
capacity strengthening, communities of practice, evaluation framework, qualitative, value creationAbstract
Communities of practice (CoPs) are groups of people who work together on an ongoing basis and share knowledge and expertise. CoPs exist both within and outside of organizations, although extra-organizational CoPs have received less evaluation attention. The primary objective of this study was to assess the applicability of a multi-level, multiple-value evaluation framework for extra-organizational CoPs. Qualitative interviews were conducted with an extra-organizational CoP—the Canadian Community of Practice in Ecosystems Approaches to Health (CoPEH-Canada). The evaluation framework oriented both the member interview guide and the deductive content analysis. The findings showed that the evaluation framework was sufficiently comprehensive to capture the values generated. Following reflection on these findings, challenges in its application and suggested revisions to the framework are provided; also discussed are limitations and strengths, evaluation research next steps, and the opportunities for future applications.
References
Amin, A., & Roberts, J. (2008). Knowing in action: Beyond communities of practice. Research Policy, 37(2), 353 – 369. https://doi.org/10.1016/j.respol.2007.11.003
Barbour, L., Armstrong, R., Condron, P., & Palermo, C. (2018). Communities of practice to improve public health outcomes: A systematic review. Journal of Knowledge Management, 22(2), 326 – 343. https://doi.org/10.1108/jkm-03-2017-0111
Brisbois, B. W., Delgado, A. B., Barraza, D., Betancourt, Ó., Cole, D., Gislason, M., ... Saint-Charles, J. (2017). Ecosystem approaches to health and knowledge-to-action: Towards a political ecology of applied health-environment knowledge. Journal of Political Ecology, 24(1), 692 – 715. https://doi.org/10.2458/v24i1.20961
Brown, J. S., & Duguid, P. (1991). Organizational learning and communities-of-practice: Toward a unified view of working, learning, and innovation. Organization Science, 2(1), 40 – 57. https://doi.org/10.1287/orsc.2.1.40
Buckley, S., & Du Toit, A. (2010). Academics leave your ivory tower: Form communities of practice. Educational Studies, 36(5), 493 – 503. https://doi.org/10.1080/03055690903425532
Charron, D. F. (Ed.). (2012). Ecohealth research in practice: Innovative applications of an ecosystem approach to health. New York, NY: Springer.
Chouinard, J. A. (2013). The case for participatory evaluation in an era of accountability. American Journal of Evaluation, 34(2), 237 – 253. https://doi.org/10.1177/1098214013478142
Contandriopoulos, D., Larouche, C., & Duhoux, A. (2018). Evaluating academic research networks. Canadian Journal of Program Evaluation, 33(1), 69-89. https://doi.org/10.3138/cjpe.42159
Fuller, A., Hodkinson, H., Hodkinson, P., & Unwin, L. (2005). Learning as peripheral participation in communities of practice: A reassessment of key concepts in workplace learning. British Educational Research Journal, 31(1), 49 – 68. https://doi.org/10.1080/0141192052000310029
Gauvreau, C. A., Le Dorze, G., Kairy, D., & Croteau, C. (2019). Evaluation of a community of practice for speech-language pathologists in aphasia rehabilitation: A logic analysis. BMC Health Services Research, 19(1). https://doi.org/10.1186/s12913-019-4338-0. Medline:31358002
Glicksman, R., Ang, M., Murray, E., Simniceanu, C., Lockhart, E., Gilbert, J., ... Warde , P. (2019). Improving quality of radiation therapy care across Ontario using a community-of-practice approach. Practical Radiation Oncology, 9(2), e242 – e248. https://doi.org/10.1016/j.prro.2018.10.017. Medline:30447404
Handley, K., Sturdy, A., Fincham, R., & Clark, T. (2006). Within and beyond communities of practice: Making sense of learning through participation, identity and practice. Journal of Management Studies, 43(3), 641 – 653. https://doi.org/10.1111/j.14676486.2006.00605.x
Hennink, M. M., Kaiser, B. N., & Marconi, V. C. (2017). Code saturation versus meaning saturation: How many interviews are enough? Qualitative Health Research, 27(4), 591 – 608. https://doi.org/10.1177/1049732316665344. Medline:27670770
Hsieh, H.-F., & Shannon, S. E. (2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15(9), 1277 – 1288. https://doi.org/10.1177/1049732305276687. Medline:16204405
Johnson, C. M. (2001). A survey of current research on online communities of practice. The Internet and Higher Education, 4(1), 45 – 60. https://doi.org/10.1016/s10967516(01)00047-1
Kothari, A., Boyko, J. A., Conklin, J., Stolee, P., & Sibbald, S. L. (2015). Communities of practice for supporting health systems change: A missed opportunity. Health Research Policy and Systems, 13. https://doi.org/10.1186/s12961-015-0023-x. Medline:26208500
Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation. Cambridge, England: Cambridge University Press.
Lee, J., Suh, E.- H., & Hong, J. (2010). A maturity model based CoP evaluation framework: A case study of strategic CoPs in a Korean company. Expert Systems with Applications, 37(3), 2670 – 2681. https://doi.org/10.1016/j.eswa.2009.08.011
McCormack, C., Kennelly, R., Gilchrist, J., Hancock, E., Islam, J., Northcote, M., & Thomson, K. (2017). From dream to reality: Sustaining a higher education community of practice beyond initial enthusiasm. In J. McDonald & A. Cater-Steel (Eds.), Communities of practice: Facilitating social learning in higher education (pp. 599 – 622). Singapore : Springer.
McCullagh, S., Berbes-Blasquez, M., Feagan, M., Houle, K., Hunter, B., Lemire, M., ... Surette, C. (2012). Teaching manual: Ecosystem approaches to health teaching resources. Canadian Community of Practice in Ecosystem Approaches to Health (CoPEH-Canada). Retrieved from http://www.copeh-canada.org/en/teaching-manual.html
McKellar, K. A. (2019). Evaluating extra-organizational communities of practice (Doctoral dissertation). University of Toronto, Toronto, ON. Retrieved from http://hdl.handle.net/1807/95912
McKellar, K. A., Pitzul, K. B., Yi, J. Y., & Cole, D. C. (2014). Evaluating communities of practice and knowledge networks: A systematic scoping review of evaluation frameworks. EcoHealth, 11(3), 383 – 399. https://doi.org/10.1007/s10393-014-0958-3. Medline:25023411
Montague, S., & Porteous, N. L. (2013). The case for including reach as a key element of program theory. Evaluation and Program Planning, 36(1), 177 – 183. https://doi.org/10.1016/j.evalprogplan.2012.03.005. Medline:22607710
Nistor , N. , Daxecker , I. , Stanciu , D. , & Diekamp , O. ( 2015 ). Sense of community in academic communities of practice: Predictors and effects . Higher Education, 69( 2 ), 257 – 273 . https://doi.org/10.1007/s10734-014-9773-6
Pahl-Wostl, C. ( 2009 ). A conceptual framework for analysing adaptive capacity and multi-level learning processes in resource governance regimes . Global Environmental Change, 19( 3 ), 354 – 365 . https://doi.org/10.1016/j.gloenvcha.2009.06.001
Parkes , M. W. , Charron , D. F. , & Sánchez , A. ( 2012 ). Better together: Field-building networks at the frontiers of ecohealth research . In D. F. Charron (Ed.), Ecohealth research in practice (pp. 231 – 253 ). New York, NY : Springer
Parkes , M. W. , Saint-Charles , J. , Cole , D. C. , Gislason , M. , Hicks , E. , Le Bourdais , C. , ... Canadian Community of Practice in Ecosystem Approaches to Health Team. ( 2017 ). Strengthening collaborative capacity: Experiences from a short, intensive fi eld course on ecosystems, health and society. Higher Education Research & Development, 36( 5 ), 1031 – 1046 . https://doi.org/10.1080/07294360.2016.1263937
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2020 Kaileah A. McKellar, Whitney Berta, Rhonda Cockerill, Donald C. Cole, Johanne Saint-Charles
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Authors contributing to The Canadian Journal of Program Evaluation agree to release their articles under the Creative Commons Attribution-Noncommercial 4.0 (CC-BY-NC) license. This licence allows this work to be copied, distributed, remixed, transformed, and built upon for any purpose provided that appropriate attribution is given, a link is provided to the license, and changes made were indicated.
Authors retain copyright of their work and grant the journal right of first publication.
Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
