Methodological and Conceptual Challenges in Studying Evaluation Process Use

Abstract

This article discusses methodological and conceptual challenges in empirically studying process use. The main difficulty lies in disentangling cause (here the evaluation process) and effects (here indicators of process use). The evaluation researcher not only needs to take into account all relevant factors regarding the evaluator, the participants, the evaluation context, and the evaluation approach and implementation, but also needs to base the research on a valid operationalization of process use. The article was inspired by the author’s experiences in conducting an exploratory study of process use in the context of two expert-facilitated self-evaluation projects involving five program staff. Before larger-scale studies can establish more generalizable knowledge on process use, evaluation researchers should engage in high-quality, real-time, in-depth qualitative studies to better understand the complex interactions at play and to help build a solid operationalization of this relevant construct.