‘Qualitative’ Evaluation as Symbolic Policy Making

Evaluating Work Training Programs for Quebec Welfare Recipients

Abstract

“Qualitative” program evaluation usually insists on the need for “impartial” representation of stakeholder views, whereas “symbolic policy making” favors program evaluation conclusions that support existing programs. These contradictory pressures can lead to evaluations that favor some stakeholder views at the expense of others. An example of a qualitative evaluation where this occurred is the Quebec government program Mésures de relance, aimed at helping welfare recipients re-enter the labour force. Several stakeholder groups were involved in the program, but only one group—program participants—was thoroughly consulted. It can be argued that employers, whose cooperation in providing work assignments was vital to the program’s success, could have provided useful input into the evaluation. However, their inclusion might have impaired the evaluation’s ability to arrive at the desired symbolic conclusions: that if certain improvements were made the program could be successful. This example suggests that qualitative evaluation is only useful to the extent that it actually does provide impartial representation of stakeholder groups, and especially of those that are important to the success of the program. The article is followed by a brief critique by Dale H. Poel, who argues that evaluators, rather than the qualitative method itself, are the source of political bias. The author appends a brief rejoinder.