Program Design Can Make Outcome Evaluation Impossible

A Review of Four Studies of Community Economic Development Programs

Abstract

Between 1981 and 1990 Employment and Immigration Canada evaluated three community economic development programs: the Community Employment Strategy, the Local Employment Assistance and Development, and the Community Futures program. In retrospect, one can see that these evaluations were hindered by two problems of program design: there was no replicable treatment, and the broad, shallow interventions were unlikely to have measurable effects in an environment “noisy” with uncontrolled factors. This article reviews the evaluation studies of these three programs, the lessons learned from each study, how subsequent programs reflected those lessons, and how the methodological limitations of the studies constrained what was learned.