Designing and Applying Project Fidelity Assessment for a Teacher Implemented Middle School Instructional Improvement Pilot Intervention

Abstract

This article argues that intervention pilot test evaluations have focused insufficient attention on the measurement of project fidelity and the subsequent use of fidelity results for (a) interpreting variations in project outcomes and (b) understanding the rationale for teachers’ deviations from implementation protocols. The authors report on the establishment and application of an evaluation methodology for measuring and analyzing implementation fidelity for a middle school instructional improvement pilot project. The authors found that the highest implementation fidelity scores were not correlated with the most desirable project outcomes, as lower fidelity scores—in the 70–79% range—produced the most favourable gains on pre-post student outcomes. Moreover, application of the fidelity evaluation methodology provided insight into teacher deviation from implementation protocols; such deviation from the implementation protocols typically reflected meaningful professional classroom judgements.