The Downward Trend of Survey Response Rates

Implications and Considerations for Evaluators

Abstract

Rapidly declining response rates and the associated threat of nonresponse bias call into question the validity of data obtained through telephone surveys, a tool often used in evaluation. This article explores changes in nonresponse bias over time by examining three data points (1991, 1996, and 2002) from an annual household telephone survey conducted by the University of Alberta’s Population Research Lab. Results demonstrate a substantial decline in response rates accompanied by an increasing level of bias in variables related to respondent education. Implications of these results are investigated through regression analyses and suggest that declining representation of individuals with less education could significantly impact a variety of survey variables, thus creating opportunity for opinions of the more educated to become more heavily weighted in evaluation results. In turn, such results could be used to inform government policies and programs in ways that advantage the educated middle class.