Missing Data in Evaluation Research

Abstract

Although research conducted in applied settings is frequently hindered by missing data, there is surprisingly little practical advice concerning effective methods for dealing with the problem. The purpose of this article is to describe several alternative methodsfor dealing with incomplete multivariate data and to examine the effectiveness of these methods. It is concluded that pairwise deletion and listwise deletion are among the least effective methods in terms of approximating the results that would have been obtained had the data been complete, whereas replacing missing values with estimates based on correlationalprocedures generally produces the most accurate results. In addition, some descriptive statistical procedures are recommended that permit researchers to investigate the causes and consequences of incomplete data more fully.

The impact of patient adherence on health outcomes for patients with chronic disease in the medical outcomes study

“…The comorbidity measure was available only on those who completed a medical history interview; those who did not (26% of the panel) were assigned the sample mean on this measure to maintain an adequate sample size. A dummy variable indicating whether or not the patient had these data available was included in the analysis to determine if those who received the medical history differed on the dependent measures from those who did not (Raymond, 1986). …”

The impact of four missing data techniques on validity estimates in human resource management

“…Monte Carlo investigations have typically found that listwise deletion is the least accurate approach for estimating correlations and regression coefficients (Afifi & Elashoff, 1969;Beale & Little, 1975;Buck, 1960;Gleason & Staelin, 1975;Raymond, 1986). Pairwise deletion is typically more accurate (Gleason & Staelin, 1975;Kim & Curry, 1977) and regression imputation is most accurate (Beale & Little, 1975;Buck, 1960;Chan & Dunn, 1972;Gleason & Staelin, 1975;Raymond & Roberts, 1987).…”

Predicting the academic achievement of Puerto Rican and Mexican-American ninth-grade students

“…In addition, 91 percent of all subjects reported their mothers to be present in the household, while only 66 percent of the sample reported fathers to be present at home at the time of the study (see Stevenson andBaker, 1987, andBianchi, 1984). Consistent with Raymond (1986) and Ward and Clark (1991), a procedure using regression analysis to estimate the missing values for mother's education was used. First, mother's education was regressed on father's education to obtain predicted values of mother's education.…”

“…Since some values were still missing, the use of Spanish in the home was included as the second most highly correlated variable with mother's education (r = .320, p = .000). The "use of Spanish in the home" was missing for only 0.4 percent of the sample; therefore, mean substitution was used to replace the missing values for Spanish use in the home (Raymond, 1986). The regression prodecure resulted in observed or estimated values for mother's education for all members of the sample.…”