Effects of phone versus mail survey methods on the measurement of health-related quality of life and emotional and behavioural problems in adolescents

Abstract

BackgroundTelephone interviews have become established as an alternative to traditional mail surveys for collecting epidemiological data in public health research. However, the use of telephone and mail surveys raises the question of to what extent the results of different data collection methods deviate from one another. We therefore set out to study possible differences in using telephone and mail survey methods to measure health-related quality of life and emotional and behavioural problems in children and adolescents.MethodsA total of 1700 German children aged 8-18 years and their parents were interviewed randomly either by telephone or by mail. Health-related Quality of Life (HRQoL) and mental health problems (MHP) were assessed using the KINDL-R Quality of Life instrument and the Strengths and Difficulties Questionnaire (SDQ) children's self-report and parent proxy report versions. Mean Differences ("d" effect size) and differences in Cronbach alpha were examined across modes of administration. Pearson correlation between children's and parents' scores was calculated within a multi-trait-multi-method (MTMM) analysis and compared across survey modes using Fisher-Z transformation.ResultsTelephone and mail survey methods resulted in similar completion rates and similar socio-demographic and socio-economic makeups of the samples. Telephone methods resulted in more positive self- and parent proxy reports of children's HRQoL (SMD ≤ 0.27) and MHP (SMD ≤ 0.32) on many scales. For the phone administered KINDL, lower Cronbach alpha values (self/proxy Total: 0.79/0.84) were observed (mail survey self/proxy Total: 0.84/0.87). KINDL MTMM results were weaker for the phone surveys: mono-trait-multi-method mean r = 0.31 (mail: r = 0.45); multi-trait-mono-method mean (self/parents) r = 0.29/0.36 (mail: r = 0.34/0.40); multi-trait-multi-method mean r = 0.14 (mail: r = 0.21). Weaker MTMM results were also observed for the phone administered SDQ: mono-trait-multi-method mean r = 0.32 (mail: r = 0.40); multi-trait-mono-method mean (self/parents) r = 0.24/0.30 (mail: r = 0.20/0.32); multi-trait-multi-method mean r = 0.14 (mail = 0.14). The SDQ classification into borderline and abnormal for some scales was affected by the method (OR = 0.36-1.55).ConclusionsThe observed differences between phone and mail surveys are small but should be regarded as relevant in certain settings. Therefore, while both methods are valid, some changes are necessary. The weaker reliability and MTMM validity associated with phone methods necessitates improved phone adaptations of paper and pencil questionnaires. The effects of phone versus mail survey modes are partly different across constructs/measures.

Psychische Auffälligkeiten und psychosoziale Beeinträchtigungen bei Kindern und Jugendlichen im Alter von 3 bis 17 Jahren in Deutschland – Prävalenz und zeitliche Trends zu 2 Erhebungszeitpunkten (2003–2006 und 2009–2012)

“…In einer methodischen Studie konnte gezeigt werden, dass der SDQ sensitiv gegenüber dem Erhebungsmodus ist [36]. Dabei wurden bei der Befragung mit der auch diesem Beitrag zugrunde gelegten Elternversion des SDQ mit Blick auf die Subskalen in der telefonischen Erhebung mehr Verhaltensprobleme und mehr prosoziales Verhalten, jedoch weniger Peer-Probleme berichtet als in der schriftlichen Befragung, bei allerdings geringen Effektstärken [36] …”

“…Mit Blick auf die Subskalen waren analog früherer Befunde die internen Konsistenzen für die Subskala "Hyperaktivität" am höchsten, diejenigen für die Subskala "Verhaltensprobleme" am geringsten [13,28,31]. Bei der telefonischen Erhebung in der KiGGS Welle 1 waren die internen Konsistenzen insgesamt etwas niedriger als beim schriftlichen Assessment in der KiGGS-Basiserhebung, was einem ähnlichen Befund der methodischen Vergleichsstudie von Erhart et al entspricht [36]. [47].…”

“…Auch die Tatsache, dass von der KiGGS-Basiserhebung bis zu KiGGS Welle 1 keine zeitlichen Veränderungen in der Lebenszeitprävalenz für eine diagnostizierte ADHS, mithin im Inanspruchnahmeverhalten bei Hyperaktivitätsproblemen, festgestellt werden konnten [48], spricht für die Validität dieses Ergebnisses. Die Hyperaktivitätsskala hat sich zudem als invariant gegenüber möglichen Modeeffekten erwiesen [36]. Diese Stabilität in der Zusammenschau mit ihren guten psychometrischen Eigenschaften, die die Unterskala "Hyperaktivität" in vielen Studien gezeigt hat [49], weisen sie als valide und reliabel für die Erfassung von ADHSSymptomen aus.…”

Short Form health surveys and related variants in spinal cord injury research: A systematic review

“…This finding has been observed in disease-specific groups, including SCI, as well as generic groups. [29][30][31][32] Given the likelihood that systematic differences in HRQoL could be attributable to the method of data collection alone, an explicit objective of the current review is to establish the state-ofplay for alternative modes of survey administration in the context of SCI. Due to fundamental differences in measurement objectives between profile measures of HRQoL (such as the SF-36 and SF-12) and utility instruments (e.g.…”

Does mode of administration affect health-related quality-of-life outcomes after stroke?

“…Postal interviews were found to have higher completion rates than telephone interviews (Erhart et al, 2009;McHorney et al, 1994) and face-to-face interviews (Smeeth et al, 2001). However, postal interviews were also characterised by higher percentages of missing data (McHorney et al, 1994;Smeeth et al, 2001), although Hawthorne et al (2003) reported no significant differences between the rate of missing data for telephone and postal interviews.…”