Comparison of Smartphone and Online Computer Survey Administration

Abstract

The dramatic rise of smartphones has profound implications for survey research. Namely, can smartphones become a viable and comparable device for self-administered surveys? The current study is based on approximately 1,500 online U.S. panelists who were smartphone users and who were randomly assigned to the mobile app or online computer mode of a survey. Within the survey, we embedded several experiments that had been previously tested in other modes (mail, PC web, mobile web). First, we test whether responses in the mobile app survey are sensitive to particular experimental manipulations as they are in other modes. Second, we test whether responses collected in the mobile app survey are similar to those collected in the online computer survey. Our mobile survey experiments show that mobile survey responses are sensitive to the presentation of frequency scales and the size of open-ended text boxes, as are responses in other survey modes. Examining responses across modes, we find very limited evidence for mode effects between mobile app and PC web survey administrations. This may open the possibility for multimode (mobile and online computer) surveys, assuming that certain survey design recommendations for mobile surveys are used consistently in both modes.

Technology Options for Engaging Respondents in Self-Administered Questionnaires and Remote Interviewing

“…However, the rate of growth in mobile use has increased rapidly overall in recent years, and it appears it will continue to grow as coverage for more-traditional communication modes (e.g., landline telephones) continues to decline. We agree with the recommendation of Wells et al (2014) that smartphones can be a comparable device for short and optimized surveys.…”

“…Though Wells et al (2014) later found contradictory results that showed no respondent avoidance of half-open "other" categories, they did find that computer respondents provided significantly longer responses to open-ended questions than mobile respondents. This was also the case in a previous study done by Mavletova (2013).…”

“…We are unable to draw a conclusion on this topic because of limited research that measures respondent burden related to using SMS/IM, regardless of the type of keyboard used. However, on the basis of research findings that openended text boxes receive shorter input on mobile devices compared to computers (Wells et al, 2014;Mavletova, 2013), it is possible that SMS responses may be shorter because of the increased burden of typing on either full or number-board keyboards compared with computer keyboards.…”

“…One study (Wells et al, 2014) did just that and ran experiments between mobile and web users to compare the two modes as well as comparing mobile behaviors to that of other modes. One of their findings was that mobile respondents followed the same previously established patterns of other modes in that high-frequency scales resulted in higher frequencies reported and larger open-ended text boxes yielded longer responses.…”

Disclosure of sensitive behaviors across self-administered survey modes: a meta-analysis

“…On the one hand, it might be argued that with the widespread availability of web-based and mobile devices (cf. Mavletova & Couper, 2013;Van Heerden, Norris, Tollman, Stein, & Richter, 2014;Wells, Bailey, & Link, 2014), paper-and-pencil surveys will soon become outdated and mode differences should be of no major concern to survey specialists. For example, data from Germany show that in the year 2000 market research firms administered paper-and-pencil surveys about four times more often than computerized formats, whereas this ratio reversed during the subsequent decade; today computerized surveys are administered over four times more often than paper-and-pencil formats (ADM, 2014).…”