Measuring Real-Time Response in Real-Life Settings

Abstract

Real-time response (RTR) measurement is an important technique to assess human processing of information. Ever since Maier et al. verified the reliability and validity of physical RTR devices in this journal a decade ago, there has been a growing trend toward virtual measurement platforms to overcome the limitations of conventional, laboratory-based methods. We introduce the Debat-O-Meter, a novel online RTR platform for mobile devices, which allows researchers to measure how viewers perceive political debates in the setting of their private homes. We draw on a large study (N = 5,660) conducted during the 2017 German general election campaign and show that virtualized measurement indeed facilitates diverse and large N field studies while simultaneously conforming to established standards of reliability and validity.

Assessing the impact of party identification in transnational election campaigns: Evidence from the 2019 European Parliament election

“…This facilitates to abandon physical input devices and conduct field studies outside the laboratory. Methodological research has shown that virtualised RTR provides both valid and reliable data and thus corresponds to established standards of data quality (Waldvogel 2020;Waldvogel and Metz 2020;Maier, Hampe and Jahn 2016).…”

“…There is a vast body of literature on the perception and effects of televised debates on the national level (see McKinney and Carlin 2004;Maier, Faas and Maier 2014;Benoit, Hansen and Verser 2003) as well as on methodological issues when implementing real time response measurement (RTR) in the applied research design (Waldvogel 2020;Waldvogel and Metz 2020;Reinemann, Maier, Faas and Maurer 2005;Papastefanou, 2013;Maier, Hampe and Jahn 2016a;Maier, Maurer, Reinemann and Faas 2007). This research on TV debates at the national level has shown that party identification is a predominant predictor when assessing perception processes and effects of debate reception as well as the quality of RTR-data.…”

What matters for keeping or losing support in televised debates

“…Initially, more than 27,830 people were recruited to the study, complete RTR data were obtained for 5660 respondents, which by far exceeds the sample sizes commonly used in RTR studies on TV debates (see e.g. Maier et al, 2016, Waldvogel and Metz, 2020). Respondents were furthermore asked to take part in a pre- and a post-survey that contained various items needed for our analysis besides the rating data from the RTR setting.…”

“…Previous studies using RTR are, however, usually not done outside the lab (but see Maier et al, 2016) and/or they are based on very small samples (for exceptions, see Boydstun et al, 2014; Waldvogel and Metz, 2020). The sample sizes per experiment are commonly not bigger than several dozen cases.…”

“…Research drawing on this methodological tool has led to a more fine-grained picture of televised debates, viewers’ predispositions and their evaluations of the candidates. Existing work indicates that viewers’ evaluations can be altered through televised debates and that perceived performance does affect the eventual candidate evaluation (Maier et al, 2016, Maier and Faas, 2011, Maurer, 2016, McKinnon et al, 1993, Schill and Kirk, 2014, Waldvogel and Metz, 2020). At the same time, RTR studies also clearly show that predispositions and particularly identification with the party of a candidate markedly bias viewers’ evaluations (Bachl, 2017, Jarman, 2005, Maier and Faas, 2011), thus reiterating an important conclusion from contributions before the advent of RTR measures in political communication studies.…”