The Impact of Behavioural Framing Effects on Market Research Conversion Rates

Abstract

The cause-and-effect relationship between incentives and market research response rates is well documented. In general, we understand that - mediated by factors such as privacy and difficulty - the greater the compensation offered to participants, the greater propensity for task completion. However, there is little understanding of how the way in which an incentive is communicated impacts results. We believe studying this field will help insight industry practitioners who are often challenged by limited or fixed incentive budgets. By exploring and ranking framing methods, we aim to recommend to researchers the tactical options that will maximise return on investment. In this initial exploration, we have narrowed framing choices to a set identified by the field of behavioural economics. In particular, we investigate the relative impact of: (1) financial incentives, (2) the endowment effect, (3) social proof, (4) altruism and, (5) financial incentives plus loss aversion. These conditions can be considered to broadly fit within one of three categories: (a) financial incentives alone, (b) behavioural principles alone, (c) financial incentives plus behavioural principles. To replicate the real-world challenges of insight professionals, this was tested in an experimental design that sought to recruit existing Aegon customers into a new research panel. The experimental case study format also lends itself to high volumes. The results of this study are based on 91,289 recruitment emails delivered to Aegon customers across various weekday mornings. Each email contained either neutral information regarding the incentive to join - the opportunity to win up to £250 - or the same message framed using one of the experimental methods in test. Messages were repeated both in the subject line and the email body. Results were measured and analysed at the points of: (A) email open, (B) email click, (C) conversion as defined by screener completion, (D) conversion as defined by panel account created, and (E) engagement as defined by completing at least one task on the panel eligible for a point reward within 3 months of signup. Additionally, the unsubscribe rate was measured and compared across all messages. Prior to the launch of the study, our hypotheses were that email communications which utitlised the endowment effect, social proof or altruism would lead to higher conversion rates than messages regarding incentivisation. Further, we hypothesised that emails which utilised both financial incentives plus loss aversion framing in an additive capacity would deliver the highest conversion rate. Our analysis led to a rejection of all hypotheses. Ultimately, it was concluded that none of the tested framing effects performed significantly better, as per chi-squared tests, than the financial incentive group across open rate, click rate, conversion rate or engagement rate. However, the addition of loss aversion performed on par with the incentive only group for panel engagement. Further, the use of loss aversion, altruism and the endowment effect all led to significantly lower unsubscribe rates than the incentive only group. This led to a nuanced conclusion which surmised that the loss aversion tactic, when used in an additive capacity to a financial incentive, appears to offer the most balanced risk profile for research professionals, whilst the endowment effect, social proof and altruism offer moderate to worse risk profiles. Finally, we highlight the need for more proof in this field - suggesting that future efforts replicate the study with a general population sample or non-financial services case studies.

Is Gen Z So Different? An Analysis of the Impact of Comparative Advertising

“…For two-dimensional descriptive statistics, the Chi-square test was used. The Chisquare test is one of the most utilised statistical analyses 56 used to determine the relation of concordance or the significance of the connection between two variables in a certain dependency relation in marketing research 57 when processing qualitative data 58 and quantitative data 59 and it is relevant for analysis of the measures of consumer behaviour. 60 A significance level was determined at 0.05 and corresponded to a 95% confidence interval for all hypotheses stated.…”

Diary studies in research: More than a research method

“…Diary studies therefore, fare better in terms of participant engagement because of the characteristically shorter time intervals, thereby reducing the longitudinal drawback of disengaged participants (Van Burg & Karlsson, 2020). While incentives have typically been used to mitigate this issue of disengagement (Martin et al, 2023;Silber et al, 2023), the ability of a research design that uses diaries for data collection to also mitigate this is laudable.…”