Measuring Simple Preferences: An Approach to Blind, Forced Choice Product Testing

Abstract

When comparing alternate product formulations, or evaluating competitors' formulations, researchers often conduct blind, forced choice product tests. In this study, the authors present an approach to designing these tests and evaluating their results. Two types of tests are considered: repeat paired comparisons formats, and formats consisting of several triangle tests and a single paired comparison. The approach consists of psychophysical assumptions, discrimination and preference constructs, and a set of analytical techniques. Using this approach, a researcher can compare product test formats to see which one most efficiently estimates a given construct of interest. The approach can also be used to gauge sample size requirements for a particular product test format and to estimate confidence intervals for an observed product test result. To illustrate the approach the authors compare two popular product test formats—the Four Paired Comparisons, and the Double Triangle Single Preference.preferences, discrimination, fisher information

Googling Bing: Reassessing the impact of brand on the perceived quality of two contemporary search engines

“…The technique of forced choice was first discussed by Ferris [8] and Irwin [9]. A comprehensive examination of forced choice in preference elicitation is presented by Buchanan and Morrison [6]. Essentially, when a user is forced to make a choice between two results, s/ he scrutinizes the options in greater detail.…”

“…The second significant shortcoming of prior work is that it fails to take into account pre-existing brand preferences for typical users [6]. Bailey et al published their work in 2007, based on research done prior to publication.…”

“…Second, when designing brand effect studies, it is necessary to use a sample that prefers the stronger brand being tested. Incorporating a weaker brand in testing alters the magnitude of brand effect (impact of the branding, its induced loyalty and its perception), and is considered poor experimental practice [6]. Choosing a weaker brand also tells us little about the overall strength of the most market dominant brand.…”

A stochastic model of list falloff with implications for repeat mailings

“…Levels of heterogeneity in between these two extremes produce values of C#I greater than 0 and less than 1. This index has been applied to studies of media viewing by Sabavala and Morrison (ll), to brand switching studies by Kalwani (5), and to sensory testing studies by Buchanan and Morrison (1). It has been shown to provide an intuitive measure of population heterogeneity and to have reasonable sampling properties.…”

“…Each person's response rate remains the same across mailings, but those with lower response rates tend to be left behind. Equation 1 defines the falloff between the ith and jth mailing. Substituting results from equation 4 we have:…”