A critical review of the validity and rationale of the forced-choice technique.

Abstract

According to a paper by Staff, Personnel Research and Procedures Branch, the Adjutant General's Office (5), the basic idea for the forcedchoice technique was developed by Paul Horst with reference specifically to personality scales. Robert Wherry developed a similar idea while working on personality measurement for the Civil Aeronautics Author-it}', and later the Staff of the Personnel Research Section of the Adjutant General's Office attempted to apply the concept first to the design of personality inventories and later to the problem of rating officers. The latter application resulted in the production of the Army Efficiency Report, which was used for some years for the official rating of officers. More recently the technique has been used by various professional persons and for a number of different purposes. The method of making a forced-choice scale as used by the Personnel Research Section, has been well described by Sisson (3).The general nature of a forced-choice rating scale is best understood in terms of the procedure which is followed in building it. The following are the essential steps involved:1. Descriptions are collected of individuals who are at each extreme of the scale to be measured. If the scale is that of effectiveness as a supervisor, descriptions are obtained of the most effective and the least effective supervisors. This procedure partly defines the scale that is to be measured by the final instrument.2. The descriptions collected are then dissected into a list of small elements. Each one of these elements describes, in essence, a rather specific item of behavior. The complete list should cover all the important aspects of the job, and the number of items covering each aspect should be related in some rational way to the importance of that aspect.3. Two values are determined for each item listed. One value, the discrimination value, indicates the degree to which the item measures the particular characteristic that is being measured. The other value indicates the extent to which individuals tend to rate others too high or too low on the particular characteristic. This latter is sometimes referred to as the preference index or preference value of the item. Both values are determined experimentally.4. The characteristics are then arranged in pairs such that the two members of each pair differ in the extent to which they discriminate. Ideally, one of the characteristics in each pair should have a discrimination value of zero and one 62

Are attractive men's faces masculine or feminine? The importance of type of facial stimuli.

“…For example, participants may find the faces from which to choose to be equally attractive or equally unattractive, but the forced-choice method does not permit this response. The literature is replete with evidence of problems with the reliability of forced-choice methods used in assessments of personality, achievement motivation, personnel decisions, knowledge, and vocational choices (Grosse & Wright, 1985;Jackson, Neill, & Bevan, 1973;Nederhof, 1985;Ray, 1980Ray, , 1982Travers, 1951). The problems of reliability result from forcing individuals to make decisions that were not necessarily accurate characterizations of the person being rated (i.e., overestimating or underestimating the quality of the individual being rated; Travers, 1951).…”

Factors Affecting Forced‐choice Response

“…A point may be made concerning the negative weight attributed to DI in the multiple prediction. Travers (1951) has criticized the forced-choice technique in that it does not actually control for bias-since all that one needs to do to fake a rating is to rate someone who would score high rather than the person intended for rating. Although this may be a way of faking a response on a forced-choice scale, the negative prediction weight given the DI for this study indicates that this procedure was not used in this situation for faking the scale.…”

Studies of Forced-Choice Methodology: Individual Differences in Social Desirability

“…Travers (1951) has suggested that forced-choice tests require the subjects to choose among alternatives none of which may be relevant for him. In defense of the forced-choice technique it is generally assumed that, if all alternatives are inappropriate, subjects will choose randomly among them as they are of equal social desirability ; this has the effect of increasing error variance but reducing &dquo;bias.&dquo; Results of the present study, however, suggest that when all alternatives are inappropriate (or equally appropriate) subjects may select the alternative which has the greatest personal social desirability.…”