Hypothesis-Testing Strategies: Why two Goals are Better than One

Abstract

An intriguing finding in the hypothesis-testing literature concerns a large increase in the proportion of subjects who discover a rule when they are asked to determine two rules rather than that rule alone. This finding is based on Wason's (1960) “2 4 6” task, in which subjects try to discover a rule (ascending numbers) by generating and testing number triples. They are initially given an example (“2, 4, 6”) of the rule that leads to overly specific hypotheses. With single-goal (SG) instructions, subjects try to discover the correct rule and are told whether each triple proposed fits the rule. With dual-goal (DG) instructions, correct and incorrect categories are labelled, respectively, as DAX and MED. Subjects try to discover both rules and are told whether each proposed triple is DAX or MED. Two explanations of why DG subjects do better at rule discovery than SG subjects are tested: the quantity of information and the testing of complementary rules using the prevalent positive-test strategy. Results support the latter explanation: DG subjects outperform SG subjects only if they know the rules are complementary, and that SG subjects’ performance does not improve when required to test more triples before announcing their first rule. A third explanation, the positivity of the linguistic label of the feedback, is ruled out. Understanding the superiority of DG instructions might suggest a general method for enhancing rule discovery.

Contrast class cues and performance facilitation in a hypothesis-testing task: evidence for an iterative counterfactual model

“…We note that the success rates presented in Table 1 for the useful versus the nonuseful contrast class cue conditions of Experiment 1 are, in fact, very similar to those typically reported in the literature for the DG version of the task versus the SG version of the task, respectively (e.g., Gale & Ball, 2006;Wharton et al, 1993). This observation supports the notion that the DG instructions per se may not lead to task success in the DG paradigm, but rather the fact that DG instructions facilitate participants' production of a salient contrast class-which, in turn, promotes successful discovery of the target DAX rule.…”

“…The high number of participants producing overly restricted initial DAX hypotheses (which included 9 of the10 solvers) runs counter to the idea that the 6-4-2 MED triple simply removed the inductive challenge and directly primed the correct DAX solution. This view is also challenged by the observation that solvers only reached an awareness of the correct DAX rule after a mean of 4.90 self-generated triples (SD = 1.05), which is similar to the value that has been reported with the standard DG paradigm (e.g., Wharton et al 1993). We therefore see no evidence to support a simple explanation of the facilitatory effects of the 6-4-2 MED triple as arising from the direct seeding of the correct DAX rule from the outset.…”

“…Our analyses of participants' triple-production profiles additionally allowed us to explore other theories of DG facilitation, such as the triple-heterogeneity theory (e.g., Vallée-Tourangeau et al, 1995), which argues that DG instructions engender a more creative exploration of the triple space, and the goal complementarity theory (Wharton et al 1993), which proposes that the DG paradigm elevates the production of variable positive triples that provide the reasoner with insight into the need for a more generalized DAX rule. Contrary to the predictions of triple-heterogeneity theory, however, we found that solvers differed from nonsolvers in only one aspect of their triple production: the number of descending triples produced.…”

“…Participants are given 2-4-6 as an example of the DAX rule and are then asked to generate triples that are classified as either conforming to the unknown DAX rule (which is "any ascending sequence") or to the unknown MED rule (which is "all other sequences"). Using these amended instructions, success rates on the task for initial rule announcements typically rise to over 60% (e.g., Gale & Ball, 2006;Gorman et al, 1987;Tukey, 1986;Vallée-Tourangeau, Austin & Rankin, 1995;Wharton, Cheng & Wickens, 1993).…”

On people’s understanding of the diagnostic implications of probabilistic data

“…This is related to the proposition (see, e.g., Evans, 1989, p. 60;Klayman & Ha, 1987) that subjects tend to focus on positive information. One clear finding in the research using Wason's 2-4-6 task is that subjects do not readily generate triples to test alternative hypotheses (Wason, 1960), unless the experimental situation is designed to get them to do so (Gorman, Stafford, & Gorman, 1987;Tweney et aI., 1980;Wharton, Cheng, & Wickens, 1993). It is our experience with the 2-4-6 task that subjects simply never spontaneously state two hypotheses when performing a test with a single triple (Tweney et aI., 1980), which is consistent with the proposition that subjects typically consider data as relevant to only one hypothesis at a time.…”