Seeking advice: A sampling approach to advice taking

Abstract

The present research addresses advice taking from a holistic perspective covering both advice seeking and weighting. We build on previous theorizing that assumes that underweighting of advice results from biased samples of information. That is, decision makers have more knowledge supporting their own judgment than that of another person and thus weight the former stronger than the latter. In the present approach, we assume that participants reduce this informational asymmetry by the sampling of advice and that sampling frequency depends on the information ecology. Advice that is distant from the decision maker’s initial estimate should lead to a higher frequency of advice sampling than close advice. Moreover, we assume that advice distant from the decision maker’s initial estimate and advice that is supported by larger samples of advisory estimates are weighted more strongly in the final judgment. We expand the classical research paradigm with a sampling phase that allows participants to sample any number of advisory estimates before revising their judgments. Three experiments strongly support these hypotheses, thereby advancing our understanding of advice taking as an adaptive process.

The Effects of Explanations in Automated Essay Scoring Systems on Student Trust and Motivation (accepted for publication)

“…The scale was reliable (α = .88). In addition, trust was measured using the Weight of Advice theorem (WoA), which reflects the degree to which the participants adapt their own judgement of their estimated essay grade towards the grade given by the system (Hütter & Ache, 2016). Here, changing the grade towards the estimate from the system is seen as a sign of trust in the system (Papenmeier et al, 2019).…”

The advice less taken: The consequences of receiving unexpected advice

“…Thus, instead of restricting advice to be of intermediate distance, future research should investigate whether the paradigmatic expectation affects other dimensions of advice taking such as shifts in confidence or the sampling of external information. It would be worthwhile investigating whether participants would still actively sample unexpected advice (Hütter & Ache, 2016). According to our reasoning, we expect the effects documented for the WOA to extend to this measure, resulting in smaller sample sizes when participants did not expect to be able to sequentially sample advice.…”

“…The effect of expectation on WOA was descriptively opposite to our hypothesis ( 2 = -1.28, 95% CI [-6.45, 3.89], SE = 2.64, d = -0.04, t(196.16) = -0.49, p = .687; Table S2, left panel). However, advice taking is typically found to vary with the distance of advice from a participant's initial beliefs (e.g., Hütter & Ache, 2016;Schultze et al, 2015). In particular, weighting is most pronounced for advice of "intermediate distance" as categorized by Moussaïd et al (2013) and flattening out for both closer and more distant values (Figure 1).…”

“…Third, the experiment focused on the region of advice distance where the exploratory post-hoc analyses of Experiment 1 revealed significant treatment effects on WOA. The critical region of intermediate distance corresponds to the region for which advice weighting is typically reported to peak if examined dependent on advice distance (Hütter & Ache, 2016;Moussaïd et al, 2013;Schultze et al, 2015). The mechanism attempted to generate an intermediately distant value from a truncated normal distribution as specified by the pretest parameters for a maximum of 1000 times.…”

“…As introduced by Sniezek and Buckley (1995), the judge (or advisee) is first asked to give an initial estimate about the unknown true value of a stimulus item. Thereafter, he or she is to render a final estimate in the light of passively presented or actively sampled pieces of information from one or multiple advisors (e.g., Fiedler et al, 2019;Hütter & Ache, 2016;Soll & Larrick, 2009). That is, there is little uncertainty with regard to the information sampling process: Participants are fully aware that they will get the opportunity to revise their initial estimate in the light of external support later in the experiment.…”