In their research articles, scholars often use 2 × 2 tables or tree diagrams including natural frequencies in order to illustrate Bayesian reasoning situations to their peers. Interestingly, the effect of these visualizations on participants’ performance has not been tested empirically so far (apart from explicit training studies). In the present article, we report on an empirical study (3 × 2 × 2 design) in which we systematically vary visualization (no visualization vs. 2 × 2 table vs. tree diagram) and information format (probabilities vs. natural frequencies) for two contexts (medical vs. economical context; not a factor of interest). Each of N = 259 participants (students of age 16–18) had to solve two typical Bayesian reasoning tasks (“mammography problem” and “economics problem”). The hypothesis is that 2 × 2 tables and tree diagrams – especially when natural frequencies are included – can foster insight into the notoriously difficult structure of Bayesian reasoning situations. In contrast to many other visualizations (e.g., icon arrays, Euler diagrams), 2 × 2 tables and tree diagrams have the advantage that they can be constructed easily. The implications of our findings for teaching Bayesian reasoning will be discussed.
competencies, specifically content knowledge, pedagogical content knowledge, diagnostic competencies, selfregulation, and motivational orientations (Kunter et al. 2013a;. In order empirically to investigate research questions regarding these competencies (e.g., concerning their structure, school-type differences, or their impact on student learning), one needs valid and reliable instruments with which to measure them. In the COACTIV research program (Cognitive Activation in the Classroom: Professional Competence of Teachers, Cognitively Activating Instruction, and Development of Students Mathematical Literacy), a variety of such instruments were developed and implemented with a large sample of German secondary mathematics teachers (who taught the grade 9/10 students assessed by the PISA study 2003/2004; see Prenzel et al. 2004 for details of the PISA supplement 2003/04 in Germany). In the present article we report on findings regarding a COACTIV instrument, in which video clips were used in order to prompt the teachers to react appropriately to critical teaching situations (for detailed information on the COACTIV research program, see Section 4.1). There are quite a few different approaches for measuring teacher competencies (for an overview, see Lindmeier 2013; Kunter et al. 2013a; van der Vleuten 1996). Depaepe et al. (2013), for instance, provide an overview of scholars who measure pedagogical content knowledge (PCK) in mathematics. They have analyzed about 60 empirical studies that obtained data by paper-and-pencil tests, classroom observations, interviews, meeting observations, document analyses, questionnaires, and concept maps, and conclude that different perspectives can all have their specific advantages and disadvantages.There is still a lively debate on how to conceptualize PCK appropriately (e.g., Buchholtz et al. 2014
Changing the information format from probabilities into frequencies as well as employing appropriate visualizations such as tree diagrams or 2 × 2 tables are important tools that can facilitate people’s statistical reasoning. Previous studies have shown that despite their widespread use in statistical textbooks, both of those visualization types are only of restricted help when they are provided with probabilities, but that they can foster insight when presented with frequencies instead. In the present study, we attempt to replicate this effect and also examine, by the method of eye tracking, why probabilistic 2 × 2 tables and tree diagrams do not facilitate reasoning with regard to Bayesian inferences (i.e., determining what errors occur and whether they can be explained by scan paths), and why the same visualizations are of great help to an individual when they are combined with frequencies. All ten inferences of N = 24 participants were based solely on tree diagrams or 2 × 2 tables that presented either the famous “mammography context” or an “economics context” (without additional textual wording). We first asked participants for marginal, conjoint, and (non-inverted) conditional probabilities (or frequencies), followed by related Bayesian tasks. While solution rates were higher for natural frequency questions as compared to probability versions, eye-tracking analyses indeed yielded noticeable differences regarding eye movements between correct and incorrect solutions. For instance, heat maps (aggregated scan paths) of distinct results differed remarkably, thereby making correct and faulty strategies visible in the line of theoretical classifications. Moreover, the inherent structure of 2 × 2 tables seems to help participants avoid certain Bayesian mistakes (e.g., “Fisherian” error) while tree diagrams seem to help steer them away from others (e.g., “joint occurrence”). We will discuss resulting educational consequences at the end of the paper.
Two different tools for assessing pedagogical content knowledge (PCK) of mathematics teachers used in the framework of the COACTIV study are systematically compared in this paper, namely the paper-and-pencil test consisting of items on the three facets knowledge of explaining and representation, knowledge of student thinking and typical mistakes, and knowledge of the potential of mathematical tasks, and the video vignettes instrument that examines teachers' proposed continuations for presented lesson video clips specific to their subject-related and methodological competence aspects. Initially, both COACTIV PCK assessment tools are systematically contrasted for the first time with respect to their predictive validity for instructional quality (N = 163 German secondary mathematics teachers) as well as student learning gains (N = 3806 PISA students from 169 different classes) by means of path models showing that PCK, when assessed by the paper-and-pencil method, can better predict instructional quality than the video vignettes instrument can. Next, we theoretically propose the cascade model as capable of integrating pertinent theories on teacher competence and instructional quality. This model implies five 'columns' that are ordered according to a sequential causal chain (teacher disposition → situation-specific skills → observable teaching behavior → student mediation → learning gains). Finally, we specify four out of the five 'columns' of this cascade model, based empirically on the COACTIV data.
In medicine, diagnoses based on medical test results are probabilistic by nature. Unfortunately, cognitive illusions regarding the statistical meaning of test results are well documented among patients, medical students, and even physicians. There are two effective strategies that can foster insight into what is known as Bayesian reasoning situations: (1) translating the statistical information on the prevalence of a disease and the sensitivity and the false-alarm rate of a specific test for that disease from probabilities into natural frequencies, and (2) illustrating the statistical information with tree diagrams, for instance, or with other pictorial representation. So far, such strategies have only been empirically tested in combination for “1-test cases”, where one binary hypothesis (“disease” vs. “no disease”) has to be diagnosed based on one binary test result (“positive” vs. “negative”). However, in reality, often more than one medical test is conducted to derive a diagnosis. In two studies, we examined a total of 388 medical students from the University of Regensburg (Germany) with medical “2-test scenarios”. Each student had to work on two problems: diagnosing breast cancer with mammography and sonography test results, and diagnosing HIV infection with the ELISA and Western Blot tests. In Study 1 (N = 190 participants), we systematically varied the presentation of statistical information (“only textual information” vs. “only tree diagram” vs. “text and tree diagram in combination”), whereas in Study 2 (N = 198 participants), we varied the kinds of tree diagrams (“complete tree” vs. “highlighted tree” vs. “pruned tree”). All versions were implemented in probability format (including probability trees) and in natural frequency format (including frequency trees). We found that natural frequency trees, especially when the question-related branches were highlighted, improved performance, but that none of the corresponding probabilistic visualizations did.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.