A Bayesian Network Approach To Modeling Learning Progressions

A methodology for applying students' interactive task performance scores from a multimedia-based performance assessment in a Bayesian Network

“…One can say that the assumptions in the psychometric model are used to translate all pieces of data that were characterized as meaningful for a student's overall evaluation into an overall score. The psychometric model that is most frequently discussed with respect to SBA is the Bayesian network (BN) (Levy, 2014;Levy & Mislevy, 2004;Mislevy, Almond, Yan, & Steinberg, 2000;West et al, 2010). BNs (Pearl, 1988) provide a graphical structure in which conditional probability relationships between a (large) number of random variables are represented.…”

Learning Trajectories in Mathematics: A Foundation for Standards, Curriculum, Assessment, and Instruction

“…The alternative, of course, is to design assessments so that they discriminate among, and report in terms of differences in, the levels or specific stages of knowledge and skill attained in particular school subjects; based on tested theories about how those subjects are learned by most students, as we and Knowing what Students Know (Pellegrino, Chudowsky, & Glaser, 2001) argue One of the big questions here is whether one should think of the growth of student learning as being an essentially continuous process, albeit a multi-dimensional one, or whether it is more fruitful to conceive of it as looking like a series of relatively discrete, and at least temporarily stable, steps or cognitive structures that can be described and made the referents of assessment (even if the processes that go on in between as students move from one step to the next might actually have a more continuous, and certainly a probabilistic, character) Chapter 4 in Knowing what Students Know provides a helpful overview of the kinds of psychometric and statistical models that have been developed to reflect these different views of the underlying reality, and many of the issues involved in their use To oversimplify, there are choices between "latent variable" and multivariable models, on the one hand, and latent class models on the other "Latent" simply refers to the fact that the variables or classes represent hypotheses about what is going on and can't be observed directly There are of course mixed cases Rupp, Templin, and Henson (2010) provide a good treatment of the alternative models and relevant issues associated with what they call "Diagnostic Classification Models " Some of the continuous models use psychometric assumptions similar to ones used in current assessments but focus more on discriminating among items than among students, and stress a more rigorous approach to item design to enhance the educational relevance and interpretability of the results, while allowing for increased complexity by assuming that there can be multiple underlying dimensions involved, even if each of them on its own has a linear character (see Wilson, 2005 for examples) The latent class models are in some ways even more exotic Among the more interesting are those that rely on Bayesian inference and Bayesian networks (West et al , 2010) since those seem in principle to be able to model, and help to clarify, indefinitely complex ideas about the number of factors that might be involved in the growth of students' knowledge and skill But for policymakers these models are more complex and even more obscure than more conventional psychometric models, and developing and implementing assessments based on them is likely to be more expensive The relative promise and usefulness of the alternative models needs to be sorted out by use in practical settings, and it seems unlikely that there will be a significant shift toward the use of assessments designed in these ways until there have been some clear practical demonstrations that such assessments provide much better information for guiding practice and policy than current assessments are able to do In mathematics, a few investigators are developing assessments that reflect what we know or can hypothesize about students' learning trajectories For example, our colleagues Jere Confrey and Alan Maloney at NCSU are working on assessments that reflect their conception of a learning trajectory for "equipartitioning" as part of the development of rational number reasoning (Confrey & Maloney in press, 2010; Maloney & Confrey 2010) They began with an extensive synthesis of the existing literature and supplemented it by conducting cross sect...…”

Learning trajectories in mathematics: A foundation for standards, curriculum, assessment, and instruction

“…Some of the continuous models use psychometric assumptions similar to ones used in current assessments but focus more on discriminating among items than among students, and stress a more rigorous approach to item design to enhance the educational relevance and interpretability of the results, while allowing for increased complexity by assuming that there can be multiple underlying dimensions involved, even if each of them on its own has a linear character (see Wilson, 2005 for examples) The latent class models are in some ways even more exotic Among the more interesting are those that rely on Bayesian inference and Bayesian networks (West et al , 2010) since those seem in principle to be able to model, and help to clarify, indefinitely complex ideas about the number of factors that might be involved in the growth of students' knowledge and skill But for policymakers these models are more complex and even more obscure than more conventional psychometric models, and developing and implementing assessments based on them is likely to be more expensive The relative promise and usefulness of the alternative models needs to be sorted out by use in practical settings, and it seems unlikely that there will be a significant shift toward the use of assessments designed in these ways until there have been some clear practical demonstrations that such assessments provide much better information for guiding practice and policy than current assessments are able to do…”