From Social Ties to Network Processes: Do Tie Definitions Matter?

Abstract

The widespread adoption of digital e-learning environments and other learning technology has provided researchers with ready access to large quantities of data. Much of this data comes from discussion forums and has been studied with analytical methods drawn from social network analysis. However, within this large body of research there exists considerable variation in the definition of what constitutes a social tie, and the consequences of this choice are rarely described or examined. This paper presents findings from two distinct learning environments regarding different social tie extraction methods and their influence on the structural and statistical properties of the induced networks, and the association between measures of centrality and academic performance. Our findings indicate that social tie definitions play an important role in shaping the results of our analyses. The primary purpose of this paper is to raise awareness of the consequences that such methodological choices may have, and to promote transparency in future research. Notes for Practice• Social network analysis has been one of the most commonly applied methods within learning analytics. However, many of the common constructs and tools these methodologies employ have not been subjected to robust validation. Such concerns pertain to construct validity: namely, does a metric actually measure what it purports to measure?• In this study, we find that different social tie extraction methods influence the structural and statistical properties of the induced networks, as well as the associations between centrality measures and academic performance.• Our results emphasize not only the importance of transparency in the choice of tie definition, but also the importance of providing a justification for that choice. Given the impact that tie definitions may have, we advise that practitioners investigate a number of options to ascertain the extent to which such methodological choices can bias their results.

What Does Methodology Mean for Learning Analytics?

“…For each of these links in the chain of argument, alternative explanations exist that potentially undermine the claim. For example, data processing may be criticized as being overly sensitive to operationalization choices (see, for example, Fincham, Gašević, & Pardo [2018] and Gardner & Brooks [2018], both in this issue). Paper authors who aim to build a strong argument are likely to engage with these alternative explanations.…”

“…Conference and journal papers in this field rarely develop brand new algorithms, instruments, or statistical estimators, but they use them, often in novel contexts. For example, applications of classifier algorithms to student affect (e.g., Bosch & Paquette [2018], in this issue) or network analysis to discussion forum activity (Fincham et al [2018], in this issue) raise issues that were not always anticipated and accounted for in the foundational work on classifier and network methods. Other applied fields have gone through their own hand-wringing over methodology.…”

Campus Connections: Student and Course Networks in Higher Education

“…Social networks as they exist in learning communities have been considered at the classroom level (Dawson, 2008;Grunspan, Wiggins, & Goodreau, 2014) and more extensively in massive open online courses (MOOCs) and E-learning (e.g., Cela, Sicilia, & Sánchez, 2014;Fincham, Gašević, & Pardo, 2018;Veletsianos, Collier, & Schneider, 2015). Additional examples can be found in education research.…”

“…As others have pointed out, networks may not measure precisely what was intended. The process of validation for a measure or instrument is an integral part of social science; for example, Fincham et al (2018) and Messick (1994) recently treated the matter in detail in the context of social ties formed in MOOC forums. In the framework of Carolan (2013), coenrollment in a course certainly creates a physical and associative connection, but what behavioral connections are formed among students is not known.…”

Looking for the “More Knowledgeable Other” through Learning Analytics

“…However, it is important to combine quantitative methods with the necessary theoretical basis in order to produce meaningful and actionable results. For example, Fincham, Gaševic, & Pardo (2018) highlighted the influence of the different definitions used to describe the concept "social tie" on the results of the analysis in order to raise awareness of the consequences that such methodological choices may have and to promote transparency in future research. Educational theories can provide a solid basis for research planning and effective data processing.…”