Developing an evaluation framework of quality indicators for learning analytics

Scheffel, Maren; Drachsler, Hendrik; Specht, Marcus

doi:10.1145/2723576.2723629

Cited by 35 publications

(34 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The main conference papers of LAK '15 still looked at privacy as a search term (Sekiya, Matsuda, & Yamaguchi, 2015), a course subject (Vogelsang & Ruppertz, 2015), or an abstract concern (Scheffel, Drachsler, & Specht, 2015), which could limit access to data (Wang, Heffernan, & Heffernan, 2015;Drachsler et al, 2015b), or one that must "be addressed given the larger scale of the tools usage compared with pilot studies" when "testing the tool in-the-wild" (Martinez-Maldonado et al, 2015, p. 6). …”

Section: Related Workmentioning

confidence: 99%

Privacy-driven Design of Learning Analytics Applications – Exploring the Design Space of Solutions for Data Sharing and Interoperability

Hoel¹,

Chen

2016

Learning Analytics

View full text Add to dashboard Cite

ABSTRACT:Studies have shown that issues of privacy, control of data, and trust are essential to implementation of learning analytics systems. If these issues are not addressed appropriately, systems will tend to collapse due to a legitimacy crisis, or they will not be implemented in the first place due to resistance from learners, their parents, or their teachers. This paper asks what it means to give priority to privacy in terms of data exchange and application design and offers a conceptual tool, a Learning Analytics Design Space model, to ease the requirement solicitation and design for new learning analytics solutions. The paper argues the case for privacy-driven design as an essential part of learning analytics systems development. A simple model defining a solution as the intersection of an approach, a barrier, and a concern is extended with a process focusing on design justifications to allow for an incremental development of solutions. This research is exploratory in nature, and further validation is needed to prove the usefulness of the Learning Analytics Design Space model.

show abstract

Section: Related Workmentioning

confidence: 99%

Privacy-driven Design of Learning Analytics Applications – Exploring the Design Space of Solutions for Data Sharing and Interoperability

Hoel¹,

Chen

2016

Learning Analytics

View full text Add to dashboard Cite

show abstract

“…For example, early work by Slade and Prinsloo (2013) provided a sociocritical perspective on the field using an ethical framework of six principles which point to the complexity of defining concepts such as student success. Since then, a substantial amount of work has been completed by various groups in the field, proposing, for example, more principles (Pardo & Siemens, 2014), checklists (Drachsler & Greller, 2016), evaluation frameworks (Scheffel, 2017) and studies of student preferences (Roberts, Howell, & Seaman, 2017) and explorations of what obligations we have to intervene given a set of predictions (Prinsloo & Slade, 2017).…”

mentioning

confidence: 99%

Practical ethics for building learning analytics

Kitto¹,

Knight²

2019

Brit J Educational Tech

View full text Add to dashboard Cite

Artificial intelligence and data analysis (AIDA) are increasingly entering the field of education. Within this context, the subfield of learning analytics (LA) has, since its inception, had a strong emphasis upon ethics, with numerous checklists and frameworks proposed to ensure that student privacy is respected and potential harms avoided. Here, we draw attention to some of the assumptions that underlie previous work in ethics for LA, which we frame as three tensions. These assumptions have the potential of leading to both the overcautious underuse of AIDA as administrators seek to avoid risk, or the unbridled misuse of AIDA as practitioners fail to adhere to frameworks that provide them with little guidance upon the problems that they face in building LA for institutional adoption. We use three edge cases to draw attention to these tensions, highlighting places where existing ethical frameworks fail to inform those building LA solutions. We propose a pilot open database that lists edge cases faced by LA system builders as a method for guiding ethicists working in the field towards places where support is needed to inform their practice. This would provide a middle space where technical builders of systems could more deeply interface with those concerned with policy, law and ethics and so work towards building LA that encourages human flourishing across a lifetime of learning. What is already known about this topic Applied ethics has a number of well‐established theoretical groundings that we can use to frame the actions of ethical agents, including, deontology, consequentialism and virtue ethics. Learning analytics has developed a number of checklists, frameworks and evaluation methodologies for supporting trusted and ethical development, but these are often not adhered to by practitioners. Laws like the General Data Protection Regulation (GDPR) apply to fields like education, but the complexity of this field can make them difficult to apply. What this paper adds Evidence of tensions and gaps in existing ethical frameworks and checklists to support the ethical development and implementation of learning analytics. A set of three edge cases that demonstrate places where existing work on the ethics of AI in education has failed to provide guidance. A “practical ethics” conceptualisation that draws on virtue ethics to support practitioners in building learning analytics systems. Implications for practice and/or policy Those using AIDA in education should collect and share example edge cases to support development of practical ethics in the field. A multiplicity of ethical approaches are likely to be useful in understanding how to develop and implement learning analytics ethically in practical contexts.

show abstract

“…Hence, it is essential to analyze both the students' opinions and learning experiences, and the data generated along the use of the laboratory. To achieve this objective, learning analytic techniques [34] will be employed. This way, the learning process can be studied deeper and, also, optimized according to the context where the learning process is applied.…”

mentioning

confidence: 99%

Students’ Acceptance and Tracking of a New Container-Based Virtual Laboratory

et al. 2020

View full text Add to dashboard Cite

Presently, the ever-increasing use of new technologies helps people to acquire additional skills for developing an applied critical thinking in many contexts of our society. When it comes to education, and more particularly in any Engineering subject, practical learning scenarios are key to achieve a set of competencies and applied skills. In our particular case, the cybersecurity topic with a distance education methodology is considered and a new remote virtual laboratory based on containers will be presented and evaluated in this work. The laboratory is based on the Linux Docker virtualization technology, which allows us to create consistent realistic scenarios with lower configuration requirements for the students. The laboratory is comparatively evaluated with our previous environment, LoT@UNED, from both the points of view of the students’ acceptance with a set of UTAUT models, and their behavior regarding evaluation items, time distribution, and content resources. All data was obtained from students’ surveys and platform registers. The main conclusion of this work is that the proposed laboratory obtains a very high acceptance from the students, in terms of several different indicators (perceived usefulness, estimated effort, social influence, attitude, ease of access, and intention of use). Neither the use of the virtual platform nor the distance methodology employed affect the intention to use the technology proposed in this work.

show abstract

Developing an evaluation framework of quality indicators for learning analytics

Cited by 35 publications

References 7 publications

Privacy-driven Design of Learning Analytics Applications – Exploring the Design Space of Solutions for Data Sharing and Interoperability

Privacy-driven Design of Learning Analytics Applications – Exploring the Design Space of Solutions for Data Sharing and Interoperability

Practical ethics for building learning analytics

Students’ Acceptance and Tracking of a New Container-Based Virtual Laboratory

Contact Info

Product

Resources

About