EstBERT: A Pretrained Language-Specific BERT for Estonian

Tanvir, Hasan; Kittask, Claudia; Eiche, Sandra; Sirts, Kairit

doi:10.48550/arxiv.2011.04784

Cited by 1 publication

(1 citation statement)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, we also considered bidirectional encoder representations from transformers (BERT), which have recently performed well in textual classification tasks (Devlin et al, 2019). Namely, we used the version pre-trained in Estonian (EstBERT) (Tanvir et al, 2021). During this phase we opted for a model with one layer and an AdamW optimizer with an initial learning rate of 2e-5, as suggested in Devlin and colleagues (2019).…”

Section: Step S2: Comparing Different Sml Models and Feature Extracti...mentioning

confidence: 99%

Design Analytics for Mobile Learning

et al. 2022

View full text Add to dashboard Cite

This research was triggered by the identified need in literature for large-scale studies about the kinds of designs that teachers create for mobile learning (m-learning). These studies require analyses of large datasets of learning designs. The common approach followed by researchers when analyzing designs has been to manually classify them following high-level pedagogically guided coding strategies, which demands extensive work. Therefore, the first goal of this paper is to explore the use of supervised machine learning (SML) to automatically classify the textual content of m-learning designs using pedagogically relevant classifications, such as the cognitive level demanded by students to carry out specific designed tasks, the phases of inquiry learning represented in the designs, or the role that the situated environment has in the designs. Because not all SML models are transparent, but researchers often need to understand their behaviour, the second goal of this paper is to consider the trade-off between models’ performance and interpretability in the context of design analytics for m-learning. To achieve these goals, we compiled a dataset of designs deployed using two tools, Avastusrada and Smartzoos. With this dataset, we trained and compared different models and feature extraction techniques. We further optimized and compared the best performing and most interpretable algorithms (EstBERT and Logistic Regression) to consider the second goal with an illustrative case. We found that SML can reliably classify designs with accuracy > 0.86 and Cohen’s kappa > 0.69.

show abstract

Section: Step S2: Comparing Different Sml Models and Feature Extracti...mentioning

confidence: 99%