The unreasonable effectiveness of machine learning in Moldavian versus Romanian dialect identification

Abstract

Motivated by the seemingly high accuracy levels of machine learning (ML) models in Moldavian versus Romanian dialect identification and the increasing research interest on this topic, we provide a follow-up on the Moldavian versus Romanian Cross-Dialect Topic Identification (MRC) shared task of the VarDial 2019 evaluation campaign. The shared task included two subtask types: one that consisted in discriminating between the Moldavian and Romanian dialects and one that consisted in classifying documents by topic across the two dialects of Romanian. Participants achieved impressive scores, for example, the top model for Moldavian versus Romanian dialect identification obtained a macro-F 1 score of 0.895. We conduct a subjective evaluation by human annotators, showing that humans attain much lower accuracy rates compared with ML models. Hence, it remains unclear why the methods proposed by participants attain such high accuracy rates. Our goal is to understand (i) why the proposed methods work so well (by visualizing the discriminative features) and (ii) to what extent these methods can keep their high accuracy levels, for example, when we shorten the text samples to single sentences or when we use tweets at inference time. A secondary goal of our

Romania: university and politics in the 1980s. Case study: the University of Bucharest, the 1980s

“…This has led to a growing need for resources on low-resource languages. Considering dialect identification datasets across different languages, we can distinguish between two types of resources: text-based datasets [20,21,22,23,24] and speechbased datasets [10,11,12,13,14,15]. While various languages have benefited from text-based resources that leverage written materials capturing linguistic variations, the auditory dimension of dialects adds an intricate layer of complexity.…”

“…To the best of our knowledge, RoDia is the first dataset to tackle spoken dialect identification in the Romanian landscape in accordance with historical, geographical, and sociocultural factors, encouraging the research in this lowresource language. Although there are two text datasets addressing Romanian dialect identification, MOROCO [21] and MOROCO-Tweets [24], these cover only two dialects: Romanian (equivalent to the Muntenesc dialect) and Moldavian (Moldovenesc). In contrast, our dataset is focused on speech and covers all five Romanian dialects, as shown in Figure 1.…”

“…We underline that the extra dialects, namely Ardelenesc, Bȃnȃt ¸ean, and Oltenesc, are mostly characterized by phonetic differences captured only in speech. This explains why MOROCO [21] and MOROCO-Tweets [24] only contain text samples from the other two dialects.…”