HeiMorph at SIGMORPHON 2022 Shared Task on Morphological Acquisition Trajectories

Ramarao, Akhilesh Kakolu; Zinova, Yulia; Tang, Kevin; Vijver, Ruben van de

doi:10.18653/v1/2022.sigmorphon-1.24

Cited by 5 publications

(4 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For instance, the SIGMORPHON 2021 shared Task 0 Part 2 was to predict the judgement ratings of wug words (Calderone et al, 2021) as opposed to using real words held-out from the training data as test data. Similarly, the SIGMORPHON 2022 challenge involved computational modeling of the data drawn from corpora of child-directed speech and evaluation on children's learning trajectories and erroneous productions (Kodner and Khalifa, 2022;Kakolu Ramarao et al, 2022).…”

Section: Related Workmentioning

confidence: 99%

Linear Discriminative Learning: a competitive non-neural baseline for morphological inflection

Jeong

Schmitz²,

Ramarao³

et al. 2023

Preprint

View full text Add to dashboard Cite

This paper presents our submission to the SIGMORPHON 2023 task 2 of Cognitively Plausible Morphophonological Generalization in Korean. We implemented both Linear Discriminative Learning and Transformer models and found that the Linear Discriminative Learning model trained on a combination of corpus and experimental data showed the best performance with the overall accuracy of around 83%. We found that the best model must be trained onboth corpus data and the experimental data of one particular participant. Our examination of speaker-variability and speaker-specific information did not explain why a particular participant combined well with the corpus data.We recommend Linear Discriminative Learning models as a future non-neural baseline system, owning to its training speed, accuracy, model interpretability and cognitive plausibility. In order to improve the model performance, we suggest using bigger data and/or performing data augmentation and incorporating speaker and item-specifics considerably.

show abstract

Section: Related Workmentioning

confidence: 99%

Linear Discriminative Learning: a competitive non-neural baseline for morphological inflection

Jeong

Schmitz²,

Ramarao³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…As anticipated, participants' models achieved lowest scores on the task of Arabic nouns plural inflection. Kakolu Ramarao et al (2022) employed a vanilla transformer architecture that takes as an input the individual characters of the source lemma, morpho-syntactic tag of the input, and morpho-syntactic tag of the output. To account of the problem of data sparsity, they use a data hallucination technique based on alignment and replacement steps similar to that of Anastasopoulos and Neubig (2019b).…”

Section: Related Workmentioning

confidence: 99%

Neural Arabic singular-to-plural conversion using a pretrained Character-BERT and a fused transformer

Radman,

Atros,

Duwairi

2023

Nat. Lang. Eng.

View full text Add to dashboard Cite

Morphological re-inflection generation is one of the most challenging tasks in the natural language processing (NLP) domain, especially with morphologically rich, low-resource languages like Arabic. In this research, we investigate the ability of transformer-based models in the singular-to-plural Arabic noun conversion task. We start with pretraining a Character-BERT model on a masked language modeling task using 1,134,950 Arabic words and then adopting the fusion technique to transfer the knowledge gained by the pretrained model to a full encoder–decoder transformer model, in one of the proposed settings. The second proposed setting directly fuses the output Character-BERT embeddings into the decoder. We then analyze and compare the performance of the two architectures and provide an interpretability section in which we track the features of attention with respect to the model. We perform the interpretation on both the macro and micro levels, providing some individual examples. Moreover, we provide a thorough error analysis showing the strengths and weaknesses of the proposed framework. To the best of our knowledge, this is the first effort in the Arabic NLP domain that adopts the development of an end-to-end fused-transformer deep learning model to address the problem of singular-to-plural conversion.

show abstract

“…They implement true mini-batch training for a substantial speed up, rendering the system more practical on larger training sets. HeiMorph (Ramarao et al, 2022): The team from Heinrich-Heine-Universität Düsseldorf developed a system with a self-attention Transformer architecture with bigram hallucination. Submitted models were trained on the enriched data setsthat include either 1,000 or 10,000 bigram-aware hallucinated word pairs, generated separately for each training set size.…”

Section: Systemsmentioning

confidence: 99%

Proceedings of the 19th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology

2022

View full text Add to dashboard Cite

Speech consists of a continuously-varying acoustic signal. Yet human listeners experience it as sequences of discrete speech sounds, which are used to recognise words. To examine how the human brain appropriately sequences the speech signal, we recorded two-hour magnetoencephalograms from 21 subjects listening to short narratives. Our analyses show that the brain continuously encodes the three most recently heard speech sounds in parallel, and maintains this information long past the sensory input. Each speech sound has a representation that evolves over time, jointly encoding both its phonetic features and time elapsed since onset. This allows the brain to represent the relative order and phonetic content of the phonetic sequence. These dynamic representations are active earlier when phonemes are more predictable, and are sustained longer when lexical identity is uncertain. The flexibility in the dynamics of these representations paves the way for further understanding of how such sequences may be used to interface with higher order structure such as morphemes and words.Bio: Laura Gwilliams received her PhD in Psychology with a focus in Cognitive Neuroscience from New York University in May 2020. Currently she is a post-doctoral researcher at UCSF, using MEG and ECoG data to understand how linguistic structures are parsed and composed while listening to continuous speech. The ultimate goal of Laura's research is to describe speech comprehension in terms of what operations are applied to the acoustic signal; which representational formats are generated and manipulated (e.g. phonetic, syllabic, morphological), and under what processing architecture.

show abstract

HeiMorph at SIGMORPHON 2022 Shared Task on Morphological Acquisition Trajectories

Cited by 5 publications

References 15 publications

Linear Discriminative Learning: a competitive non-neural baseline for morphological inflection

Linear Discriminative Learning: a competitive non-neural baseline for morphological inflection

Neural Arabic singular-to-plural conversion using a pretrained Character-BERT and a fused transformer

Proceedings of the 19th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology

Contact Info

Product

Resources

About