Using Sequence Similarity Networks to Identify Partial Cognates in Multilingual Wordlists

Abstract

Increasing amounts of digital data in historical linguistics necessitate the development of automatic methods for the detection of cognate words across languages. Recently developed methods work well on language families with moderate time depths, but they are not capable of identifying cognate morphemes in words which are only partially related. Partial cognacy, however, is a frequently recurring phenomenon, especially in language families with productive derivational morphology. This paper presents a pilot approach for partial cognate detection in which networks are used to represent similarities between word parts and cognate morphemes are identified with help of state-of-theart algorithms for network partitioning. The approach is tested on a newly created benchmark dataset with data from three sub-branches of Sino-Tibetan and yields very promising results, outperforming all algorithms which are not sensible to partial cognacy.

Using support vector machines and state-of-the-art algorithms for phonetic alignment to identify cognates in multi-lingual wordlists

“…Our collection was taken from six major sources (Greenhill et al, 2008;Dunn, 2012;List, 2014b;List et al, 2016b;Mennecier et al, 2016) covers datasets ranging between 100 and 210 concepts translated into 5 to 100 languages from 13 different language families. Modifications introduced in the process of preparing the datasets included (a) the correction of errata (e.g.…”

“…We also used some auxiliary features from (Jäger and Sofroniev, 2016), which are derived from string similarities. For the clustering subtask (2), we followed List et al (2016b) and List et al (2017) in using the Infomap algorithm (Rosvall and Bergstrom, 2008).…”

“…In order to cluster the words into sets of potentially cognate words, we follow recent approaches by List et al (2016b) and List et al (2017) in using Infomap (Rosvall and Bergstrom, 2008), an algorithm which was originally designed for the detection of communities in large social networks, to detect "communities" of related words. Infomap uses random walks in undirected networks to identify the best way to assign the nodes in the network, that is, in our case, the words, to distinct groups which form a homogeneous class.…”

Identifying Cognate Sets Across Dictionaries of Related Languages

“…List and Moran (2013) propose an approach based on sound class alignments and an average score clustering algorithm. List et al (2016) extend the approach to include partial cognates within word lists.…”

Computational analysis of Gondi dialects

“…The word similarity matrix was converted into a word distance matrix using the following transformation: (1 + exp(x)) −1 where, x is the PMI score between two words. We use the InfoMap clustering algorithm (List et al, 2016) for the purpose of identifying cognate clusters.…”