Representing Tone in Levenshtein Distance

Abstract

Levenshtein distance, also known as string edit distance, has been shown to correlate strongly with both perceived distance and intelligibility in various Indo-European languages (Gooskens and Heeringa, 2004;Gooskens, 2006). We apply Levenshtein distance to dialect data from Bai (Allen, 2004), a Sino-Tibetan language, and Hongshuihe (HSH) Zhuang (Castro and Hansen, accepted), a Tai language. In applying Levenshtein distance to languages with contour tone systems, we ask the following questions: 1) How much variation in intelligibility can tone alone explain? and 2) Which representation of tone results in the Levenshtein distance that shows the strongest correlation with intelligibility test results? This research evaluates six representations of tone: onset, contour and offset; onset and contour only; contour and offset only; target approximation Wang, 2001), autosegments of H andChao's (1930) pitch numbers. For both languages, the more fully explicit onset-contouroffset and onset-contour representations showed significantly stronger inverse correlations with intelligibility. This suggests that, for cross-dialectal listeners, the optimal representation of tone in Levenshtein distance should be at a phonetically explicit level and include information on both onset and contour.

Incorporating tone in the modelling of wordlikeness judgements

“…One straightforward way is to simply add them together. However, this does not allow for different weightings for segments and tones, for which there is experimental evidence from perception studies (Cham 2003), word-recognition studies (Cutler & Chen 1997, Keung & Hoosain 1979), word-reconstruction studies (Wiener & Turnbull 2016) and phonological distance studies (Yang & Castro 2008, Do & Lai forthcoming). To model empirically informed weights of segments and tones, we chose the weights that can predict native speakers’ phonological distance judgements in Do & Lai's study, using the fixed intercept and coefficients for segmental and tonal distance.…”

One size fits all? Dialectometry in a small clan-based indigenous society

“…In addition, dialectometry studies in Chinese area linguistics are considered as well (Cheng, 1997;Tang & van Heuven, 2009;Yang & Castro, 2008). All of these studies explore how aggregate measurements of dialect features can be correlated with geographic distance.…”

“…The methods of dialectometry applied in this study are based on Gooskens (2005), Gooskens and Heeringa (2004), Heeringa and Nerbonne (2001), Kretzschmar et al (2010), Nerbonne (2009, 2010), Nerbonne and Heeringa (2007, 2010), and Nerbonne and Kleiweg (2007). In addition, dialectometry studies in Chinese area linguistics are considered as well (Cheng, 1997; Tang & van Heuven, 2009; Yang & Castro, 2008). All of these studies explore how aggregate measurements of dialect features can be correlated with geographic distance.…”

“…5.Other work on tone in Levenshtein Distances includes Yang and Castro's (2008) examination of intelligibility in Bai and Zhuang, comparing different calculations for tone. The present Sui study is somewhat different as all of the Sui dialects studied here have a comparatively high degree of mutual intelligibility.…”