Statistical Analysis of the Indus Script Using n-Grams

Abstract

The Indus script is one of the major undeciphered scripts of the ancient world. The small size of the corpus, the absence of bilingual texts, and the lack of definite knowledge of the underlying language has frustrated efforts at decipherment since the discovery of the remains of the Indus civilization. Building on previous statistical approaches, we apply the tools of statistical language processing, specifically n-gram Markov chains, to analyze the syntax of the Indus script. We find that unigrams follow a Zipf-Mandelbrot distribution. Text beginner and ender distributions are unequal, providing internal evidence for syntax. We see clear evidence of strong bigram correlations and extract significant pairs and triplets using a log-likelihood measure of association. Highly frequent pairs and triplets are not always highly significant. The model performance is evaluated using information-theoretic measures and cross-validation. The model can restore doubtfully read texts with an accuracy of about 75%. We find that a quadrigram Markov chain saturates information theoretic measures against a held-out corpus. Our work forms the basis for the development of a stochastic grammar which may be used to explore the syntax of the Indus script in greater detail.

Clustering Indus Texts using K-means

“…The total number of signs constituting 80% of text beginners, text enders and all signs, in EBUDS and clusters C1 to C9, vary significantly. As in the case of EBUDS, there exists an asymmetry in the usage pattern of text beginners and text enders in all the clusters with far fewer signs ending the texts than beginning them [14]. However, the absolute number of signs that can begin or end texts varies for each cluster (Table 4).…”

“…These and other studies [12][13][14][15][16][17][18][19][20], therefore provide a unique set of constraints on any interpretation of this undeciphered script. It suggests a high level of standardization, nine clusters of texts each with its own preferred set of signs and sign sets, a uniformity of grammar and an agreed function of each sign across the culture.…”

“…There exists an asymmetry in the usage of text beginners and text enders in EBUDS [14]. The number of signs required to account for 80% of the text beginners, text enders and all sign occurrences in EBUDS and the nine clusters is given in Table 4.…”

“…Earlier studies have explored the sequential structure of the Indus script using various statistical and computational techniques [12][13][14][15][16][17][18][19][20], the design of Indus signs [21] and different types of patterns inscribed on these objects [22][23][24]. These studies do not make any assumptions about its nature, content or meaning.…”

Entropy, the Indus Script, and Language:A Reply to R. Sproat

“…. of the Indus script (Yadav et al 2008a(Yadav et al , 2008b(Yadav et al , 2010Rao et al 2009aRao et al , 2009bRao 2010b). In each case, we compare these properties with those of linguistic systems to ascertain whether the property tilts the evidence towards or away from the linguistic hypothesis H L .…”