Fast and Accurate Spelling Correction Using Trie and Bigram

Mawardi, Viny Christanti; Rudy, Rudy; Naga, Dali Santun

doi:10.12928/telkomnika.v16i1.6890

Cited by 6 publications

(5 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Stemming converts words to their base form (stem) [58] . The same word appears in different forms in the texts that are consolidated into the same features [59] . For example, the semantic meaning of ‘read’ and ‘reading’ is the same, thus they are combined as ‘read’ to avoid any confusion.…”

Section: Data and Study Methodsmentioning

confidence: 99%

Socioeconomic factors analysis for COVID-19 US reopening sentiment with Twitter and census data

Rahman

Ali

et al. 2021

Heliyon

View full text Add to dashboard Cite

Investigating and classifying sentiments of social media users (e.g., positive, negative) towards an item, situation, and system are very popular among researchers. However, they rarely discuss the underlying socioeconomic factor associations for such sentiments. This study attempts to explore the factors associated with positive and negative sentiments of the people about reopening the economy, in the United States (US) amidst the COVID-19 global crisis. It takes into consideration the situational uncertainties (i.e., changes in work and travel patterns due to lockdown policies), economic downturn and associated trauma, and emotional factors such as depression. To understand the sentiment of the people about the reopening economy, Twitter data was collected, representing the 50 States of the US and Washington D.C, the capital city of the US. State-wide socioeconomic characteristics of the people (e.g., education, income, family size, and employment status), built environment data (e.g., population density), and the number of COVID-19 related cases were collected and integrated with Twitter data to perform the analysis. A binary logit model was used to identify the factors that influence people toward a positive or negative sentiment. The results from the logit model demonstrate that family households, people with low education levels, people in the labor force, low-income people, and people with higher house rent are more interested in reopening the economy. In contrast, households with a high number of family members and high income are less interested in reopening the economy. The accuracy of the model is reasonable (i.e., the model can correctly classify 56.18% of the sentiments). The Pearson chi-squared test indicates that this model has high goodness-of-fit. This study provides clear insights for public and corporate policymakers on potential areas to allocate resources, and directional guidance on potential policy options they can undertake to improve socioeconomic conditions, to mitigate the impact of pandemic in the current situation, and in the future as well.

show abstract

Section: Data and Study Methodsmentioning

confidence: 99%

Socioeconomic factors analysis for COVID-19 US reopening sentiment with Twitter and census data

Rahman

Ali

et al. 2021

Heliyon

View full text Add to dashboard Cite

show abstract

“…Stemming converts words to their base form (stem) [56]. The same word appears in different forms in the texts that are consolidated into the same features [57]. For example, the semantic meaning of read and reading is the same, thus they are combined as read to avoid any confusion.…”

Section: Study Methodsmentioning

confidence: 99%

Twitter and Census Data Analytics to Explore Socioeconomic Factors for Post-COVID-19 Reopening Sentiment

Rahman

Ali

et al. 2020

SSRN Journal

View full text Add to dashboard Cite

Investigating and classifying sentiments of social media users (e.g., positive, negative) towards an item, situation, and system are very popular among the researchers. However, they rarely discuss the underlying socioeconomic factor associations for such sentiments. This study attempts to explore the factors associated with positive and negative sentiments of the people about reopening the economy, in the United States (US) amidst the COVID-19 global crisis. It takes into consideration the situational uncertainties (i.e., changes in work and travel pattern due to lockdown policies), economic downturn and associated trauma, and emotional factors such as depression. To understand the sentiment of the people about the reopening economy, Twitter data was collected, representing the 51 states including Washington DC of the US. Statewide socioeconomic characteristics of the people (e.g., education, income, family size, and employment status), built environment data (e.g., population density), and the number of COVID-19 related cases were collected and integrated with Twitter data to perform the analysis. A binary logit model was used to identify the factors that influence people toward a positive or negative sentiment. The results from the logit model demonstrate that family households, people with low education levels, people in the labor force, low-income people, and people with higher house rent are more interested in reopening the economy. In contrast, households with a high number of members and high income are less interested to reopen the economy. The accuracy of the model is good (i.e., the model can correctly classify 56.18% of the sentiments). The Pearson chi2 test indicates that overall this model has high goodness-of-fit. This study provides a clear indication to the policymakers where to allocate resources and what policy options they can undertake to improve the socioeconomic situations of the people and mitigate the impacts of pandemics in the current situation and as well as in the future.

show abstract

“…Many algorithms, techniques, and methods have addressed this problem in NLP [49]. Many techniques and methods are available for researchers including hashing-based and context-sensitive spelling correction techniques [50], as well as spelling correction using Trie and Damerau-Levenshtein distance bigram [51].…”

Section: Spelling Correctionmentioning

confidence: 99%

Text Classification Algorithms: A Survey

et al. 2019

View full text Add to dashboard Cite

In recent years, there has been an exponential growth in the number of complex documents and texts that require a deeper understanding of machine learning methods to be able to accurately classify texts in many applications. Many machine learning approaches have achieved surpassing results in natural language processing. The success of these learning algorithms relies on their capacity to understand complex models and non-linear relationships within data. However, finding suitable structures, architectures, and techniques for text classification is a challenge for researchers. In this paper, a brief overview of text classification algorithms is discussed. This overview covers different text feature extractions, dimensionality reduction methods, existing algorithms and techniques, and evaluations methods. Finally, the limitations of each technique and their application in real-world problems are discussed.Spelling correction is an optional pre-processing step. Typos (short for typographical errors) are commonly present in texts and documents, especially in social media text data sets (e.g., Twitter). Many algorithms, techniques, and methods have addressed this problem in NLP [49]. Many techniques and methods are available for researchers including hashing-based and context-sensitive spelling correction techniques [50], as well as spelling correction using Trie and Damerau-Levenshtein distance bigram [51]. StemmingIn NLP, one word could appear in different forms (i.e., singular and plural noun form) while the semantic meaning of each form is the same [52]. One method for consolidating different forms of a word into the same feature space is stemming. Text stemming modifies words to obtain variant word forms using different linguistic processes such as affixation (addition of affixes) [53,54]. For example, the stem of the word "studying" is "study". LemmatizationLemmatization is a NLP process that replaces the suffix of a word with a different one or removes the suffix of a word completely to get the basic word form (lemma) [54][55][56]. Syntactic Word RepresentationMany researchers have worked on this text feature extraction technique to solve the loosing syntactic and semantic relation between words. Many researchers addressed novel techniques for solving this problem, but many of these techniques still have limitations. In [57], a model was introduced in which the usefulness of including syntactic and semantic knowledge in the text representation for the selection of sentences comes from technical genomic texts. The other solution for syntactic problem is using the n-gram technique for feature extraction. N-GramThe n-gram technique is a set of n-word which occurs "in that order" in a text set. This is not a representation of a text, but it could be used as a feature to represent a text.BOW is a representation of a text using its words (1-gram) which loses their order (syntactic). This model is very easy to obtain and the text can be represented through a vector, generally of a manageable size of the text. On the ...

show abstract

Fast and Accurate Spelling Correction Using Trie and Bigram

Cited by 6 publications

References 0 publications

Socioeconomic factors analysis for COVID-19 US reopening sentiment with Twitter and census data

Socioeconomic factors analysis for COVID-19 US reopening sentiment with Twitter and census data

Twitter and Census Data Analytics to Explore Socioeconomic Factors for Post-COVID-19 Reopening Sentiment

Text Classification Algorithms: A Survey

Contact Info

Product

Resources

About