Pipilika N-Gram Viewer: An Efficient Large Scale N-Gram Model for Bengali

Ahmad, Adnan; Talha, Mahbubur Rub; Amin, Md. Ruhul; Chowdhury, Farida

doi:10.1109/icbslp.2018.8554474

Cited by 7 publications

(5 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Different studies recommend the number of words to use in a keyphrase (Ahmad et al, 2018; Gledec et al, 2019; Zammit et al, 2020). In this research, we wanted to understand how the size of a keyphrase impacts accuracy and performance.…”

Section: Proposed Solutionmentioning

confidence: 99%

“…In text mining such keyphrases are also referred to as N‐grams (Ribeiro et al, 2017) and various research has been done to explore their use, including for non‐English languages (Ahmad et al, 2018; Gledec et al, 2019). A possible solution for an effective keyphrase assignment framework is to gather data and train a classification algorithm, but some authors outlined this as a disadvantage since it has a dependency on external documents (Gledec et al, 2019) and training requires time and effort.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Reducing the dependency of having prior domain knowledge for effective online information retrieval

et al. 2022

View full text Add to dashboard Cite

Sometimes Internet users struggle to find what they are looking for on the Internet due to information overload. Search engines intend to identify documents related to a given keyphrase on the Internet and provide suggestions. Having some background knowledge about a topic or a domain will help in building effective search keyphrases that will lead to accurate results in information retrieval. This is further pronounced among students that rely on the internet to learn about a new topic. Students might not have the required background knowledge to build effective keyphrases and find what they are looking for. In this research, we are addressing this problem, and aim to help students find relevant information online. This research furthers existing literature by enhancing information retrieval frameworks through keyphrase assignment, aiming to expose students to new terminologies, therefore reducing the dependency of having background knowledge about the domain under study. We evaluated this framework and identified how it can be enhanced to suggest more effective search keyphrases. Our proposed suggestion is to introduce a keyphrase Ranking Mechanism that will improve the keyphrase assignment part of the framework by taking into consideration the part‐of‐speech of the generated keyphrases. To evaluate the proposed approach, various data sets were downloaded and processed. The results obtained showed that our proposed approach produces more effective keyphrases than the existing framework.

show abstract

Section: Proposed Solutionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Reducing the dependency of having prior domain knowledge for effective online information retrieval

et al. 2022

View full text Add to dashboard Cite

show abstract

“…There are multiple versions of ngram: 1-gram (unigram),2-gram (bigram),3-gram (trigram),etc. [15] Discussion and Analysis:…”

Section: N-grammentioning

confidence: 99%

Highlighting the importance of semantics in COVID-19 fake news detection using a CNN hashmap color-based method

Azhari

2022

Preprint

View full text Add to dashboard Cite

With the exponential development and exploitation of social media sites and platforms such as face- book, twitter and instagram, a diversity type of news are reached to the users,resulting in a major influence on human health and safety.Spreading misinformation and disinformation during the Covid- 19 pandemic has become increasingly significant. Although it is usually not a criminal act, it can cause serious endangerment to public health. Such infodemic movement is often lead to advance geopolitical interests by the states, to achieve some sort of profit by some opportunists and indi- viduals or discredit official sources. Hence,it has become crucial to automate the detection of fake news in order to shield people from any harmful repercussions. In this paper, the importance of semantics in Covid-19 fake news detection is highlighted based on a convolutional neural network classifier and a hashmap color-based technique. The experiments are performed with CoAID(Covid- 19 heAlthcare mIsinformation Dataset),and the results prove that the loss of semantics yields to a poor performance of the classifier. This implicates additional constraints to the training images,with focus on creating a CNN-based color hashmap classifier that includes anterior and posterior neighbors.

show abstract

“…The out-of-vocabulary (OOV) words, that are not included in the speech/text corpus, are handled by our lexicon-free acoustic model without any modification. This text corpus is built by web scrapping open-source websites -mainly Bangla newspaper portals [17] and Wikipedia pages. We use Kenlm language modeling toolkit to build the LM [18].…”

Section: Text Corpusmentioning

confidence: 99%

Deep learning based large vocabulary continuous speech recognition of an under-resourced language Bangladeshi Bangla

Samin

Kobir

Kibria

et al. 2021

Acoust. Sci. & Tech.

View full text Add to dashboard Cite

Research in corpus-driven Automatic Speech Recognition (ASR) is advancing rapidly towards building a robust Large Vocabulary Continuous Speech Recognition (LVCSR) system. Under-resourced languages like Bangla require benchmarking large corpora for more research on LVCSR to tackle their limitations and avoid the biased results. In this paper, a publicly published large-scale Bangladeshi Bangla speech corpus is used to implement deep Convolutional Neural Network (CNN) based model and Recurrent Neural Network (RNN) based model with Connectionist Temporal Classification (CTC) loss function for Bangla LVCSR. In experimental evaluations, we find that CNN-based architecture yields superior results over the RNN-based approach. This study also emphasizes assessing the quality of an open-source large-scale Bangladeshi Bangla speech corpus and investigating the effect of the various high-order N-gram Language Models (LM) on a morphologically rich language Bangla. We achieve 36.12% word error rate (WER) using CNN-based acoustic model and 13.93% WER using beam search decoding with 5-gram LM. The findings demonstrate by far the state-of-the-art performance of any Bangla LVCSR system on a specific benchmarked large corpus.

show abstract

Pipilika N-Gram Viewer: An Efficient Large Scale N-Gram Model for Bengali

Cited by 7 publications

References 4 publications

Reducing the dependency of having prior domain knowledge for effective online information retrieval

Reducing the dependency of having prior domain knowledge for effective online information retrieval

Highlighting the importance of semantics in COVID-19 fake news detection using a CNN hashmap color-based method

Deep learning based large vocabulary continuous speech recognition of an under-resourced language Bangladeshi Bangla

Contact Info

Product

Resources

About