Entity Extraction for Malayalam Social Media Text Using Structured Skip-gram Based Embedding Features from Unlabeled Data

Devi, G. Remmiya; Veena, Pamalla; Kumar, M. Anand; Soman, K P

doi:10.1016/j.procs.2016.07.276

Cited by 23 publications

(7 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Jayan et al (2013) propose a hybrid approach that combines rules with supervised machine learning. Devi et al (2016) tackle named entity extraction from social media and combine supervised machine learning (SVMs) with skipgram features. Shruthi and Pranav (2016) propose another supervised approach based on the TnT tagger (Brants, 2002) and maximum entropy models.…”

Section: Related Workmentioning

confidence: 99%

“…It has featured in a limited number of NLP tasks, including morphological analysis (Bhavukam et al, 2018), POS tagging (Akhil et al, 2020) and NER (Ajees and Idicula, 2018). However, many studies use small locally generated data sets (Nambiar et al, 2019) or domain specific data sets (Kumar et al, 2019), (Devi et al, 2016), which usually are not freely available.…”

Section: Malayalammentioning

confidence: 99%

See 1 more Smart Citation

EmployingWikipedia as a Resource for Named Entity Recognition in Morphologically Complex Under-Resourced Languages

Krishnan¹,

Ziehe²,

Pannach³

et al. 2021

Proceedings of the 14thWorkshop on Building and Using Comparable Corpora (BUCC 2021)

View full text Add to dashboard Cite

We propose a novel approach for rapid prototyping of named entity recognisers through the development of semi-automatically annotated data sets. We demonstrate the proposed pipeline on two under-resourced agglutinating languages: the Dravidian language Malayalam and the Bantu language isiZulu. Our approach is weakly supervised and bootstraps training data from Wikipedia and Google Knowledge Graph. Moreover, our approach is relatively language independent and can consequently be ported quickly (and hence cost-effectively) from one language to another, requiring only minor language-specific tailoring.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Malayalammentioning

confidence: 99%

EmployingWikipedia as a Resource for Named Entity Recognition in Morphologically Complex Under-Resourced Languages

Krishnan¹,

Ziehe²,

Pannach³

et al. 2021

Proceedings of the 14thWorkshop on Building and Using Comparable Corpora (BUCC 2021)

View full text Add to dashboard Cite

show abstract

“…In this section, our proposed framework that uses DL for NER for Malayalam language is compared with some state-of-the-art approaches. We have chosen [35][36][37] as the baselines and evaluated the performance of our proposed DL-based approach with these approaches and found that the DL-based approach significantly outperforms the state-of-the-art approaches. The performance comparison is shown in Figure 5.…”

Section: Comparison With the State-of-the-art Malayalam Ner Approachesmentioning

confidence: 99%

A Framework for Named Entity Recognition for Malayalam—A Comparison of Different Deep Learning Architectures

Rajimol¹,

Anoop²

2020

NLPRE

View full text Add to dashboard Cite

Information extraction (IE) is the process of extracting relevant and useful patterns or information from unstructured data. Named entity recognition (NER) is a subtask of IE that identifies entities from unstructured text documents and organize them into different predefined categories such as person, location, organization, number, date, etc. NER is considered to be one of the important steps in natural language processing which may find direct applications in areas such as question answering (QA), entity linking, and co-reference resolution, to name a few. NER systems perform comparatively well in high-resource languages such as English but there is a lack of well-developed NER systems for low-resource languages such as Malayalam, which is an Indic language spoken in the state of Kerala, India. This work is an approach in this direction which makes use of deep learning (DL) techniques for developing a NER system for Malayalam. We have compared different DL approaches such as recurrent neural networks, gated recurrent unit, long short-term memory, and bi-directional long short-term memory and found that DL based approaches significantly outperform traditional shallow-learning based -approaches for NER. When compared with some state-of-the-art approaches our proposed framework is found to be outperforming in terms of precision, recall, and F-measure and could achieve an improved precision of 7.89% and 8.92% of F-measure.

show abstract

“…Vectors, pre-trained on a very large text [(1-10) ×10 6 ] corpus (one to ten million words), are readily available for almost all European languages and Asian languages such as Arabic, Chinese, Hindi, and Korean [13][14][15]. Morphologically rich languages like Slavic can also have equally effective vector representations [16][17][18][19][20].…”

Section: Introductionmentioning

confidence: 99%

Generation of Cross-Lingual Word Vectors for Low-Resourced Languages Using Deep Learning and Topological Metrics in a Data-Efficient Way

Menon²,

Rajendran

et al. 2021

Electronics

View full text Add to dashboard Cite

Linguists have been focused on a qualitative comparison of the semantics from different languages. Evaluation of the semantic interpretation among disparate language pairs like English and Tamil is an even more formidable task than for Slavic languages. The concept of word embedding in Natural Language Processing (NLP) has enabled a felicitous opportunity to quantify linguistic semantics. Multi-lingual tasks can be performed by projecting the word embeddings of one language onto the semantic space of the other. This research presents a suite of data-efficient deep learning approaches to deduce the transfer function from the embedding space of English to that of Tamil, deploying three popular embedding algorithms: Word2Vec, GloVe and FastText. A novel evaluation paradigm was devised for the generation of embeddings to assess their effectiveness, using the original embeddings as ground truths. Transferability across other target languages of the proposed model was assessed via pre-trained Word2Vec embeddings from Hindi and Chinese languages. We empirically prove that with a bilingual dictionary of a thousand words and a corresponding small monolingual target (Tamil) corpus, useful embeddings can be generated by transfer learning from a well-trained source (English) embedding. Furthermore, we demonstrate the usability of generated target embeddings in a few NLP use-case tasks, such as text summarization, part-of-speech (POS) tagging, and bilingual dictionary induction (BDI), bearing in mind that those are not the only possible applications.

show abstract

Entity Extraction for Malayalam Social Media Text Using Structured Skip-gram Based Embedding Features from Unlabeled Data

Cited by 23 publications

References 3 publications

EmployingWikipedia as a Resource for Named Entity Recognition in Morphologically Complex Under-Resourced Languages

EmployingWikipedia as a Resource for Named Entity Recognition in Morphologically Complex Under-Resourced Languages

A Framework for Named Entity Recognition for Malayalam—A Comparison of Different Deep Learning Architectures

Generation of Cross-Lingual Word Vectors for Low-Resourced Languages Using Deep Learning and Topological Metrics in a Data-Efficient Way

Contact Info

Product

Resources

About