An introduction to Deep Learning in Natural Language Processing: Models, techniques, and tools

Lauriola, Ivano; Lavelli, Alberto; Aiolli, Fabio

doi:10.1016/j.neucom.2021.05.103

Cited by 250 publications

(89 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…However, these methods are relatively simple and exhibit poor fitting abilities and difficulty in excavating deep information and complex nonlinear relationships for large-scale datasets. Over the past decade, deep learning has made breakthroughs in computer vision and natural language processing due to its powerful representation learning capacity and excellent fitting ability [ 18 ]. Data-driven deep learning was proposed to learn abstract features automatically instead of manually designed or specified feature extraction [ 19 ], and to avoid the complexity, accuracy limitations, and poor stability caused by the manual feature design.…”

Section: Introductionmentioning

confidence: 99%

Raman Spectroscopy and Improved Inception Network for Determination of FHB-Infected Wheat Kernels

Qiu

Zheng

Tang

et al. 2022

Foods

View full text Add to dashboard Cite

Detection of infected kernels is important for Fusarium head blight (FHB) prevention and product quality assurance in wheat. In this study, Raman spectroscopy (RS) and deep learning networks were used for the determination of FHB-infected wheat kernels. First, the RS spectra of healthy, mild, and severe infection kernels were measured and spectral changes and band attribution were analyzed. Then, the Inception network was improved by residual and channel attention modules to develop the recognition models of FHB infection. The Inception–attention network produced the best determination with accuracies in training set, validation set, and prediction set of 97.13%, 91.49%, and 93.62%, among all models. The average feature map of the channel clarified the important information in feature extraction, itself required to clarify the decision-making strategy. Overall, RS and the Inception–attention network provide a noninvasive, rapid, and accurate determination of FHB-infected wheat kernels and are expected to be applied to other pathogens or diseases in various crops.

show abstract

Section: Introductionmentioning

confidence: 99%

Raman Spectroscopy and Improved Inception Network for Determination of FHB-Infected Wheat Kernels

Qiu

Zheng

Tang

et al. 2022

Foods

View full text Add to dashboard Cite

show abstract

“…Using its Tunisian Arabic version in a mental health application is a good option as it has proven its proficiency for many years. The ML model, bidirectional encoder representations from transformers (BERT) [ 4 ], is also a great NLP tool [ 5 , 6 ].…”

Section: Introductionmentioning

confidence: 99%

Mental Health Intent Recognition for Arabic-Speaking Patients Using the Mini International Neuropsychiatric Interview (MINI) and BERT Model

Mezzi

Yahyaoui

Krir

et al. 2022

Sensors

View full text Add to dashboard Cite

For many years, mental health has been hidden behind a veil of shame and prejudice. In 2017, studies claimed that 10.7% of the global population suffered from mental health disorders. Recently, people started seeking relaxing treatment through technology, which enhanced and expanded mental health care, especially during the COVID-19 pandemic, where the use of mental health forums, websites, and applications has increased by 95%. However, these solutions still have many limits, as existing mental health technologies are not meant for everyone. In this work, an up-to-date literature review on state-of-the-art of mental health and healthcare solutions is provided. Then, we focus on Arab-speaking patients and propose an intelligent tool for mental health intent recognition. The proposed system uses the concepts of intent recognition to make mental health diagnoses based on a bidirectional encoder representations from transformers (BERT) model and the International Neuropsychiatric Interview (MINI). Experiments are conducted using a dataset collected at the Military Hospital of Tunis in Tunisia. Results show excellent performance of the proposed system (the accuracy is over 92%, the precision, recall, and F1 scores are over 94%) in mental health patient diagnosis for five aspects (depression, suicidality, panic disorder, social phobia, and adjustment disorder). In addition, the tool was tested and evaluated by medical staff at the Military Hospital of Tunis, who found it very interesting to help decision-making and prioritizing patient appointment scheduling, especially with a high number of treated patients every day.

show abstract

“…Recently, researchers have attracted significant attention to Deep Learning (DL) [11][12][13][14] owing to its numerous applications in speech processing [15], natural language processing [16], and CV [17,18]. In video recognition [19] and large-scale images, a model of DL so-called convolutional neural network (CNN) has lately attained several encouraging results.…”

Section: Introductionmentioning

confidence: 99%

Face Recognition Based on Deep Learning and FPGA for Ethnicity Identification

et al. 2022

View full text Add to dashboard Cite

In the last decade, there has been a surge of interest in addressing complex Computer Vision (CV) problems in the field of face recognition (FR). In particular, one of the most difficult ones is based on the accurate determination of the ethnicity of mankind. In this regard, a new classification method using Machine Learning (ML) tools is proposed in this paper. Specifically, a new Deep Learning (DL) approach based on a Deep Convolutional Neural Network (DCNN) model is developed, which outperforms a reliable determination of the ethnicity of people based on their facial features. However, it is necessary to make use of specialized high-performance computing (HPC) hardware to build a workable DCNN-based FR system due to the low computation power given by the current central processing units (CPUs). Recently, the latter approach has increased the efficiency of the network in terms of power usage and execution time. Then, the usage of field-programmable gate arrays (FPGAs) was considered in this work. The performance of the new DCNN-based FR method using FPGA was compared against that using graphics processing units (GPUs). The experimental results considered an image dataset composed of 3141 photographs of citizens from three distinct countries. To our knowledge, this is the first image collection gathered specifically to address the ethnicity identification problem. Additionally, the ethnicity dataset was made publicly available as a novel contribution to this work. Finally, the experimental results proved the high performance provided by the proposed DCNN model using FPGAs, achieving an accuracy level of 96.9 percent and an F1 score of 94.6 percent while using a reasonable amount of energy and hardware resources.

show abstract

An introduction to Deep Learning in Natural Language Processing: Models, techniques, and tools

Cited by 250 publications

References 20 publications

Raman Spectroscopy and Improved Inception Network for Determination of FHB-Infected Wheat Kernels

Raman Spectroscopy and Improved Inception Network for Determination of FHB-Infected Wheat Kernels

Mental Health Intent Recognition for Arabic-Speaking Patients Using the Mini International Neuropsychiatric Interview (MINI) and BERT Model

Face Recognition Based on Deep Learning and FPGA for Ethnicity Identification

Contact Info

Product

Resources

About