Attention-Enabled Ensemble Deep Learning Models and Their Validation for Depression Detection: A Domain Adoption Paradigm

Abstract

Depression is increasingly prevalent, leading to higher suicide risk. Depression detection and sentimental analysis of text inputs in cross-domain frameworks are challenging. Solo deep learning (SDL) and ensemble deep learning (EDL) models are not robust enough. Recently, attention mechanisms have been introduced in SDL. We hypothesize that attention-enabled EDL (aeEDL) architectures are superior compared to attention-not-enabled SDL (aneSDL) or aeSDL models. We designed EDL-based architectures with attention blocks to build eleven kinds of SDL model and five kinds of EDL model on four domain-specific datasets. We scientifically validated our models by comparing “seen” and “unseen” paradigms (SUP). We benchmarked our results against the SemEval (2016) sentimental dataset and established reliability tests. The mean increase in accuracy for EDL over their corresponding SDL components was 4.49%. Regarding the effect of attention block, the increase in the mean accuracy (AUC) of aeSDL over aneSDL was 2.58% (1.73%), and the increase in the mean accuracy (AUC) of aeEDL over aneEDL was 2.76% (2.80%). When comparing EDL vs. SDL for non-attention and attention, the mean aneEDL was greater than aneSDL by 4.82% (3.71%), and the mean aeEDL was greater than aeSDL by 5.06% (4.81%). For the benchmarking dataset (SemEval), the best-performing aeEDL model (ALBERT+BERT-BiLSTM) was superior to the best aeSDL (BERT-BiLSTM) model by 3.86%. Our scientific validation and robust design showed a difference of only 2.7% in SUP, thereby meeting the regulatory constraints. We validated all our hypotheses and further demonstrated that aeEDL is a very effective and generalized method for detecting symptoms of depression in cross-domain settings.

GeneAI 3.0: powerful, novel, generalized hybrid and ensemble deep learning frameworks for miRNA species classification of stationary patterns from nucleotides

“…The objective was to accurately differentiate between the two species using the provided dataset features. We created multiple datasets with the aim of achieving generalization [78][79][80][81] in species classification. The purpose behind this initiative was to train our model on a variety of datasets, ensuring its effectiveness in real-life scenarios.…”

“…The stability of the system was thoroughly assessed and validated using three statistical tests conducted on the EDL models across all ten testing sets. There are several published studies which uses statistical tests for establishing the reliability and stability of the AI system 80,81,140,141 . These tests are conducted on the employed models, and the specific tests we carried out are all showcased in the manuscript, namely Adjusted R2, Z (Two-Tailed), and ANOVA tests.…”

“…For generalizations, the models have to undergo training and testing on multiple datasets. Our group has done several methods for generalization [78][79][80][81] . In 81 , we developed an ensemble-based transfer learning paradigm, successfully classifying skin lesion images from two different and diverse datasets.…”

“…Study 79 We achieved generalization in these systems by simplifying the model, enabling it to work across multiple domains effectively in various situations through the mixing of domains. In Study 80 , we trained a model to be robust enough for depression detection as well as sentiment analysis by facilitating inter-dataset (cross-domain) training and leveraging knowledge from a multi-domain dataset. Our model demonstrated the capability to detect depression even when trained on a sentiment dataset, while also analysing sentiment when trained on a depression dataset.…”

GeneAI 3.0: Powerful, Novel, Generalized Hybrid and Ensemble Deep Learning Frameworks for miRNA Classification of species-specific Stationary Patterns from Nucleotides

“…They constructed an extensive feature database and trained it using LR and RF models. [135,136] can offer a more focused and streamlined classification of species, ultimately improving the accuracy and reliability of the gene classification scheme.…”