A Novel Attention-Based Multi-Modal Modeling Technique on Mixed Type Data for Improving TFT-LCD Repair Process

Yi, Liu; Lu, Hsueh-Ping; Lai, Cheng-Chou

doi:10.1109/access.2022.3158952

Cited by 3 publications

(2 citation statements)

References 62 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Multimodal machine learning is an advanced method that combines data from several sources or modalities (text, pictures, audio, sensor data, etc.) to improve the prediction power and robustness of machine learning models [26][27][28][29]. This paradigm has received a lot of interest in a variety of fields ranging from computer vision and natural A CNN is a deep learning architecture designed primarily for tasks involving grid-like data such as images and, more recently, sequential data like text and speech [25].…”

Section: One2mfusionmentioning

confidence: 99%

A New Approach for Multimodal Usage of Gene Expression and Its Image Representation for the Detection of Alzheimer’s Disease

Akkaya,

Kalkan

2023

Biomolecules

View full text Add to dashboard Cite

Alzheimer’s disease (AD) is a complex neurodegenerative disorder and the multifaceted nature of it requires innovative approaches that integrate various data modalities to enhance its detection. However, due to the cost of collecting multimodal data, multimodal datasets suffer from an insufficient number of samples. To mitigate the impact of a limited sample size on classification, we introduce a novel deep learning method (One2MFusion) which combines gene expression data with their corresponding 2D representation as a new modality. The gene vectors were first mapped to a discriminative 2D image for training a convolutional neural network (CNN). In parallel, the gene sequences were used to train a feed forward neural network (FNN) and the outputs of the FNN and CNN were merged, and a joint deep network was trained for the binary classification of AD, normal control (NC), and mild cognitive impairment (MCI) samples. The fusion of the gene expression data and gene-originated 2D image increased the accuracy (area under the curve) from 0.86 (obtained using a 2D image) to 0.91 for AD vs. NC and from 0.76 (obtained using a 2D image) to 0.88 for MCI vs. NC. The results show that representing gene expression data in another discriminative form increases the classification accuracy when fused with base data.

show abstract

Section: One2mfusionmentioning

confidence: 99%

A New Approach for Multimodal Usage of Gene Expression and Its Image Representation for the Detection of Alzheimer’s Disease

Akkaya,

Kalkan

2023

Biomolecules

View full text Add to dashboard Cite

show abstract

“…At last, machine learning has also been employed in medicine, via automatic thermography examination for breast cancer using the selective reflection of cholesteric liquid crystals [26]. A rather different application of machine learning in relation to liquid crystals can be found in industrial quality control during the production of TFT-LCD substrates [27][28][29].…”

Section: Introduction 11 Background and Motivationmentioning

confidence: 99%

Testing different supervised machine learning architectures for the classification of liquid crystals

et al. 2023

View full text Add to dashboard Cite

Different convolutional neural network (CNN) and inception network architectures were trained for the classification of isotropic, nematic, cholesteric and smectic liquid crystal phase textures to test the prediction accuracy for each one of these models. Varying the number of layers and inception blocks, as well as the regularization, and application to different phase transitions and classification tasks, it is shown that in general the architecture of an inception network with two blocks leads to the best classification results. Regularization, such as image flipping, and dropout layers additionally somewhat increases the classification accuracy. Even for simple tasks like the isotropic-nematic transition, which is of importance for applications in the automatic readout of sensors, convolutional neural networks need more than one layer. Care must be taken to not apply architectures of too large complexity, as this will again reduce the classification accuracy due to overfitting. Architecture complexity needs to be adjusted to the given classification task.

show abstract

Explainable Multi-Modal Deep Learning With Cross-Modal Attention for Diagnosis of Dyssynergic Defecation Using Abdominal X-Ray Images and Symptom Questionnaire

Sangnark,

Rattanachaisit,

Patcharatrakul

et al. 2024

IEEE Access

View full text Add to dashboard Cite

A Novel Attention-Based Multi-Modal Modeling Technique on Mixed Type Data for Improving TFT-LCD Repair Process

Cited by 3 publications

References 62 publications

A New Approach for Multimodal Usage of Gene Expression and Its Image Representation for the Detection of Alzheimer’s Disease

A New Approach for Multimodal Usage of Gene Expression and Its Image Representation for the Detection of Alzheimer’s Disease

Testing different supervised machine learning architectures for the classification of liquid crystals

Explainable Multi-Modal Deep Learning With Cross-Modal Attention for Diagnosis of Dyssynergic Defecation Using Abdominal X-Ray Images and Symptom Questionnaire

Contact Info

Product

Resources

About