Previous studies in multimodal sentiment analysis have used limited datasets, which only contain unified multimodal annotations. However, the unified annotations do not always reflect the independent sentiment of single modalities and limit the model to capture the difference between modalities. In this paper, we introduce a Chinese single-and multimodal sentiment analysis dataset, CH-SIMS, which contains 2,281 refined video segments in the wild with both multimodal and independent unimodal annotations. It allows researchers to study the interaction between modalities or use independent unimodal annotations for unimodal sentiment analysis. Furthermore, we propose a multi-task learning framework based on late fusion as the baseline. Extensive experiments on the CH-SIMS show that our methods achieve state-of-the-art performance and learn more distinctive unimodal representations. The full dataset and codes are available for use at https://github.com/ thuiar/MMSA.
Image-text matching task aims to learn the fine-grained correspondences between images and sentences. Existing methods use attention mechanism to learn the correspondences by attending to all fragments without considering the relationship between fragments and global semantics, which inevitably lead to semantic misalignment among irrelevant fragments. To this end, we propose a Bidirectional Correct Attention Network (BCAN), which leverages global similarities and local similarities to reassign the attention weight, to avoid such semantic misalignment. Specifically, we introduce a global correct unit to correct the attention focused on relevant fragments in irrelevant semantics. A local correct unit is used to correct the attention focused on irrelevant fragments in relevant semantics. Experiments on Flickr30K and MSCOCO datasets verify the effectiveness of our proposed BCAN by outperforming both previous attention-based methods and state-of-the-art methods. Code can be found at: https://github.com/liuyyy111/BCAN.
To predict the epidermal growth factor receptor (EGFR) mutation status in patients with lung adenocarcinoma using quantitative radiomic biomarkers and semantic features. We analyzed the computed tomography (CT) images and medical record data of 104 patients with lung adenocarcinoma who underwent surgical excision and EGFR mutation detection from 2016 to 2018 at our center. CT radiomic and semantic features that reflect the tumors’ heterogeneity and phenotype were extracted from preoperative non-enhanced CT scans. The least absolute shrinkage and selection operator method was applied to select the most distinguishable features. Three logistic regression models were built to predict the EGFR mutation status by combining the CT semantic with clinicopathological characteristics, using the radiomic features alone, and by combining the radiomic and clinicopathological features. Receiver operating characteristic (ROC) curve analysis was performed using five-fold cross-validation and the mean area under the curve (AUC) values were calculated and compared between the models to obtain the optimal model for predicting EGFR mutation. Furthermore, radiomic nomograms were constructed to demonstrate the performance of the model. In total, 1025 radiomic features were extracted and reduced to 13 features as the most important predictors to build the radiomic signature. The combined radiomic and clinicopathological features model was developed based on the radiomic signature, sex, smoking, vascular infiltration, and pathohistological type. The AUC was 0.90 ± 0.02 for the training, 0.88 ± 0.11 for the verification, and 0.894 for the test dataset. This model was superior to the other prediction models that used the combined CT semantic and clinicopathological features (AUC for the test dataset: 0.768) and radiomic features alone (AUC for the test dataset: 0.837). The prediction model built by radiomic biomarkers and clinicopathological features, including the radiomic signature, sex, smoking, vascular infiltration, and pathological type, outperformed the other two models and could effectively predict the EGFR mutation status in patients with peripheral lung adenocarcinoma. The radiomic nomogram of this model is expected to become an effective biomarker for patients with lung adenocarcinoma requiring adjuvant targeted treatment.
Highlights It is vital to distinguish indolent pulmonary adenocarcinomas from invasive pulmonary adenocarcinomas before surgery. Radiomics is a cutting-edge technology that mines quantitative features from CT images. We designed a nomogram, which incorporated clinical and CT morphological characteristics with the radiomics signature. We applied the radiomics nomogram to preoperatively predict the invasiveness of GGNs.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.