Event Prediction Based on Unsupervised Graph-Based Rank-Fusion Models

Dourado, Ícaro Cavalcante; Tabbone, Salvatore; Torres, Ricardo da S.

doi:10.1007/978-3-030-20081-7_9

Cited by 5 publications

(8 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Finally, in run 3, the models using only neural network solutions were the best performing, being above other baselines by a significant margin. Both previously published results [5,7] improve slightly in the multimodal scenario compared to the ones where only images or text are available. The MLP showed the largest improvement in the use of fastText embeddings, while the RN had very similar performance both with and without fastText, as was the case in run 2.…”

Section: Resultsmentioning

confidence: 73%

“…While we are focused on the performance of the RN for learning from text data (run 2 ), we still consider the performance of the CNN on the image-only scenario (run 1 ), thus we can compare how the CNN performs by itself and when used in the multimodal scenario (run 3 ). Even though our CNN based on ResNet-18 has almost 4 times fewer parameters than one of the baselines [4], while the other [7] uses multiple deep neural networks as feature extractors, all approaches have similar performance, with the baselines being slightly better.…”

Section: Resultsmentioning

confidence: 99%

“…In all cases, ResNet-18 was initialized from weights of a pre-trained (ImageNet) model. We use the best result by Dourado et al [7] of their proposed variants in each scenario.…”

Section: Resultsmentioning

confidence: 99%

“…More recently, Dourado et al [7] proposed the use of multiple descriptors instead of relying on a single representation for objects, producing one similarity-based ranking per object. Then, those rankings are aggregated using graphs capturing the relationships to other objects based on the different rankings, resulting in one graph for each object.…”

Section: Related Workmentioning

confidence: 99%

“…We compare our method to three baselines. The first two, proposed by Bischke et al [4] and Dourado et al [7], described earlier, use a SVM with the RBF kernel as a classifier to produce scores for ranking. We also compare to a fullyconnected neural network with multiple layers (MLP), with no relational step, using the mean vector of all words as input, to highlight the influence of the relational step in our approach.…”

Section: Baselinesmentioning

confidence: 99%

See 4 more Smart Citations

Neural relational inference for disaster multimedia retrieval

Fadel

Torres

2020

Multimed Tools Appl

View full text Add to dashboard Cite

Events around the world are increasingly documented on social media, specially by the people experiencing them, as these platforms become more popular over time. As a consequence, social media turns into a valuable source of data for understanding those events. Due to their destructive potential, natural disasters are among events of particular interest to response operations and environmental monitoring agencies. However, this amount of information also makes it challenging to identify relevant content pertaining to those events. In this paper, we use a relational neural network model for identifying this type of content. The model is particularly suitable for unstructured text, that is, text with no particular arrangement of words, such as tags, which is commonplace in social media data. In addition, our method can be combined with a CNN for handling multimodal data where text and visual data are available. We perform experiments in three different scenarios, where different modalities are evaluated: visual, textual, and both. Our method achieves competitive performance in both modalities by themselves, while significantly outperforms the baseline on the multimodal scenario. We also demonstrate the behavior of the proposed method in different applications by performing additional experiments in the CUB-200-2011 multimodal dataset.

show abstract

Section: Resultsmentioning

confidence: 73%

Section: Resultsmentioning

confidence: 99%

“…In all cases, ResNet-18 was initialized from weights of a pre-trained (ImageNet) model. We use the best result by Dourado et al [7] of their proposed variants in each scenario.…”

Section: Resultsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Baselinesmentioning

confidence: 99%

See 3 more Smart Citations

Neural relational inference for disaster multimedia retrieval

Fadel

Torres

2020

Multimed Tools Appl

View full text Add to dashboard Cite

show abstract

Learning Vocabularies to Embed Graphs in Multimodal Rank Aggregation Tasks

Dourado

Torres

2021

2021 International Conference on Content-Based Multimedia Indexing (CBMI)

View full text Add to dashboard Cite

VRBagged-Net: Ensemble Based Deep Learning Model for Disaster Event Classification

2021

View full text Add to dashboard Cite

A flood is an overflow of water that swamps dry land. The gravest effects of flooding are the loss of human life and economic losses. An early warning of these events can be very effective in minimizing the losses. Social media websites such as Twitter and Facebook are quite effective in the efficient dissemination of information pertinent to any emergency. Users on these social networking sites share both textual and rich content images and videos. The Multimedia Evaluation Benchmark (MediaEval) offers challenges in the form of shared tasks to develop and evaluate new algorithms, approaches and technologies for explorations and exploitations of multimedia in decision making for real time problems. Since 2015, the MediaEval has been running a shared task of predicting several aspects of flooding and through these shared tasks, many improvements have been observed. In this paper, the classification framework VRBagged-Net is proposed and implemented for flood classification. The framework utilizes the deep learning models Visual Geometry Group (VGG) and Residual Network (ResNet), along with the technique of Bootstrap aggregating (Bagging). Various disaster-based datasets were selected for the validation of the VRBagged-Net framework. All the datasets belong to the MediaEval Benchmark Workshop, this includes Disaster Image Retrieval from Social Media (DIRSM), Flood Classification for Social Multimedia (FCSM) and Image based News Topic Disambiguation (INTD). VRBagged-Net performed encouraging well in all these datasets with slightly different but relevant tasks. It produces Mean Average Precision at different levels of 98.12, and Average Precision at 480 of 93.64 on DIRSM. On the FCSM dataset, it produces an F1 score of 90.58. Moreover, the framework has been applied on the dataset of Image-Based News Topic Disambiguation (INTD), and exceeds the previous best result by producing an F1 evaluation of 93.76. The VRBagged-Net with a slight modification also ranked first in the flood-related Multimedia Task at the MediaEval Workshop 2020.

show abstract

Event Prediction Based on Unsupervised Graph-Based Rank-Fusion Models

Cited by 5 publications

References 10 publications

Neural relational inference for disaster multimedia retrieval

Neural relational inference for disaster multimedia retrieval

Learning Vocabularies to Embed Graphs in Multimodal Rank Aggregation Tasks

VRBagged-Net: Ensemble Based Deep Learning Model for Disaster Event Classification

Contact Info

Product

Resources

About