Multimodal Fusion of Satellite Images and Crowdsourced GPS Traces for Robust Road Attribute Detection

Yin, Yifang; Tran, An C.; Zhang, Ying; Hu, Wenmiao; Wang, Guanfeng; Varadarajan, Jagannadan; Zimmermann, Roger; Ng, See-Kiong

doi:10.1145/3474717.3483917

Cited by 5 publications

(3 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…From 8 articles in the image domain, 2 were in analysis [339], [340], translation [341], [342] and caption generating [343], [344], and 1 was in retrieval [345] and description generating [346]. Of 8 papers in the geoscience domain, 5 were in classification [347]- [351], 1 was in recognition [352], detection [353], and precision [354]. In total 10 articles were identified in the social media domain, where 4 were in detection [355]- [358], 2 were in analysis [359], [360] and 1 was in classification [361], prediction [362], recommendation [363] and verification [364].…”

Section: Inclusion Criteriamentioning

confidence: 99%

“…[98], [100], [104], [116], [125], [127], [187], [206], [241], [326], [365], [395], [411], Image & Numerical [62], [75], [119], [126], [167], [313], [331], [353], [405], [410], Audio & Text & Sensor [384], Audio & Text [180], [282], [377], [391], [392], Text & Signal [109], Text & Numerical [304], [349], Sensor & Signal [240], [242], [258], [389], Sensor & Numerical [183], Signal & Numerical [205], [257], [260], [318]. Figure 10 displays the extracted information related to each modality and data type with the links between them.…”

Section: B Taskmentioning

confidence: 99%

See 1 more Smart Citation

A Systematic Literature Review on Multimodal Machine Learning: Applications, Challenges, Gaps and Future Directions

2023

View full text Add to dashboard Cite

Multimodal machine learning (MML) is a tempting multidisciplinary research area where heterogeneous data from multiple modalities and machine learning (ML) are combined to solve critical problems. Usually, research works use data from a single modality, such as images, audio, text, and signals. However, real-world issues have become critical now, and handling them using multiple modalities of data instead of a single modality can significantly impact finding solutions. ML algorithms play an essential role by tuning parameters in developing MML models. This paper reviews recent advancements in the challenges of MML, namely: representation, translation, alignment, fusion and co-learning, and presents the gaps and challenges. A systematic literature review (SLR) applied to define the progress and trends on those challenges in the MML domain. In total, 1032 articles were examined in this review to extract features like source, domain, application, modality, etc. This research article will help researchers understand the constant state of MML and navigate the selection of future research directions.

show abstract

Section: Inclusion Criteriamentioning

confidence: 99%

Section: B Taskmentioning

confidence: 99%

A Systematic Literature Review on Multimodal Machine Learning: Applications, Challenges, Gaps and Future Directions

2023

View full text Add to dashboard Cite

show abstract

“…Because of its unique prospective and large ground coverage, satellite imagery has been used in different geo-information extraction applications [7,8,11,18,53,54,59]. In recent years, satellite imagery-based cross-view matching has become one of the mainstream approaches for street-view imagery geo-localization [9,16,17,25,30,32,35,36,38,40,42,44,45,48,50,52,57,60,61].…”

Section: Related Workmentioning

confidence: 99%

Beyond Geo-localization: Fine-grained Orientation of Street-view Images by Cross-view Matching with Satellite Imagery

Zhang

Liang

et al. 2022

Proceedings of the 30th ACM International Conference on Multimedia

Self Cite

View full text Add to dashboard Cite

Street-view imagery provides us with novel experiences to explore different places remotely. Carefully calibrated street-view images (e.g., Google Street View) can be used for different downstream tasks, e.g., navigation, map features extraction. As personal highquality cameras have become much more affordable and portable, an enormous amount of crowdsourced street-view images are uploaded to the internet, but commonly with missing or noisy sensor information. To prepare this hidden treasure for "ready-to-use" status, determining missing location information and camera orientation angles are two equally important tasks. Recent methods have achieved high performance on geo-localization of street-view images by cross-view matching with a pool of geo-referenced satellite imagery. However, most of the existing works focus more on geo-localization than estimating the image orientation. In this work, we re-state the importance of finding fine-grained orientation for street-view images, formally define the problem and provide a set of evaluation metrics to assess the quality of the orientation estimation. We propose two methods to improve the granularity of the orientation estimation, achieving 82.4% and 72.3% accuracy for images with estimated angle errors below 2 • for CVUSA and CVACT datasets, corresponding to 34.9% and 28.2% absolute improvement compared to previous works. Integrating fine-grained orientation estimation in training also improves the performance on geo-localization, giving top 1 recall 95.5%/85.5% and 86.8%/80.4% for orientation known/unknown tests on the two datasets.

show abstract

A Context-enriched Satellite Imagery Dataset and an Approach for Parking Lot Detection

Yin

Tran³

et al. 2022

2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)

View full text Add to dashboard Cite

Multimodal Fusion of Satellite Images and Crowdsourced GPS Traces for Robust Road Attribute Detection

Cited by 5 publications

References 15 publications

A Systematic Literature Review on Multimodal Machine Learning: Applications, Challenges, Gaps and Future Directions

A Systematic Literature Review on Multimodal Machine Learning: Applications, Challenges, Gaps and Future Directions

Beyond Geo-localization: Fine-grained Orientation of Street-view Images by Cross-view Matching with Satellite Imagery

A Context-enriched Satellite Imagery Dataset and an Approach for Parking Lot Detection

Contact Info

Product

Resources

About